stamsam/Instella-Prometheus
LoRA-style vs amd/Instella-MoE-16B-A3B-SFT. Looks like a merged LoRA (or a fine-tune with frozen norms): only the weight matrices of gate_proj, kv_a_proj_with_mqa, kv_b_proj, o_proj changed, in 27 of 27 layers; norms, biases and embeddings are untouched.
Tensors changed: MLP 914/2781, Attention 108/135, Embeddings 0/1, Norms 0/82, lm_head 0/1