allura-org/MoE-Girl-1BA-7BT

Full fine-tune vs allenai/OLMoE-1B-7B-0924. Fine-tune: 100% of the weights changed, across 16 of 16 layers.

Tensors changed: Embeddings 1/1, Norms 41/65, MLP 2869/2869, Attention 64/64, lm_head 1/1

3 derivatives checked