learning-unit/L1-30B-A5B
Full fine-tune vs trillionlabs/Gravity-30B-A5B-Base. Fine-tune: 100% of the weights changed, across 52 of 52 layers.
Tensors changed: Embeddings 1/1, Norms 157/157, MLP 2633/2633, Attention 208/208, lm_head 1/1
Full fine-tune vs trillionlabs/Gravity-30B-A5B-Base. Fine-tune: 100% of the weights changed, across 52 of 52 layers.
Tensors changed: Embeddings 1/1, Norms 157/157, MLP 2633/2633, Attention 208/208, lm_head 1/1