Xmaster6y/Mixtral-8x400M-v0.1
Partial fine-tune (fewer layers) vs mistralai/Mixtral-8x7B-v0.1. Xmaster6y/Mixtral-8x400M-v0.1 matches part of mistralai/Mixtral-8x7B-v0.1 (65 of its 995 tensors; the rest of mistralai/Mixtral-8x7B-v0.1 has no counterpart). On the shared part: Partial change: 46% of the weights changed (attention, mlp, norms, other), in 1 of 2 layers.
Tensors changed: Embeddings 0/1, MLP 24/48, other 1/2, Norms 2/5, Attention 4/8, lm_head 0/1
1 derivatives checked
- mradermacher/Mixtral-8x400M-v0.1-GGUF: Same weights, converted