Aratako/Mixtral-8x7B-Instruct-v0.1-upscaled
Partial fine-tune (fewer layers) vs mistralai/Mixtral-8x7B-Instruct-v0.1. mistralai/Mixtral-8x7B-Instruct-v0.1 matches part of Aratako/Mixtral-8x7B-Instruct-v0.1-upscaled (995 of its 1739 tensors; the rest of Aratako/Mixtral-8x7B-Instruct-v0.1-upscaled has no counterpart). On the shared part: Partial change: 75% of the weights changed (attention, mlp, norms, other), in 24 of 32 layers.
Tensors changed: Embeddings 0/1, MLP 576/768, other 24/32, Norms 48/65, Attention 96/128, lm_head 0/1
2 derivatives checked
- mradermacher/Mixtral-8x7B-Instruct-v0.1-upscaled-i1-GGUF: Not comparable yet
- mradermacher/Mixtral-8x7B-Instruct-v0.1-upscaled-GGUF: Not comparable yet