estrogen/olmoe-upscale
Partial fine-tune (fewer layers) vs allenai/OLMoE-1B-7B-0125-Instruct. allenai/OLMoE-1B-7B-0125-Instruct matches part of estrogen/olmoe-upscale (3217 of its 4827 tensors; the rest of estrogen/olmoe-upscale has no counterpart). On the shared part: Partial change: 62% of the weights changed (attention, mlp, norms), in 10 of 16 layers.
Tensors changed: Norms 40/65, MLP 1163/1871, Attention 40/64
2 derivatives checked
- mradermacher/olmoe-upscale-i1-GGUF: Quantization
- mradermacher/olmoe-upscale-GGUF: Same weights, converted