DrRiceIO7/Franken-MoE-Large-Base
MoE built from dense models vs Qwen/Qwen3-0.6B. DrRiceIO7/Franken-MoE-Large-Base is a mixture of 16 experts assembled from dense models (mergekit-moe style): expert 0 is an exact copy of Qwen/Qwen3-0.6B's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 226 of 226 identical.
2 derivatives checked
- mradermacher/Franken-MoE-Large-Base-i1-GGUF: Quantization
- mradermacher/Franken-MoE-Large-Base-GGUF: Same weights, converted