DrRiceIO7/Franken-MoE-Large-SFT-Preview
MoE built from dense models vs Qwen/Qwen3-0.6B. DrRiceIO7/Franken-MoE-Large-SFT-Preview is a mixture of 16 experts with the same shape as Qwen/Qwen3-0.6B's MLP, but none of them is a copy of it: they come from other models or were trained further. Shared tensors (attention, norms, embeddings): 114 of 226 identical.