blascotobasco/Rover-A5B
MoE built from dense models vs Qwen/Qwen3-1.7B. blascotobasco/Rover-A5B is a mixture of 32 experts assembled from dense models (mergekit-moe style): experts 16, 17, 28, 29 are an exact copy of Qwen/Qwen3-1.7B's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 198 of 226 identical.
2 derivatives checked
- mradermacher/Rover-A5B-i1-GGUF: Quantization
- mradermacher/Rover-A5B-GGUF: Quantization