BenevolenceMessiah/Yi-Coder-9B-Chat-8x-MoE
MoE built from dense models vs 01-ai/Yi-Coder-9B-Chat. BenevolenceMessiah/Yi-Coder-9B-Chat-8x-MoE is a mixture of 8 experts assembled from dense models (mergekit-moe style): experts 0, 1, 2, 3, 4, 5, 6, 7 are an exact copy of 01-ai/Yi-Coder-9B-Chat's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 291 of 291 identical.
2 derivatives checked
- mradermacher/Yi-Coder-9B-Chat-8x-MoE-i1-GGUF: Quantization
- mradermacher/Yi-Coder-9B-Chat-8x-MoE-GGUF: Quantization