ehristoforu/Falcon3-MoE-2x7B-Insruct
MoE built from dense models vs tiiuae/Falcon3-7B-Instruct. ehristoforu/Falcon3-MoE-2x7B-Insruct is a mixture of 2 experts assembled from dense models (mergekit-moe style): experts 0, 1 are an exact copy of tiiuae/Falcon3-7B-Instruct's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 171 of 171 identical.
2 derivatives checked
- mradermacher/Falcon3-MoE-2x7B-Insruct-i1-GGUF: Quantization
- mradermacher/Falcon3-MoE-2x7B-Insruct-GGUF: Quantization