sixtyoneeighty/FNCARL9000
MoE built from dense models vs NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story. sixtyoneeighty/FNCARL9000 is a mixture of 4 experts assembled from dense models (mergekit-moe style): expert 1 is an exact copy of NeuralNovel/Mistral-7B-Instruct-v0.2-Neural-Story's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 48 of 193 identical.
2 derivatives checked
- mradermacher/sixtyoneeighty-4x7B-v2-GGUF: Not comparable yet
- mradermacher/FNCARL9000-GGUF: Quantization