powermove72/GK-inv-MoE-0.1
MoE built from dense models vs GritLM/GritLM-7B. powermove72/GK-inv-MoE-0.1 is a mixture of 2 experts assembled from dense models (mergekit-moe style): expert 0 is an exact copy of GritLM/GritLM-7B's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 195 of 195 identical.
1 derivatives checked
- mradermacher/GK-inv-MoE-0.1-GGUF: Quantization