jsfs11/RandomMergeNoNormWEIGHTED-7B-2x7BMOE
MoE built from dense models vs macadeliccc/WestLake-7B-v2-laser-truthy-dpo. jsfs11/RandomMergeNoNormWEIGHTED-7B-2x7BMOE is a mixture of 2 experts assembled from dense models (mergekit-moe style): expert 1 is an exact copy of macadeliccc/WestLake-7B-v2-laser-truthy-dpo's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 0 of 195 identical.
2 derivatives checked
- mradermacher/RandomMergeNoNormWEIGHTED-7B-2x7BMOE-i1-GGUF: Quantization
- mradermacher/RandomMergeNoNormWEIGHTED-7B-2x7BMOE-GGUF: Quantization