lipcut/shizhi-twilight-7B
MoE built from dense models vs MediaTek-Research/Breeze-7B-Instruct-v0_1. lipcut/shizhi-twilight-7B is a mixture of 2 experts assembled from dense models (mergekit-moe style): expert 1 is an exact copy of MediaTek-Research/Breeze-7B-Instruct-v0_1's MLP; the other experts come from other models. Shared tensors (attention, norms, embeddings): 195 of 195 identical.
2 derivatives checked
- mradermacher/shizhi-twilight-7B-i1-GGUF: Quantization
- mradermacher/shizhi-twilight-7B-GGUF: Quantization