mergekit-community/uncensored-mix
Abliterated vs NousResearch/DeepHermes-3-Llama-3-8B-Preview. Looks like abliteration (refusal removal) done on a re-parametrized copy of the model: the output projections (down_proj, embed_tokens, o_proj) changed, in 32 of 32 layers, and the norm weights were re-scaled (x1.02 median) with the matrices after them (q/k/v, gate/up) shrinking by about the inverse, which changes no behavior by itself.
Tensors changed: Embeddings 1/1, Norms 0/65, MLP 32/96, Attention 32/128, lm_head 0/1
1 derivatives checked
- mradermacher/uncensored-mix-GGUF: Same weights, converted