trl-lib/Qwen2-0.5B-OnlineDPO
Full fine-tune vs Qwen/Qwen2-0.5B-Instruct. Fine-tune: 100% of the weights changed, across 24 of 24 layers.
Tensors changed: Embeddings 1/1, Norms 13/49, MLP 72/72, Attention 152/168
1 derivatives checked
- mradermacher/Qwen2-0.5B-OnlineDPO-GGUF: Same weights, converted