wren11ws/sunup
Partial fine-tune vs distilbert/distilgpt2. Partial change: 65% of the weights changed (attention, mlp, norms), in 4 of 6 layers.
Tensors changed: Attention 16/24, Norms 18/26, MLP 16/24, Embeddings 0/1
Partial fine-tune vs distilbert/distilgpt2. Partial change: 65% of the weights changed (attention, mlp, norms), in 4 of 6 layers.
Tensors changed: Attention 16/24, Norms 18/26, MLP 16/24, Embeddings 0/1