Spectral Surgery — Instruction Following
Collection
Instruction-following LoRA and Spectral Surgery adapters evaluated on IFEval. • 12 items • Updated
Llama-3.1-8B-Instruct + Instruction-Following LoRA
This repository contains the vanilla LoRA adapter used as the instruction-following SFT baseline for the Spectral Surgery experiments.
Base Model
meta-llama/Llama-3.1-8B-Instruct
Training
- Sequence length: 1024
- Global batch size: 128
- LoRA rank: 16
- Seed: 42
Evaluation
Evaluation on IFEval.
| Model | Prompt Strict | Prompt Loose | Instruction Strict | Instruction Loose |
|---|---|---|---|---|
| Base | 65.80% | 72.83% | 75.54% | 81.18% |
| LoRA SFT | 65.99% | 70.98% | 74.82% | 79.26% |
| Spectral Surgery HNS 8+2 | 69.50% | 73.94% | 77.94% | 82.13% |
The vanilla LoRA checkpoint is used as the source checkpoint for the Spectral Surgery HNS 8+2 adapter.
- Downloads last month
- 10
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for tianzl66/Llama-3.1-8B-Instruct-InstructionFollowing-LoRA
Base model
meta-llama/Llama-3.1-8B Finetuned
meta-llama/Llama-3.1-8B-Instruct