Supervised fine-tuning / instruction tuning
Supervised fine-tuning (SFT) trains a pretrained model on curated prompt-response pairs. For LLMs, instruction tuning is SFT that teaches the model how to behave as an assistant: follow requests, use the right format, and prefer useful response patterns.
What SFT changes
SFT does not create a new foundation model. It nudges an existing base or instruct model toward examples you provide.
| Training signal | Learns |
|---|---|
| Prompt and ideal response | How the task should be answered |
| Chat template | How roles and messages are formatted |
| Repeated style patterns | Tone, structure, refusal style, verbosity |
| Domain procedures | Stable workflows that should become default |
Dataset shape
Each row should be a clean demonstration of desired behavior:
- The input resembles real production prompts.
- The response is something you would be happy to ship.
- The format is consistent across examples.
- Edge cases and refusals are included, not only happy paths.
- Train, validation, and test examples are split before iteration.
What it does not solve
SFT does not reliably optimize preferences between two plausible answers; that is where DPO or RLHF-like methods fit. It also does not make missing knowledge appear unless that knowledge was already latent in the model or present in the prompt.
Pitfall
SFT faithfully learns your mess. Inconsistent labels, mixed formats, or mediocre answers teach the model to be inconsistent, mixed, and mediocre.
Connects to: base vs instruct · dataset construction · DPO