Assembling context: order & format
Once you've selected what to include (context management), how you lay it out measurably changes results. The model reads a flat token stream — structure and position are signals.
Order matters
- Edges beat the middle — put the most important instructions and context near the start or end (lost in the middle).
- Stable first — keep the unchanging prefix (system prompt, long reference docs) at the top so it's cacheable.
- Instruction placement — for long inputs, repeating the key instruction after the context (as well as before) often helps the model not "forget" the task.
Format matters
- Delimit clearly — separate sections with headers, XML-like tags, or fences so the model knows where the document ends and instructions begin. This also reduces injection confusion.
- Label retrieved chunks — include source/title so the model can cite and you can trace grounding.
- Match the model's training — markdown and clean structure are well-represented in pretraining; consistent formatting is easier for the model to parse than walls of text.
A workable default layout
- System prompt / role + rules (stable, cacheable).
- Long reference material or retrieved context, clearly delimited and labeled.
- Few-shot examples (if any).
- The user's request and the key instruction, at the end.
Pitfall
Don't blur boundaries between instructions and data. Untrusted retrieved or user-supplied text placed without delimiters invites prompt injection and makes the model treat data as commands.
Connects to: context management · attention bias · injection via context