Phase 03 — Training and Inference
Training and inference execute related graphs under different constraints. Training stores activations and gradients to update parameters; inference optimizes loading, memory traffic, batching, caching, and response latency.
Mental model
Training is a repeated measure-and-update system; inference is a scheduled read-only execution system. Both are bounded by tensor shapes, memory movement, precision, parallelism, and reliability requirements.
Roadmap through the branches
Exit criteria
You can describe data/model/pipeline parallelism, mixed precision, checkpointing, adaptation methods, prefill versus decode, KV-cache pressure, continuous batching, quantization, and latency-throughput-cost tradeoffs.
Connects to: Phase 02 — Learning and Models · Phase 04 — Context and Agency
Core sources
- PyTorch Distributed Overview — training parallelism primitives.
- QLoRA — memory-efficient adaptation.
- vLLM — memory-aware high-throughput serving.