Data-centric AI

Mechanism: data failure → measurable intervention → held-out outcome

labels_before, labels_after = .82, .91
print("improved" if labels_after > labels_before else "investigate")

Run with python3; expected output is improved. Improve data coverage, labels, measurement, and lineage before assuming a model change is the remedy; validate every intervention on a held-out task slice.

Sources

• Data Cascades in High-Stakes AI — compounding data failures.
• Datasheets for Datasets — data lifecycle questions.

Data-centric AI treats the dataset as the main product of iteration. Instead of immediately swapping architectures or prompts, you improve labels, coverage, examples, documentation, and feedback loops.

The shift in focus

The model still matters. Data-centric work just prevents model iteration from hiding dataset problems.

Where it pays off

• Small or medium datasets where label quality dominates.
• Domain tasks where examples need expert judgment.
• Evaluation suites where one bad label can mislead release decisions.
• RAG systems where document quality and chunk boundaries decide answer quality.
• Fine-tuning where consistent examples matter more than raw volume.

Data iteration loop

1. Measure performance by slice.
2. Inspect errors by hand.
3. Label the root cause: missing coverage, bad label, noisy input, ambiguous task, stale data, or leakage.
4. Fix the data or rubric.
5. Re-run the same eval.
6. Version the dataset change.

Pitfall

Data-centric does not mean "collect more data". More of the same noisy distribution can make the system harder to debug. Improve the examples that change decisions.

Connects to: error analysis · designing eval sets · data quality for fine-tuning