Data for AI

Foundations covers the concepts and Machine Learning covers the methods, but in practice the data is where projects are won or lost. This branch is the data-centric view: treating the dataset, not the model, as the primary thing you iterate on.

Mental model

Data is a sampled, transformed, and governed measurement of the world—not the world itself. Dataset quality is therefore coverage plus provenance: what was observed, what was omitted, how labels were produced, which transformations ran, and whether deployment inputs still match those assumptions.

Roadmap: data quality and design

• Data-centric AI explains why improving data often beats changing the model.
• Data quality dimensions defines accuracy, completeness, consistency, freshness, coverage, and validity.
• Dataset design and sampling covers representativeness, splits, balance, and slice coverage.
• Data cleaning and deduplication shows how messy examples become training and evaluation failures.

Labels, documentation, and feedback

• Labeling and annotation covers guidelines, inter-annotator agreement, weak supervision, and review loops.
• Datasheets and data documentation records provenance, collection, limits, and intended use.
• Feedback data and active learning turns production mistakes into better datasets.

LLM-era data

• Synthetic data explains where generated examples help and where they collapse quality.
• Data contamination and benchmark leakage protects evaluation from hidden training exposure.
• Data for LLMs covers pretraining corpora, filtering, mixture design, and deduplication at scale.

Data operations and governance

• Data pipelines, versioning, and lineage keeps datasets reproducible and auditable.
• Privacy and PII in datasets handles consent, minimization, anonymization, and access boundaries.

Data strategy

• The data flywheel turns usage into a compounding loop of better data and a better product.

Connects to: Data Splits and Leakage · Statistical Machine Learning · AI Ethics and Governance

Core sources

• Datasheets for Datasets — provenance and lifecycle documentation for datasets.
• Data Cards — structured communication of dataset context and limitations.
• Hidden Technical Debt in Machine Learning Systems — data dependencies and feedback loops as systems debt.
• The Curse of Recursion — synthetic-data feedback and model-collapse analysis.
• Hugging Face Datasets — reproducible loading, processing, streaming, and dataset metadata.