Datasheets and data documentation
Mechanism: dataset decision → documented evidence → consumer constraint
card = {"purpose":"support", "collection":"opt-in", "limits":"English only", "owner":"data"}
assert all(card.values())
print("dataset contract exists")
Run with python3; expected output is dataset contract exists. Document motivation, composition, collection, preprocessing, recommended and prohibited use, rights, maintenance, and change history; consumers must treat gaps as constraints, not footnotes.
Sources
- Datasheets for Datasets — original documentation framework.
- Data Cards — structured communication.
Dataset documentation makes data assumptions visible. A datasheet or data card records how a dataset was created, what it represents, what it excludes, and how it should or should not be used.
What to document
- Motivation and intended use.
- Collection process, sources, time range, and sampling strategy.
- Labeling guidelines, annotator population, and agreement.
- Preprocessing, filtering, deduplication, and transformations.
- Composition by slices, language, geography, domain, and sensitive attributes where appropriate.
- Known gaps, biases, limitations, and prohibited uses.
- Privacy, consent, licensing, access, and retention.
- Maintenance plan and owner.
Why it matters
| Use | Documentation helps |
|---|---|
| Training | choose appropriate data and avoid misuse |
| Evaluation | interpret scores and slice gaps |
| Auditing | trace provenance and decisions |
| Governance | assess privacy, consent, and risk |
| Handoffs | prevent context loss between teams |
Keep it alive
Documentation should update when sources, labeling guidelines, filters, splits, licensing, or known limitations change. A stale datasheet becomes a false sense of control.
Pitfall
Do not bury important limitations in prose nobody reads. Put critical restrictions, coverage gaps, and prohibited uses in the first screen of the documentation.
Connects to: dataset design · dataset privacy · AI ethics and governance