Classification
Data & Privacy
Overview
Data provenance and lineage track where data comes from, how it was collected, transformed, labeled, versioned, and used across the AI lifecycle. Provenance documents sources, collection methods, consent and licenses, while lineage records the chain of transformations (cleaning, augmentation, feature engineering), dataset splits, and the linkage between datasets and model versions. Strong provenance/lineage improves reproducibility, auditability, and the ability to diagnose failures or bias. Limitations include incomplete historical records, fragmented pipelines across teams, and reliance on manual documentation that can drift from reality. In modern stacks, automated lineage capture (through data catalogs and ML observability tools) is essential to meet regulatory expectations and support incident response.
Governance Context
GDPR and the EU AI Act require transparency about data origins and characteristics, especially for high-risk AI. Article 10 of the EU AI Act mandates data quality, relevance, and representativeness; organizations must show where data came from and how it was prepared. NIST AI RMF recommends controls for Data Provenance and Traceability, and ISO/IEC 23894:2023 emphasizes documenting data pipeline assumptions, limitations, and change logs. Two concrete obligations: (1) maintain an auditable data inventory and lineage graph linking sources transformations model versions, including licenses/consents; (2) implement change management with approvals and automatic logging so that model cards and technical documentation are updated when upstream data shifts.
Ethical & Societal Implications
Poor provenance obscures bias sources, unlawful collection, or misuse of data beyond consented purposes. Communities affected by AI decisions lose transparency and redress pathways without traceability. Conversely, clear lineage supports accountability, fosters trust, and enables corrective action when errors or harms are identified.
Key Takeaways
Provenance tracks origins; lineage tracks transformations and usage.; Regulations expect auditable links from data sources to model versions.; Automated lineage reduces drift between documentation and reality.; Clear traceability accelerates incident response and root-cause analysis.; Licensing and consent must travel with data through the pipeline.