NIST Asked Where the Data Came From; The Pipeline Went Quiet

NIST's Generative AI Profile lists content provenance as a named risk-management practice. Then, in plain text, it concedes the awkward part: most model builders cannot actually tell you what they trained on.

On 26 July 2024, NIST published AI 600-1, the Generative AI Profile, a companion to the AI Risk Management Framework developed with public input. It frames content provenance and data documentation as governance controls spanning the AI lifecycle, and describes high-integrity information as that which 'can be linked to the original source(s) with appropriate evidence' and has 'a clear chain of custody.'

The same document states the operating reality without flinching: 'most model developers do not disclose specific data sources on which models were trained, limiting user awareness of whether personally identifiable information (PII) was trained on and, if so, how it was collected.' A framework that recommends provenance is also documenting, in the same pages, that provenance frequently does not exist.

This is the recurring gap dressed in federal typography. Provenance, lineage, and a chain of custody are not properties a model acquires at inference time; they are records a data function maintains from collection onward, or they are absent forever. The Profile can ask for the receipt. It cannot conjure the bookkeeping that organizations declined to staff while they were busy standing up the model.

Watch whether 'data provenance' shows up in AI programs as a funded role with a system of record, or as a slide. The honest test is simple and cheap: pick one production data set and ask who can produce its origin, its consent basis, and its known gaps without phoning a contractor. The answer reveals whether you have governance or just a vocabulary for it.