006·333
chunking
/ˈtʃʌŋkɪŋ/ - n.
1 [colloq.] Cutting the policy in half exactly where the exception lived.Keep. Punchy.This is the problem.
Working definition
2. Splitting source documents into passages sized for retrieval and model context windows.
Filed
See also
- context windowA budget the organization fills with everything except the one document that decided the matter.
- embeddingsCoordinates that faithfully encode whatever bias the source corpus never declared.
- retrieval-augmented generationA pipeline that retrieves the company's contradictory documents and asks the model to sound certain about all of them.
- vector databaseThe new system of record for documents no one would govern in the old system of record.