Metadata

Why metadata beats full-text search for records#

Full-text search finds documents that mention a word. Metadata finds documents that are a thing. "Every contract with this counterparty expiring in the next quarter" is a metadata query, and no amount of text search answers it — the renewal date exists as a sentence in a PDF, not as a value anything can sort by.

This is also why recognition accuracy matters less than it first appears. A document with correct metadata remains findable even when its OCR is imperfect.

The rule that decides whether a model survives#

Optional fields decay to blanks. Every metadata model that has been in place for a few years contains fields that were added for a reason nobody remembers and have been empty ever since. The only reliable defence is to enforce a small number of fields at the point of upload and accept that everything else will be sparsely populated.

Three to six mandatory fields per document type is usually the right order of magnitude. More than that and people work around the system; fewer and the repository is unqueryable.

Which fields earn their place#

A field earns mandatory status if a report, a retention rule or a routing decision depends on it. Renewal date earns it because renewals are missed without it. Notice period earns it because renewal minus notice is the date that actually matters. "Department" usually does not, because the folder already says so and it changes when people reorganise.

Extraction and confirmation#

Where AI extraction populates fields from document content, the value should be presented for confirmation with its source passage retained rather than applied silently. Metadata nobody verified is metadata nobody can report from safely.