Stop the model from making things up
Context
Our extractor sometimes reports a patent number or a claim limit that is nowhere in the document, and right now those values go straight into the analytics store. We use an LLM to pull claim level data out of patent documents: the publication number, the number of claims, the patent numbers the document cites, and for each claim its number, the claim it depends on, and the numeric range it states (for example "between 5 and 15 wt%"). Most of the time it is right. We want every extracted identifier and number to be traceable to the source document, and anything we cannot verify to go to a person instead.
Your task
- Write
verify.py with verify_extraction(text, extraction) -> dict, which
checks every value the model reported against the document text, and have
process_document() in extract.py include its result alongside
extraction. The return shape is spelled out under "What verify_extraction
returns" below, because our tests read those keys.
- Two families of check. Identifiers and numbers have to be traceable to the
text, accepting harmless formatting differences (
US 2019/0123456 and
US20190123456, 1,200.00 and 1200, 5.0 and 5) but never near misses
(US 2019/0123457 for US 2019/0123456, or 50 for 5). And the claim
structure has to hold together: a dependent claim refers to a claim number
that exists in the document, an independent claim does not depend on
anything, the low end of a range is not above the high end, and the number of
claims reported matches the count the document states.
- Write a short
NOTES.md, three or four lines: which formatting differences
you accept and which you reject and why, and what kinds of made-up values
your checks would still miss.
Nothing gets dropped, fixed or silently skipped. A field you cannot verify is
reported as unverified, and an extra claim the model invented is reported rather
than removed. A clean document comes back with every field verified and
needs_review false. Adding tests of your own in tests/ is welcome but not
required.
Start here
extract.py process_document(), the pipeline your checks plug into
llm.py the simulated extractor, including the ways it invents values
tests/ describes today's behaviour
data/claims/ holds ten documents. You do not need to read them all; the
extractor's docstring tells you which ones it gets wrong.
What verify_extraction returns
Our tests read these keys, so keep them.
{
"needs_review": bool,
"fields": { # one entry per field, keyed by path
"publication_number": {"value": ..., "verified": bool, "reason": str | None},
"claim_count": {...},
"cited_references[0]": {...}, # for every citation the model returned
"claims[0].claim_number":{...}, # plus depends_on, value_low, value_high, unit
... # for every claim the model returned
},
"issues": [str, ...] # human-readable, for the reviewer
}
value is exactly what the model returned, not a cleaned-up version of it.
What's here
llm.py model client (simulated model, see below). Treat it as an external model: don't edit it.
extract.py prompt + model call; returns the model's output as-is and never asks for review
data/claims/ 10 patent documents as text, each with a front page block and a claim set
tests/ existing tests (they pass now; keep them passing)
llm.py is a simulated stand-in for the model we use in production. It is deterministic, and some of the documents trigger the kinds of mistakes described above.
Running it
python3 extract.py data/claims/*.txt # extract every document and show whether it needs review
python3 -m unittest discover -s tests -v # run the tests
Python 3.11, standard library only.
Time
Aim for about 30 minutes. You do not need to finish everything. We care more
about how you approach it than about how much you get through.