The search quietly ignores half the query
Context
Searchers ask for one thing and get another. Support has complaints from three searchers this
month: result lists come back with the wrong companies on them, with technology areas nobody asked
for, with documents published outside the requested dates, and with documents that contain the exact
words the searcher asked to leave out. Other times the same query comes back empty even though the
searcher can point at a document that should match.
Patent searchers type one structured string into a single search box, for example
("neural network" OR "deep learning") AND assignee:Siemens AND cpc:G06N3/08 AND pubdate:2018-2021 NOT "quantum".
assignee is the company that owns the patent, cpc is a CPC code (Cooperative Patent
Classification: a hierarchical code that says what technology a patent covers), and pubdate is the
publication date. The service parses that string into a set of filters plus a text query, then
searches the publication store.
This repo is a trimmed copy of that service with a 90 record synthetic corpus in place of the real
publication store, so the whole query path runs offline in milliseconds.
The query language it has to support
This is the contract. The current code does not honor all of it.
Structure:
- A query is a list of clauses, and a document has to satisfy every clause.
AND between clauses is
optional, so turbine "deep learning" means the same as turbine AND "deep learning".
- A clause is one or more alternatives joined by
OR, and any one of them satisfies the clause.
OR groups its alternatives before AND splits the clauses, so a OR b AND c means
(a OR b) AND c. Parentheses may be written around an OR group and must be respected.
- Text in double quotes is a phrase. A phrase matches only when those words appear next to each
other, in that order, in the title or abstract.
NOT applies to the single term, phrase or parenthesized group that follows it. An excluded term
or phrase takes the document out of the results. It does not merely lower its score.- Field filters are
assignee:, cpc: and pubdate:. A filter value may be quoted when it
contains spaces, as in assignee:"Siemens AG". Every filter in a query has to be satisfied, and
filters are written outside parentheses.
Field semantics:
assignee matches on whole words, case insensitively. assignee:Siemens matches
"Siemens AG", "Siemens Healthineers AG" and "Siemens Energy Global GmbH".
assignee:"Siemens AG" matches only "Siemens AG".cpc asks for a branch of the classification tree, not for one exact code. The same code can be
written several ways, and they all mean the same request (G06N3/08, G06N 3/08 and g06n3/08).
A stored code matches when it sits at or under the requested code: cpc:G06N3/08 matches stored
G06N 3/08, G06N 3/084 and G06N 3/0895, but not G06N 3/8 and not G06N 5/04. The match has
to line up with the divisions in the code, at every level. cpc:G06F3 must not match
G06F 30/27, because main group 30 is not main group 3.pubdate accepts 2019, 2018-2021, 2020-, -2019, 2019-07-05 and
2019-07-05..2020-03-01. Bounds are inclusive and are real dates, not strings. pubdate:2018-2021
includes a document published on 2021-11-02.- An empty query, or a query that cannot be understood (a filter with no value, a dangling
operator, an unbalanced quote or parenthesis), raises a clear error. It must never fall through to
"return everything".
The parsed query shape
parse_query(text) returns a dict. Other parts of the service, and the tests, rely on these keys:
{
"terms": ["turbine"],
"phrases": ["neural network"],
"exclude": ["quantum"],
"filters": {
"assignee": ["Siemens AG"],
"cpc": ["G06N3/08"],
"pubdate": {"start": "2018-01-01",
"end": "2021-12-31"},
},
"groups": [
[{"kind": "phrase", "value": "neural network"},
{"kind": "phrase", "value": "deep learning"}],
],
}
terms and phrases are a flat list of everything the searcher asked for, with the AND and OR
structure dropped. groups keeps that structure. A query with one clause therefore appears in both.
apply_filters(docs, parsed) applies the field filters and the exclusions and returns the records
that are allowed into the result set.
search(query_text, k=10) returns a list of at most k result dicts with the keys id, title,
assignee, cpc, pubdate and score, best first. Keep these names and this shape. If you decide
a different structure is better, keep parse_query, apply_filters and search working with the
shape above and describe the alternative in NOTES.md.
Your task
- Run
python3 demo.py before you change anything and write down, for each of the five example
queries, how many hits come back and which of them are wrong (wrong assignee, wrong CPC, out of
the date range, contains an excluded word, missing a requested phrase). That is your before
number.
- Make the query path honor the contract above: parsing, filtering and the boolean structure.
Work in priority order, and say in
NOTES.md what order you chose and why.
- Add tests for the behavior you fix. The existing tests in
tests/ describe how the service
behaves today, so adjust them where today's behavior is wrong and keep them passing.
- Write
NOTES.md, 5 to 10 lines: what you found, what you fixed first and why, the before and
after numbers from step 1, and anything you would do next with more time.
What's here
qsearch/parser.py parse_query(): query string to parsed query dict
qsearch/filters.py apply_filters(): field filters and exclusions
qsearch/retriever.py scores and ranks the candidate set, weighting rare words more
heavily than common ones (idf weighting)
qsearch/search.py search(): parse, filter, rank
qsearch/corpus.py loads data/patents.json
qsearch/textutil.py tokenizing and phrase helpers
data/patents.json 90 synthetic patent records (title, abstract, assignee, cpc, pubdate)
demo.py runs example queries and prints what comes back
tests/ unittest suite that passes on the code as it stands
The corpus is synthetic but it is shaped like the real store: assignee names arrive with legal
suffixes, CPC codes arrive with and without the space before the main group, and publication dates
are ISO day strings.
Running it
From the repo root:
python3 -m unittest discover -s tests -v
python3 demo.py
python3 demo.py 'assignee:"Siemens AG" AND ("neural network" OR "deep learning")'
Time
Aim for about 25-30 minutes. You don't need to finish everything; we care more about how you
approach it than about completeness.