MediumSWE (AI-Era)Coding

Let searchers narrow the results

Context

Patent searchers use this service to look for prior art. One of them wrote in: "I typed neural network assignee:"Siemens AG" and got back 24 results from a dozen different companies. I only want the Siemens ones. The same thing happens when I ask for a classification code or a publication year: the words are searched, the rest of what I typed is thrown away."

Right now the search box does plain keyword search and nothing else. Your job is to make the filters a searcher types into that box actually narrow the results.

Your task

1. Fix the empty-result bug (about 5 minutes). A search that matches nothing comes back with the whole corpus instead of an empty result list. Find it and fix it. A search that matches nothing must return no results.

2. Support the three filters (the main task). A searcher types filters into the same box as the words, mixed in any order. Parse them and apply them. A record is only a result if it passes every filter in the query, on top of carrying every word.

| Filter | Means | Examples | | --- | --- | --- | | assignee: | the company that owns the patent | assignee:Siemens, assignee:"Siemens AG" | | cpc: | the classification code | cpc:"G06N 20/00", cpc:"G05B 13/02" | | pubdate: | the publication date or a date range | pubdate:2019, pubdate:2018-2021, pubdate:2020-, pubdate:-2017, pubdate:2019-07-05 |

A CPC code (Cooperative Patent Classification) is the code that says what technology a patent covers, for example G06N 3/08 for neural network learning methods. Codes are stored on each record in the cpc list.

Syntax, exactly as it arrives from the search box:

neural network                                             words only
neural network assignee:Siemens                            words plus one filter
assignee:"Siemens AG" cpc:"G06N 20/00" pubdate:2018-2021   filters only, no words
neural assignee:"Siemens AG" pubdate:2020- cpc:"G05B 13/02"  any order

A filter is field:value with no spaces around the colon. Wrap the value in double quotes when it contains a space, as in assignee:"Siemens AG" or cpc:"G06N 3/08".
Anything that is not one of those three filters is a search word.
Keep the three filters and only those three. There are no boolean operators, no quoted search phrases and no NOT; do not add any.

Details worth getting right:

assignee matches on whole words, case insensitively. assignee:Siemens matches "Siemens AG", "Siemens Healthineers AG" and "Siemens Energy Global GmbH". assignee:"Siemens AG" matches only "Siemens AG".
pubdate bounds are inclusive and are real dates, not strings. pubdate:2018-2021 includes a record published on 2021-11-02. An open range (pubdate:2020-, pubdate:-2017) filters on the one bound it has. A date value you cannot read is a QueryError, not a filter that matches everything.
For this phase, matching a cpc value against the code as it is stored on the record is enough.
Two filters on the same field both have to hold; it is your call what that means, say so in NOTES.md.
A query of filters with no words is a valid search: it returns the records the filters allow.

The parsed query is a dict, and apply_filters and search read it. Keep this shape:

{
  "terms": ["neural", "network"],                 # search words, lowercased, no repeats
  "filters": {
      "assignee": ["Siemens AG"],                 # values as the searcher wrote them
      "cpc": ["G06N 3/08"],                       # values as the searcher wrote them
      "pubdate": {"start": "2018-01-01",          # inclusive ISO bounds, either side may be
                  "end": "2021-12-31"},           # None; None when there is no date filter
  },
}

search(query_text, k=10) keeps returning at most k result dicts with the keys id, title, assignee, cpc, pubdate and score, best first. Keep these names.

3. Stretch: make a code cover its branch (clearly labelled a stretch, most people will not get here). CPC codes are hierarchical: G06N 3/08 is a branch of the tree, and anything beneath it is in that branch too. So cpc:G06N3/08 should also match the more specific stored codes G06N 3/084 and G06N 3/0895, and it should match them whichever way the code is written: the same code arrives as G06N 3/08, G06N3/08 and g06n3/08, and all three mean the same request. The match has to line up with the divisions in the code: G06N 3/08 must not match G06N 3/8, and a request for G06F3 must not match the stored G06F 30/27, because main group 30 is not main group 3.

Start here

python3 demo.py and look at the five example queries. The company, code and date a searcher typed are visible in the query and absent from the parsed line; one query is rejected because once its filters are dropped there is nothing left to search for; and one returns 50 hits for a word that is in no record at all. That is your before picture.
python3 -m unittest discover -s tests -v to see the suite that passes today.
Phase 1 first: it is small, and it is easier to tell whether a filter works when a query that matches nothing comes back empty.
qsearch/parser.py already recognizes the filter expressions in order to keep them out of the search words. It throws them away instead of reporting them. qsearch/filters.py is where a filter decides whether a record is allowed through.
Add tests for what you build, and keep the existing suite passing. If you make a judgement call worth flagging, leave a couple of lines in NOTES.md.

What's here

qsearch/parser.py      parse_query(): query string to parsed query dict
qsearch/filters.py     apply_filters(): decides which records are allowed into the results
qsearch/retriever.py   scores and ranks the records that got through, weighting rare words
                       more heavily than common ones
qsearch/search.py      search(): parse, filter, rank
qsearch/corpus.py      loads data/patents.json
qsearch/textutil.py    tokenizing and phrase helpers
data/patents.json      90 synthetic patent records (title, abstract, assignee, cpc, pubdate)
demo.py                runs example queries and prints what comes back
tests/                 unittest suite that passes on the code as it stands

The corpus is synthetic but shaped like the real store: company names arrive with their legal suffixes, CPC codes arrive with and without the space before the main group, and publication dates are ISO day strings. Python 3.11, standard library only, no network.

Running it

From the repo root:

python3 -m unittest discover -s tests -v
python3 demo.py
python3 demo.py 'neural network assignee:"Siemens AG"' 'cpc:"G06N 20/00" pubdate:2020'

Time

Aim for about 30 minutes: roughly 5 on phase 1 and the rest on phase 2. Phase 3 is a stretch and most people will not reach it. Finishing all three phases is not expected. A clean, tested phase 2 is a good outcome; we care more about how you work than about how much you cover.

Sign in to attempt this problem

Free account gives you full access to community problems with the complete solution reveal: golden answer, senior walkthrough, and score breakdown, after submission.

Start free →Already have an account? Sign in