Drafts cut off halfway
Context
Attorneys tell us that a generated draft sometimes stops in the middle of a sentence, that
words show up glued together or scrambled, and that refreshing the page loses everything the
model had written so far. When an attorney asks for a draft, the text arrives in the editor
while the model writes it, over server-sent events (a long-lived HTTP response where the
server pushes text to the browser as it is produced, instead of making the browser wait for
the whole draft). The endpoint and the browser side of that stream are in this repo. Nobody
has found a pattern in the complaints yet, and the editor always ends up saying the draft is
complete, even the times the attorney can see it is not.
Your task
Before you change anything, run node measure.js and write down the numbers it prints. It
replays two captured responses through the client: one whole draft and one that was cut off
part way. Record, for each of them, how many events the client applied, how many characters
of the expected draft it ended up with, whether the text matched exactly, and the status the
client finished in.
- Make the client read the stream safely. A read from the network gives whatever bytes
happened to arrive, so one read can hold several events, or half an event, or the end of
one event and the start of the next.
parseEvents(buffer) should return only the events
that are complete (an event ends with a blank line) plus a remainder string with the
text left over, and the caller should feed that remainder into the next read.
- Make a finished draft different from a dropped connection. The server should send one
last event with
event: done and a data payload of
{"tokens": <number of pieces in the whole draft>, "sha256": "<sha256 hex of the whole draft text>"}, and the client should treat a response that ends without it as an error
rather than as a finished draft. createState() starts at status: "streaming", a done
event puts it in status: "complete", and a stream that ends any other way must not come
out as "complete" and must carry an error the editor can show.
- Save the draft while it streams, and let a client pick up where it left off. Add
append_chunk(draft_id, text) to store.py and call it as pieces arrive, so a refresh or
a dropped connection does not lose the text that was already generated. Then support
resume: a request carrying the id of the last event the client received, either in the
Last-Event-ID request header or in a last_event_id query parameter, continues the same
draft, and the stored draft ends up exactly what an uninterrupted run would have stored.
- Keep the text exactly as the model produced it. Pieces are not words: a piece can be a
single space or half of a long word, so nothing on the way from the wire to the editor may
trim or reflow it.
- Add tests for the above in
tests/, and write a short NOTES.md (5-10 lines): what you
changed, the before and after numbers from node measure.js, and what you would do next.
Keep the names other parts of the app already use: create_app(options) and
GET /api/drafts/<id>/stream in stream_server.py, stream_tokens(prompt, options) in
model_stream.py, append_chunk(draft_id, text) and get_draft_text(draft_id) in
store.py, and parseEvents, applyChunk, createState and finishStream in
web/stream_client.js.
What's here
stream_server.py the streaming endpoint (a WSGI app) and the event format on the wire
model_stream.py drafting model client (simulated, see below). Treat it as external: don't edit it.
store.py where a generated draft is kept, as ordered pieces of text
web/stream_client.js browser side: parse events, fold them into editor state
measure.js replays the captured responses in data/wire through the client
data/drafts.json the draft texts the simulated model writes
data/wire/ two responses captured from the endpoint, plus the draft text they should produce
tests/ existing tests (they pass now; keep them passing)
One event on the wire looks like this, and ends with a blank line:
id: 12
event: token
data: electrode
There is exactly one space after data:, and a reader removes that one space and keeps
everything after it, so a piece that begins with a space (" electrode") survives the trip.
model_stream.py is a deterministic stand-in for the drafting model we call in production.
The same prompt always produces the same pieces, the pieces do not line up with words, and
{"disconnect_after": 12} makes the upstream connection drop part way through a draft, so
both the good path and the bad path can be tested without a network.
Running it
node measure.js # replay the captured responses, print what the editor gets
python3 -m unittest discover -s tests -v # python tests
node --test --no-warnings 'tests/*.test.js' # frontend tests (Node 22.18+)
python3 stream_server.py serve # serve on http://127.0.0.1:8000
curl -N http://127.0.0.1:8000/api/drafts/d-1001/stream # watch the raw events
python3 stream_server.py capture # re-capture data/wire after changing the server
Python 3.11, standard library only. No npm install, no pip install.
Time
Aim for about 25-30 minutes. You don't need to finish everything; we care more about how you approach it than about completeness.