Give each kind of context a job
A returning conversation needs continuity. It also needs room for the person to change their mind. Storing everything and recalling everything are different design choices; neither guarantees that the next answer will use the right context.
One useful structure separates the assistant's instructions, the task context, durable conversation memory, recent conversation state and a short transcript. Each layer has a different purpose and a different failure mode.
Implementation reviewed September 13, 2026.
Separate durable facts from unfinished business
The implementation uses two summary fields. The static summary holds relatively durable information about the person and their preferences. The rolling summary holds recent topics and unresolved threads. Recent messages are also supplied in their original conversational roles.
| Context | What it is for | What should not be inferred from it |
|---|---|---|
| Instructions and task context | The assistant's role and the current task | The task can never change |
| Durable summary | Preferences, recurring interests and important facts | A preference can never change |
| Rolling summary | Recent discussion and unfinished threads | An old task is still active |
| Recent transcript | Exact wording and immediate corrections | It contains the whole conversation |
These distinctions are useful when debugging. If the answer carries an obsolete preference, inspect the durable summary. If it resumes an abandoned topic, inspect the rolling state. If it misses a correction made one turn earlier, inspect the transcript and context construction before adding another retrieval component.
Keep the write path separate
In the current flow, completed message pairs are persisted before the summary refresh runs. The refresh receives existing summaries and recent messages, then returns structured replacement summaries. A failed refresh is logged and skipped; the completed conversation remains stored.
This makes summarisation a secondary operation. The application has a durable record from which it can recover, rather than relying on generated prose as the only account of what happened.
It does not make the summaries automatically correct. Compression can omit a qualification, promote a tentative statement into a fact or carry an outdated preference into a later conversation. A successfully parsed response proves that the output has the expected fields. It does not prove that the summary is faithful.
Build an evaluation around corrections
A useful test conversation is deliberately ordinary. In a synthetic example, a person first says they work in Brussels, later says they have moved to Lisbon, then asks for a suggestion relevant to where they now live. A summary that retains both cities without explaining the change creates ambiguity that vector similarity alone cannot resolve.
A compact evaluation set should include:
- A corrected fact that must replace an earlier statement.
- A temporary preference that should not become a permanent trait.
- Two parallel conversations whose context must remain separate.
- A request to forget something, checked against both summaries and retained history.
- A summary failure followed by a normal conversation turn.
Score the next answer as well as the stored summary: a good summary is only useful if it produces appropriate behaviour.
Know when this structure stops being enough
Summaries lose detail. A question about an exact phrase from months earlier may need access to the original record. That is a different requirement from maintaining the tone and continuity of a conversation.
The current context builder has a place for retrieved grounding, but the inspected turn service does not populate it. Retrieval remains a possible addition when concrete failures justify it.
Research such as Lost in the Middle also cautions against assuming that supplying more context guarantees effective use of it. Context arrangement deserves evaluation alongside context quantity. See the paper in the references for its experimental conditions.
A summary is a working view
The next improvements should follow observed failures. A version check would help prevent overlapping summary jobs from replacing newer state with older output. A deletion policy should explicitly cover derived summaries. Exact recall could use a separate retrieval path with source references.
The starting point is simpler: keep the record, assign each context layer a purpose, and make it possible to inspect why a particular memory reached the model.