Swap the model under a notes-based agent memory and accuracy moves 13 points, in a direction that depends on which way you swapped

Ankit Goyal and Jaideep Ray of LinkedIn measured what happens to an agent’s memory when a different model reads it than wrote it.

Memory written as prose is coupled to its author. Memory written into a schema is not, at least on this task.

The setup isolates the writer from the reader. Forty-eight synthetic histories with randomised answer codes were turned into four stores: the raw history for long-context reading, chunks for retrieval, model-written natural-language notes, and a fixed-schema knowledge graph of subject-predicate-object claims where the schema fixes the keys and relation names and the model fills in only the values. Two similarly sized open models, Llama 3.1 8B Instruct and Qwen2.5 7B Instruct, each wrote a store and each read the other’s.

Qwen reading Llama’s notes lost 13.3 points against reading its own; Llama reading Qwen’s notes gained 9.9. The average is close to zero, and the authors are explicit that this is two large effects cancelling, not interchangeability. The asymmetry is in the writing: in a calibration sweep Qwen retained 85.6 per cent of the required evidence in its notes and Llama 64.5 per cent, so the better writer’s notes help the worse reader and the worse writer’s notes hurt the better one. The knowledge graph moved by 0.0004, against a swing of 13 points for notes.

Retrieval fails earlier than the reader. The RAG configuration, a single-stage dense retriever over event chunks with cosine top-8 and no reranker, surfaced the needed evidence about 60 per cent of the time regardless of reader, and 81 per cent of its accuracy deficit was that miss. Replacing the embedding model, bge-large v1.0 for v1.5, matching the old model’s 1,024 dimensions so nothing errors, and re-embedding everything gained 11.90 points; a 50/50 mixed index captured 4.96 points, less than half of that.

Repair is where raw history earns its storage cost. Rewriting notes from the store alone never reached the 90 per cent recovery target in any of the 48 cases; with the raw history retained, one migration direction recovered 34 of the 48 cases. The authors list their limits themselves: two sub-10-billion-parameter models, one cross-family migration, histories that fit both context windows, a schema built for a synthetic task. None of those limits reverses the direction of the notes result, which is the finding a team can act on.

Appendix C is a 20-probe migration check meant to run before a model swap ships, which is a smaller ask than the regression suite the result implies.