# Synap Is #1 on LongMemEval: 95.6% on All 500 Questions > Maximem Synap is #1 on LongMemEval: 95.6% on all 500 questions with Gemini 3.8 Flash, at about 7.4K tokens per question. Every answer and verdict is public. _Gaurav Dadhich ยท 2026-10-29_ Maximem Synap is #1 on LongMemEval. It answers 478 of the benchmark's 500 questions correctly, 95.6%, ahead of every published score from Mastra, Hindsight, Mem0, Honcho, Zep and Supermemory. That is answer accuracy, not a retrieval metric such as Recall@20 or Recall@200, and Synap gets there with Gemini 3.8 Flash, a fast and inexpensive answer model, and with about 7,400 tokens of memory per question on an ingested corpus of ~115,000 tokens. Every answer, every retrieved memory and every judge verdict from the run is public, so you can count the 478 yourself. We ran it on our \[open eval harness\](https://github.com/maximem-ai/memory\_and\_context\_eval\_harness) on the cleaned LongMemEval\_S dataset, with gpt-4o as the judge and LongMemEval's own grading prompts, the same judge setup the benchmark's authors use. !\[Bar chart of LongMemEval accuracy as published by each vendor. Maximem Synap 95.6%, Mastra 94.87%, Hindsight 94.6%, Mem0 94.4%, Honcho 90.4%, Zep 90.2%, Supermemory 85.2%.\](https://qirwvzn87kgnbyp6.public.blob.vercel-storage.com/assets/images/blogs/covers/1793252702413-w1d8x5xdhjg.png) ## What LongMemEval tests \[LongMemEval\](https://arxiv.org/abs/2410.10813) (Wu et al., ICLR 2025) is the standard benchmark for long-term memory in chat assistants and agents. Each question sits on top of roughly 115,000 tokens of conversation between one user and an assistant, spread across dozens of sessions, and can only be answered from something said earlier. The 500 questions cover five abilities: pulling a fact out of one session, combining facts across sessions, reasoning about when things happened, noticing that a fact has changed, and declining to answer when the history does not contain the answer. Those five abilities are where agents fail in production. An agent that misses a knowledge update books the old address. One that cannot combine sessions asks a customer what they already said last week; one that cannot abstain invents a preference the user never stated. ## The LongMemEval leaderboard \*\*Table 1: LongMemEval\_S answer accuracy, all 500 questions\*\* | System | Accuracy | Context per question (tokens) | Run data public | Eval harness open | Answer model | Judge model | Score date | |---|---|---|---|---|---|---|---| | \*\*Maximem Synap\*\* | \*\*95.6%\*\* | ๐ŸŸข 7.4K (1x) | ๐ŸŸข Yes | ๐ŸŸข Yes | ๐ŸŸข Gemini 3.8 Flash | ๐ŸŸข gpt-4o | 28 Oct 2026 | | Mastra | 94.87% | ๐Ÿ”ด ~30K (4.1x) | ๐Ÿ”ด Not found | ๐ŸŸข Yes | ๐ŸŸข gpt-5-mini | ๐ŸŸข gpt-4o | 9 Feb 2026 | | Hindsight | 94.6% | ๐Ÿ”ด ~43.6K (5.9x) | ๐ŸŸข Yes | ๐ŸŸข Yes | ๐ŸŸข Gemini 3.1 Pro | ๐ŸŸก Gemini 2.5 Flash-Lite | 23 Mar 2026 | | Mem0 | 94.4% | ๐ŸŸข ~6.8K (0.9x) | ๐Ÿ”ด No | ๐ŸŸข Yes | ๐Ÿ”ด Not stated | ๐Ÿ”ด Not stated | 13 May 2026 | | Honcho | 90.4% | ๐ŸŸก ~11.6K (1.6x) | ๐ŸŸข Yes | ๐ŸŸข Yes | ๐ŸŸข Claude Haiku 4.5 | ๐ŸŸข gpt-4o | 12 Dec 2025 | | Zep | 90.2% | ๐ŸŸข ~4.4K (0.6x) | ๐Ÿ”ด No | ๐Ÿ”ด No | ๐ŸŸข gpt-5.4 | ๐ŸŸก gpt-5.4 | Not dated | | Supermemory | 85.2% | ๐Ÿ”ด Not published | ๐Ÿ”ด Not found | ๐ŸŸก Yes, unnamed | ๐ŸŸข Gemini 3 Pro | ๐ŸŸข gpt-4o | May 2026 | \\\* Figures as each vendor publishes them, checked 28 Oct 2026; context is retrieved context only. Every row except ours is the vendor's own published figure, checked on 28 October 2026 against the vendor's live page or repository: \[Mastra\](https://mastra.ai/research/observational-memory), \[Hindsight\](https://hindsight.vectorize.io/blog/2026/03/23/agent-memory-benchmark), \[Mem0\](https://github.com/mem0ai/memory-benchmarks), \[Honcho\](https://honcho.dev/evals/), \[Zep\](https://www.getzep.com/research), \[Supermemory\](https://supermemory.ai/research/longmembench/). We did not re-run competitors. ## Why the model and the context size matter A benchmark score is the product of the memory layer and the model that reads its output. A frontier model can rescue weak retrieval by reading a large slice of the conversation and finding the answer itself, which tells you about the model, not the memory. Synap's 95.6% comes from a fast-tier model reading a small, selected context. The two systems closest to us on score sit 0.73 and 1.0 points behind, and they hand their answer model far more: Mastra about 30,000 tokens per question and Hindsight about 43,600, roughly 4x and 6x our 7,400. Out of a 115,000-token history, that is a quarter to over a third of the whole conversation on every call. On a benchmark that works. In production it shows up as cost and latency on every turn, and it gets worse as a real user's history grows past what LongMemEval simulates. Zep and Mem0 send less context than we do, around 4,400 and 6,800 tokens. Neither scores as high, and Mem0 does not say which model produced its 94.4%. ## The same memory, a different model The memory layer is what carries the result. With frontier answer models in place of Gemini 3.8 Flash, the same Synap retrieval goes higher: 95.2% (476/500) with gpt-5.4 and 97.0% (485/500) with Gemini 4 Argon. We lead with Flash because that is the class of model most production agents actually run on, and because a frontier model can read its way past weak retrieval in a way a Flash model cannot. Both figures are listed in the results repository alongside the headline run. ## What is public, and what is not A score you cannot recount is a claim. For ours, the \[results repository\](https://github.com/maximem-ai/eval\_benchmark\_runs\_output) holds, for every one of the 500 questions, the memories Synap retrieved, the answer the model wrote and the judge's verdict, along with the exact answering prompt and the run manifest. To recount the headline: \`\`\`bash git clone https://github.com/maximem-ai/eval\_benchmark\_runs\_output.git cd eval\_benchmark\_runs\_output python3 -c "import json; r=json.load(open('results/longmemeval/question\_records\_gemini-3.8-flash.json')); ok=\[x for x in r if x\['judge'\]\['judge\_raw'\].strip().lower().startswith('yes')\]; print(len(ok), 'of', len(r))" \`\`\` It prints 478 of 500. Across the table, Hindsight and Honcho also publish per-question results. Mastra, Zep and Supermemory do not, as far as we could find. Mem0 publishes neither the model behind its 94.4% nor the run outputs for it. ## Recall@20: 98.1% Accuracy grades the final answer. Recall@20 grades the step before it: was the evidence needed to answer among the first 20 memories Synap returned? On the 470 answerable questions, it was there 98.1% of the time (461 of 470), judged by gpt-4o with the same relevance prompt Supermemory uses for its published 97% Recall@20. Recall is a retrieval measure, so it sits here and not in the accuracy table. ## Run it on your own memory system The harness runs LongMemEval and LoCoMo against any memory system that implements its provider interface, and ships adapters for Synap, Mem0, Zep and Supermemory. \`\`\`bash git clone https://github.com/maximem-ai/memory\_and\_context\_eval\_harness.git cd memory\_and\_context\_eval\_harness pip install -e . cp .env.example .env # add your API keys python scripts/download\_datasets.py --variant s python -m runner.server # dashboard at http://localhost:8766 \`\`\` To add your own system, implement the \`Provider\` class in \`docs/BUILD\_YOUR\_OWN\_ADAPTER.md\` and check it with \`pytest tests/unit/test\_provider\_contract.py\`. If your run disagrees with ours, open an issue with your config and your output. ## What this means if you are building an agent Your users stop repeating themselves. The agent keeps what they told it last month, notices when it changes, and says so when it does not know. Because the memory layer does the selecting, you can run that on a fast model with a few thousand tokens of context instead of resending a third of the conversation on every turn, which matters most for voice agents, where every token is latency the caller hears. On production systems on the recommended config, Synap serves P75 reads in under 15ms. The number that matters most is the one on your own conversations. LongMemEval tells you which memory layer is worth testing on them, and the harness is how you run that test. \[Start free with Synap\](https://synap.maximem.ai). ## FAQ \*\*What is LongMemEval?\*\* LongMemEval is a benchmark from Wu et al. (ICLR 2025) that tests long-term memory in chat assistants. Its standard S setting has 500 questions, each answered from about 115,000 tokens of earlier conversation, across single-session recall, multi-session reasoning, temporal reasoning, knowledge updates and abstention. \*\*What is the highest LongMemEval score?\*\* Among agent memory platforms that publish a LongMemEval score, Maximem Synap's 95.6% on all 500 questions is the highest as of 28 October 2026, followed by Mastra (94.87%), Hindsight (94.6%) and Mem0 (94.4%). \*\*Which model did Synap use?\*\* Gemini 3.8 Flash wrote the answers and gpt-4o judged them with LongMemEval's own prompts. With frontier answer models, the same memory scores 95.2% (gpt-5.4) and 97.0% (Gemini 4 Argon). \*\*How many tokens does Synap send per question?\*\* About 7,400 tokens (7,391 mean) of retrieved memory per question, out of a roughly 115,000-token history. \*\*Can I verify the result?\*\* Yes. Every question's retrieved memories, answer and verdict are in the public results repository, with a one-line command that recounts the score. \*\*Is LongMemEval the same as LoCoMo?\*\* No. LoCoMo is a separate benchmark built on shorter conversations of roughly 16,000 to 26,000 tokens. Synap's LoCoMo score is 93.2% from an earlier run with gpt-5-mini. --- Source: [https://www.maximem.ai/blog/synap-number-one-longmemeval](https://www.maximem.ai/blog/synap-number-one-longmemeval)