Pipecat keeps conversation memory inside one pipeline run. LLMContext and LLMContextAggregatorPair hold the messages, and enable_auto_context_summarization compresses older turns once the context passes a token or message threshold. To carry memory from one call to the next you place a processor before the LLM that injects what you know about the caller, and you record each turn as it happens; Pipecat's docs include Mem0MemoryService for this, scoped by user_id, agent_id or run_id. Maximem Synap ships the same pattern as two frame processors: SynapMemoryProcessor ahead of the user context aggregator and SynapRecorder after the assistant context aggregator, both scoped to the caller.
Every Synap name and argument below matches the open-source package maximem-synap-pipecat 0.2.1, which requires pipecat-ai 0.0.80 or later and Python 3.11 or later.
Where Pipecat Excels
Pipecat is an open-source framework for realtime voice and multimodal agents, built around one idea: a pipeline is an ordered list of frame processors, and everything that happens in a call is a frame moving through it. Audio becomes a TranscriptionFrame, the model's reply streams out as LLMTextFrames, and the spoken words come back as TTSTextFrames. STT, LLM and TTS services are processors like any other, so you can change a provider without rewriting the pipeline around it.
That design is also why memory fits in cleanly. A memory layer is two more processors in the list, one on the read side and one on the write side, and the rest of the pipeline does not need to know they exist.
How Pipecat Memory Works Today
Pipecat and Pipecat Flows give you three context mechanisms, and the Pipecat docs add one long-term memory service. For a deeper look at each, see the context management guide.
LLMContext and LLMContextAggregatorPair manage the conversation inside a run. The pair gives you a user aggregator, placed after STT, and an assistant aggregator, placed after transport.output(); between them they append each turn to the shared context. Frames such as LLMMessagesAppendFrame and LLMMessagesUpdateFrame let you edit that context while the call is live. It is in-memory state for one session.
Auto context summarization is off by default. Set enable_auto_context_summarization=True on LLMAssistantAggregatorParams and Pipecat summarizes older messages when the context passes 8000 estimated tokens or 20 new messages, whichever comes first, keeping the initial system message and the most recent turns uncompressed. It keeps a long call inside the context window; it does not carry anything to the next call.
Pipecat Flows state (flow_manager.state) is a dictionary shared across the nodes of a flow for the length of the conversation. The Flows docs also describe it as the place to put the facts a session starts with, which you load yourself.
Mem0MemoryService is the long-term option in Pipecat's own docs, and it does recall across sessions. Placed before the LLM service, it stores conversation history in Mem0 and retrieves relevant past context, scoped by user_id, agent_id or run_id. Its API calls run in background threads and message storage is fire-and-forget, so it does not delay downstream processing. If Mem0 is the memory engine you want, that service is a complete answer inside Pipecat.
What Pipecat leaves to the memory layer you choose is everything behind the processor: which facts are extracted from a transcript, how a changed fact replaces an old one, how mentions of the same person are linked, and how well the right memory is found a month later.
What Synap Adds
Synap is agentic context management. It does not replace LLMContext or the aggregators. It reads and writes the same shared LLMContext they use.
The package exports three frame processors:
SynapMemoryProcessor sits before the user context aggregator. On each TranscriptionFrame it calls sdk.fetch with the transcript as the search query and adds the result to the shared LLMContext as a system message. On the next turn it replaces that message, so the prompt carries one current memory block and not a stack of stale ones. The transcription frame then continues downstream unchanged.
SynapRecorder sits after the assistant context aggregator, at the end of the pipeline. On LLMFullResponseEndFrame it records the user turn and the assistant turn. In a standard voice pipeline the aggregators and the TTS service consume the frames that carry those turns, so the recorder recovers the user text from the shared context and rebuilds the assistant text from the TTSTextFrame stream, which is what the caller heard and therefore the correct record when the bot was interrupted.
SynapShortTermProcessor injects a compacted summary of the current conversation, refreshed on each user turn. It is optional and covered below.
Pass context= to both main processors. Without it SynapMemoryProcessor injects nothing and SynapRecorder has no user turn to recover, and neither will raise to tell you.
We built this because voice agents kept resetting in production. The problem was not bad audio. It was the caller having to repeat themselves every call. Production testing hit 92% LongMemEval, 93.2% on LoCoMo.
For why context management is infrastructure and not a feature, read What Is Agentic Context Management?. For build-versus-buy numbers, see The Real Cost of DIY Agent Memory.
Technical Deep Dive
LongMemEval Benchmark
LongMemEval and LoCoMo test whether a system can answer a question in one session using facts stated in earlier ones, which is the situation a returning caller creates. The score we report, 92% LongMemEval, 93.2% on LoCoMo, comes from those text benchmarks and not from a voice or multimodal variant. Read it as a measure of the memory engine behind the Pipecat processors. The processors themselves handle text: the read path keys on TranscriptionFrame, and the write path records transcribed and spoken text, so images or audio that never become text are not part of what is stored.
Entity Resolution Mechanism
Two different identity problems tend to get merged here. The first is who is calling, and the processors do not answer it for you: both require a user_id at construction, and mapping a SIP From header, a WebRTC participant or an authenticated user to that id is your code, run before you build the pipeline. The second is who and what the caller talks about. When the same person or thing appears across conversations under different labels ("Maria", "Ms. Alvarez", "my account manager"), Synap's ingestion pipeline links them to one canonical entity, and that resolution respects scope, so entities resolved for one user do not leak to another.
Compaction and Short-Term Context
A fifteen-minute call produces a long context full of fillers and acknowledgements. Pipecat's auto summarization is one way to keep it in bounds. Synap's alternative is SynapShortTermProcessor, placed before the user context aggregator: on each TranscriptionFrame it asks Synap for the conversation's compacted summary plus its recent turns and refreshes a single tagged system message in the shared LLMContext. It takes a required conversation_id and a style of structured, narrative or bullet_points. It does not delete messages from the LLMContext, so use it alongside Pipecat's own trimming and not in place of it.
Fast and Accurate Retrieval Modes
SynapMemoryProcessor takes a mode. The SDK documents fast as vector similarity only and accurate as vector search plus graph traversal and re-ranking, which is the slower of the two. One detail matters more in voice than anywhere else: the processor awaits the fetch before it forwards the transcription frame, so retrieval time is part of every turn. The constructor defaults to accurate. For a live call, start with mode="fast" and measure before you move up.
Multi-Tenant Scoping
Pipecat deployments often run one worker pool for many customers' agents. Scope in Synap is set by constructor arguments: user_id is required, and customer_id is for B2B Synap instances, where it is required. A B2C instance does not accept one, and the SDK refuses the call before it leaves your process. Because the processors hold those ids for their lifetime, you construct a fresh pair for each call with that caller's ids.
What Synap Adds to Pipecat
Persistence
Pipecat Native. LLMContext lives for one run; the documented memory service adds recall across sessions. With Synap. SynapRecorder stores both turns under the caller's user_id, and SynapMemoryProcessor brings them back on later calls.
Entity Resolution
Pipecat Native. Out of scope for the framework; it depends on the memory service you attach. With Synap. People and things mentioned under different labels are linked to one canonical entity during ingestion, inside the caller's scope.
Compaction
Pipecat Native. Auto context summarization, off by default, within one run. With Synap. A compacted summary plus recent turns from SynapShortTermProcessor, refreshed each user turn.
Retrieval
Pipecat Native. None of its own across runs. With Synap. A fetch on each TranscriptionFrame, awaited in the frame path, in fast or accurate mode.
Long-Term Recall
Pipecat Native. Not benchmarked for cross-run recall. With Synap. 92% on LongMemEval.
Failure Handling
Pipecat Native. Errors travel through the pipeline as frames. With Synap. A failed fetch is logged at ERROR and the frame continues with no memory injected. A failed write becomes a SynapIntegrationError that the recorder pushes upstream as an ErrorFrame. The pipeline keeps running in both cases.
User Scoping
Pipecat Native. Per run, unless a memory service adds its own ids. With Synap. user_id on every processor, customer_id on B2B instances, both fixed at construction.
What Production Teams Gain
Memory as two entries in the processor list. The Synap processors go into Pipeline([...]) the same way STT, the LLM service and TTS do. There is no wrapper around your pipeline and no callback to register, so the change is small enough to review in one pull request.
A caller who does not start from zero. Someone calls about a flight on Monday and calls again on Thursday. The second run starts with a new LLMContext, and on the first transcription the memory processor fills in what the first call established. Use the same user_id from a text channel and the same memory is available there, because scope follows the id and not the transport.
A record of what was said aloud. Under interruption, the text an LLM generated and the text a caller heard are different things. In a standard voice pipeline SynapRecorder records the spoken stream, so what Synap remembers matches the conversation the caller experienced.
Tool calls reported while the caller waits. If you have opened the SDK's live stream with await sdk.instance.listen(), the recorder also reports FunctionCallInProgressFrame and FunctionCallResultFrame, tied together by Pipecat's own tool_call_id, and turns go out as stream events. Without a stream, turns are written through sdk.conversation.record_message and tool events are skipped. A turn is never written both ways.
Failure modes that keep the call alive. Networks drop and APIs time out. A read failure costs one turn its memory; a write failure reaches your error handling as an ErrorFrame. A memory layer that takes down the call when it has a bad day is worse than no memory layer at all.
How to Get Started
The whole integration is one install and two processors.
Step 1: Install
pip install maximem-synap-pipecatThis pulls in maximem-synap and pipecat-ai. The import name is synap_pipecat, and the SDK itself is imported from maximem_synap.
Step 2: Add the memory processors to the pipeline
import os
import uuid
from pipecat.pipeline.pipeline import Pipeline
from pipecat.processors.aggregators.llm_context import LLMContext
from pipecat.processors.aggregators.llm_response_universal import (
LLMContextAggregatorPair,
)
from maximem_synap import MaximemSynapSDK
from synap_pipecat import SynapMemoryProcessor, SynapRecorder
sdk = MaximemSynapSDK(api_key=os.environ["SYNAP_API_KEY"])
async def build_pipeline(transport, stt, llm, tts, caller_id: str) -> Pipeline:
await sdk.initialize()
context = LLMContext(
messages=[{"role": "system", "content": "You are a helpful assistant."}]
)
user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context)
# Read path: inject caller memory into the shared context on each transcription.
memory = SynapMemoryProcessor(
sdk, user_id=caller_id, context=context, mode="fast"
)
# Write path: record both turns at the end of each LLM response.
# The conversation id must be a UUID.
recorder = SynapRecorder(
sdk,
user_id=caller_id,
conversation_id=str(uuid.uuid4()),
context=context,
)
return Pipeline(
[
transport.input(),
stt,
memory, # before the user context aggregator
user_aggregator,
llm,
tts,
transport.output(),
assistant_aggregator,
recorder, # after the assistant context aggregator
]
)</code></pre><p><strong>Step 3: Check three things before you deploy</strong></p><p>Call <code>await sdk.initialize()</code> before the pipeline runs. The read path degrades silently by design, so an SDK that was never initialized produces a log line and an agent with no memory, not an exception.</p><p>Pass your own UUID as <code>conversation_id</code>. Left unset, the recorder generates an id of the form <code>pipecat-<hex></code>, and the current SDK validates conversation ids as UUIDs before it writes a message.</p><p>On a B2B instance, add <code>customer_id=</code> to both processors. On a B2C instance, leave it out.</p><p>Full config, scoping rules, and error handling: see the integration docs (<a target="_blank" rel="noopener noreferrer nofollow" class="text-blue-400 underline hover:text-blue-300" href="https://docs.maximem.ai/integrations/pipecat">https://docs.maximem.ai/integrations/pipecat</a>)</p><hr><h2>Memory Is Infrastructure</h2><p>Pipecat gives pipeline developers a framework for realtime voice and multimodal composition, and the context layer it ships handles one run well. Carrying that context from one call to the next, deciding what is worth keeping from a transcript and finding it again later is a different layer of the stack.</p><p>Teams that put voice agents in front of repeat callers reach this layer sooner or later. They either build the memory infrastructure themselves, or they plug in a system built for the problem.</p><p>Memory is infrastructure, not a feature.</p><p>Start building Pipecat voice agents that remember across calls (<a target="_blank" rel="noopener noreferrer nofollow" class="text-blue-400 underline hover:text-blue-300" href="https://synap.maximem.ai">https://synap.maximem.ai</a>)</p><p>Synap pricing is usage-based. You pay for memory operations: storage, retrieval, compaction. No per-seat or per-framework surcharge. The $49/month starter plan includes a base allocation; usage beyond that is metered by operation. Every new account gets $25 in free credits to test before committing. See the full pricing page (<a target="_blank" rel="noopener noreferrer nofollow" class="text-blue-400 underline hover:text-blue-300" href="https://synap.maximem.ai/pricing">https://synap.maximem.ai/pricing</a>).</p><hr><h3>Related Posts</h3><ul><li><p><a target="_blank" rel="noopener noreferrer nofollow" class="text-blue-400 underline hover:text-blue-300" href="/blog/what-is-agentic-context-management">What Is Agentic Context Management?</a></p></li><li><p><a target="_blank" rel="noopener noreferrer nofollow" class="text-blue-400 underline hover:text-blue-300" href="/blog/real-cost-diy-agent-memory">The Real Cost of DIY Agent Memory</a></p></li><li><p><a target="_blank" rel="noopener noreferrer nofollow" class="text-blue-400 underline hover:text-blue-300" href="/blog/skills-new-microservices">Skills Are the New Microservices</a></p></li></ul>
