LLM Cost & Production
Running LLM applications and AI agents in production: token cost, model routing, latency, caching, rate limits and reliability.
An agent that costs cents in a demo can cost dollars per conversation in production, because retries, tool calls, long system prompts and ever-growing context multiply the tokens behind every answer. Cost, latency and reliability are one problem seen from different sides, and the fixes, from routing easy turns to cheaper models to trimming context, tend to help all of them at once.
These articles break down where agent spend actually goes and how to control it without making the agent worse.
Start here: Total cost of ownership calculator · Token budget (glossary)
Which LLM Should Write Your AI Agent's Memory? How to Choose and Test a Memory Extraction Model
Which LLM to use for memory extraction: in Mem0's October 2026 test of 17 writer models on LoCoMo, the writer alone moved accuracy from 84.1% to 93.5% and cost from $0.68 to $158.55 per 1,000 writes, yet the top six sat within 0.9 points. How to test writer models on your own transcripts, the failure modes to measure (rules written as memory, over-extraction, missed updates), and where a small judge model fits with a type-aware write policy.
Memory for Background Agents: Where It Lives When the Sandbox Is Discarded, What to Remember With No User in the Loop, and Who May Write or Forget
A background agent wakes on a cron, a heartbeat or an event, runs in a sandbox that is thrown away afterwards, and keeps only what it wrote somewhere else. Where that memory has to live (outside compute, scoped to the user and the agent, loaded at the start of every run), what to remember when no user is there to confirm anything, how async subagent results get written back without duplicates, and who may write to and delete from a memory several bots share.
LLM API Pricing Comparison 2026: Every Major Provider per Million Tokens, Cached Input, and What a 40-Turn Conversation Costs
LLM API pricing compared as of 30 October 2026: the flagship and cheap model of OpenAI, Anthropic, Google, xAI, DeepSeek, Mistral, Moonshot, Alibaba, Zhipu, Groq, Fireworks and Together per million tokens, with cached input and cache-write rates, the three list prices that repeat across providers, the Gemini rises on 1 December and 1 January, and what the same 40-turn conversation costs on every model, from $0.016 on Qwen3.7-Flash to $5.46 on GPT-6 Astra and Claude Fable 5.1.
Memory Write Policy for AI Agents: What to Remember, When to Overwrite, When to Ask, and What Never to Store, by Memory Type
A memory write policy decides, before anything persists, whether an AI agent may store a candidate memory, and the rules have to differ by type: facts supersede with history, preferences need a direct statement, values and beliefs need the highest bar and a confirmation, instructions may only come from the user or operator, and episodes append and expire. A policy table, the evidence for per-type thresholds, and a tested Python write gate.
OpenRouter Pricing 2026: Credit Fees, BYOK, Free Model Limits, and What Routing Costs Next to Going Direct
OpenRouter pricing as of 29 October 2026: model tokens at the provider's list price, a 5.5% fee on credit purchases on Standard ($0.80 minimum by card) and 8% on Business, BYOK free up to $25,000 of list-price inference a month and 5% after that, free models at 20 requests a minute and 50 or 1,000 a day, and what a 40-turn conversation costs through OpenRouter next to going direct to Anthropic, Google, DeepSeek, Mistral, Moonshot and Groq.
Agent Memory Consolidation: What a Nightly "Dreaming" Job Should Do to Your Agent's Memory, and How to Check It Did
Agent memory consolidation is a background job that merges duplicate memories, supersedes replaced facts, expires time-bound ones and promotes repeated episodes into durable facts. Run it on a copy, verify the copy against invariants and known answers, then swap. How Anthropic Dreams, Letta and Mem0 Dream do it, the failure modes of unattended runs, and a tested Python job.
Fireworks AI vs Together AI Pricing 2026: Same Models Side by Side, Cached Input, Batch, GPUs, Fine-Tuning and Cost per Conversation
Fireworks AI vs Together AI pricing as of 29 October 2026: identical list prices on Kimi K3, GLM 5.3, DeepSeek V4.1 Flash, Muse Spark Open and four more shared models, but GPT-OSS 120B cached input at $0.015 on Fireworks and no cached rate on Together, a 50% batch discount on Fireworks against one discounted batch model on Together, H100s at $8.00 an hour against $5.49, cheaper fine-tuning on Together, a $1 credit against a $15 minimum, and what a 40-turn conversation costs on each.
GLM 5.2 API Pricing 2026: GLM-5.2, GLM-5.3 and Every Z.ai Model per Million Tokens, Cache, Free Models, Coding Plan and Cost per Conversation
GLM 5.2 API pricing as of 29 October 2026: $1.40 input, $0.26 cached input and $4.40 output per million tokens on Z.ai, the same as the newer GLM-5.3 that replaced it as Zhipu's flagship in August. Every current GLM model in USD and on bigmodel.cn in CNY, free Flash models, how the automatic cache bills, the Coding Plan's credits and what they mean for API users, open-weight hosts from Together to DeepInfra, and what a 40-turn conversation costs next to Kimi, Qwen, DeepSeek, Groq, Gemini and Claude.
Qwen API Pricing 2026: Every Qwen Model per Million Tokens on Alibaba Cloud, by Region, Free Quota, Cache, Batch and Cost per Conversation
Qwen API pricing per million tokens as of 29 October 2026: Qwen3.8-Max at $2 input and $6 output on Alibaba Cloud Model Studio's Singapore region and $1.65 and $4.95 on Global-scope rows, Qwen3.7-Plus at $0.40 and $1.60 (20% off for now), Qwen3.8-Flash at $0.15 and $0.47, Qwen3.7-Flash from $0.03 and $0.13, 1 million free tokens per model, context tiers, cache and Batch rules, CNY prices, where open-weight Qwen runs cheaper, and what a 40-turn conversation costs next to DeepSeek, Groq, Gemini and Claude.
AI Agent Deprovisioning: Revoking Access Does Not Erase What the Agent Remembered
Revoking an AI agent's OAuth token, removing a tool permission or offboarding a user stops the next read, but the summaries, embeddings, cached tool results and shared team memory the agent already wrote stay readable. What an offboarding checklist has to cover for memory: which of three revocation events you are handling, memory scoped to the credential it came through, provenance on every derived fact, permission checks at read time, purge on revocation, and an audit record that proves it.
Kimi K3 API Pricing 2026: K3, K3.1 Pro and Every Kimi Model per Million Tokens, Cache Writes, Limits and Cost per Conversation
Kimi K3 API pricing as of 30 October 2026: $3.00 input, $0.30 cached input and $15.00 output per million tokens on Moonshot's platform, while the newer Kimi K3.1 Pro costs less at $2.00, $0.20 and $10.00. Every current Kimi model in USD and CNY, the new 5-minute and 1-hour cache writes, rate-limit tiers, what Together, Fireworks, DeepInfra and OpenRouter charge, and what a 40-turn conversation costs next to DeepSeek, Mistral, Groq, Gemini and Claude.
How Do You Prevent Memory Poisoning Attacks on AI Agents?
How to prevent memory poisoning in AI agents: the five ways poisoned content gets into memory, the published attacks on ChatGPT, Bedrock Agents and Claude Code, what OWASP ASI06 requires, eight defences from write-path validation to rollback, which ones still hold when the model is fooled, and a test plan you can run in CI.
How Do You Make AI Agent Memory GDPR Compliant? Erasure That Reaches Derived Facts, Embeddings and Backups
AI agent memory is personal data, so GDPR applies to every fact, summary and embedding it keeps. What compliance takes in practice: a lawful basis per memory purpose, minimisation on the write path, retention limits, erasure that reaches derived facts, embeddings and backups, access and portability exports, processor contracts with the memory vendor and the LLM provider, transfer safeguards, a DPIA where risk is high, and where the EU AI Act fits after the July 2026 changes. Read against the GDPR text and EDPB guidance on 28 October 2026.
Mistral API Pricing 2026: Every Model per Million Tokens, Free Mode, Batch, Priority and Cost per Conversation
Mistral API pricing per million tokens as of 30 October 2026: Mistral Small 4 at $0.15 input and $0.60 output, Large 3 at $0.50 and $1.50, Large 4 at $0.70 and $2.10 after a price cut, Medium 3.5 at $1.50 and $7.50, cached input at 10%, Batch at 50% off, Priority at 1.75x, EU and US endpoints at 1.1x, what Free mode includes, and what a 40-turn conversation costs next to DeepSeek, Groq, Gemini and Claude.
What Breaks When You Build Your Own Agent Memory? Nine Failure Modes and How to Detect Each
When you build your own agent memory, the write path breaks first: unresolved contradictions, stale facts, duplicates that outvote corrections, entity confusion, deletes that miss derived copies, cross-user leakage and write cost. What each looks like in production, what causes it, and the signal that detects it, grounded in STALE, FEB-30 and HaluMem.
Groq API Pricing 2026: Every GroqCloud Model per Million Tokens, Free Tier Limits, Batch and Cost per Conversation
Groq API pricing per million tokens as of 27 October 2026: GPT-OSS 120B at $0.15 input and $0.60 output, GPT-OSS 20B at $0.075 and $0.30, Qwen 3.8 27B at $0.80 and $4.00, cached input at half price, Batch at 50% off, free and Developer plan limits, the Llama models that moved to enterprise, and what a 40-turn conversation costs next to DeepSeek, Gemini and Claude.
DeepSeek API Pricing 2026: deepseek-flash and V4 Pro per Million Tokens, Peak vs Off-Peak, and Cost per Conversation
DeepSeek API pricing per million tokens as of 27 October 2026: deepseek-flash from $0.15 input and $0.60 output off-peak, V4 Pro from $0.66 and $1.98, cache hits from $0.003, the weekday peak hours that double every rate, and what a 40-turn conversation costs next to Claude and Gemini.
Gemini API Pricing 2026: Every Model per Million Tokens, and the Rates That Double in December and January
Gemini API pricing per million tokens for every current model as of 27 October 2026, from Flash-Lite at $0.25 to Gemini 4 Argon at $2 input, the December and January rises that double Argon and 3.8 Flash, and what a 40-turn conversation costs next to Claude and OpenAI.
Prompt Caching in Coding Agents: How Claude Code, Codex CLI, OpenCode and Pi Keep the Prefix Stable
A coding agent sends its entire context again on every request, so prompt caching decides most of what a session costs: a 60-request session on Claude Sonnet 5.5 costs $12.99 with no caching and $1.31 when every request hits the cache.
How to Reduce LLM Token Costs in Long Conversations: What Caching Saves, and Where It Stops
A long conversation costs far more than its number of turns suggests, because every request carries the system prompt and the entire history again, so total input grows with the square of the number of turns.
Claude API Pricing 2026: Every Model per Million Tokens, Cache Read and Write Prices, and Cost per Conversation
Every Claude model's API price per million tokens on one table, as of 27 October 2026, with cache write and read prices, Haiku 5.5's 100K-token tier, the new API credits on Max and Team, and what a 40-turn conversation actually costs.
The Real Cost of DIY Agent Memory
Building agent memory from scratch costs $40K-$120K and 6+ months of engineering time. See the real numbers behind DIY memory systems and when to build vs buy.
Your AI Agent Is A Cash Guzzler. Here's a Framework for Thinking About It.
Most founders misjudge agent costs, focusing only on token price. In reality, stacked expenses from context accumulation and infrastructure can explode bills 10x at scale. Learn to identify the actual growth curve in your billing stack and why smart context management is the only viable path to sustainable unit economics.