AI & LLM Glossary
Clear, practical definitions of AI concepts, from context windows to agentic memory. Built for engineering and product teams working with LLMs.
For the argument rather than the definitions, read the memory problem.
Showing 145 of 145 terms
A
Adherence (Instruction Adherence)
Measuring how well AI systems follow the specific instructions and constraints provided by users
Agent Observability
Monitoring and tracking what autonomous AI agents are doing in real-time across distributed systems.
Agent Skills
Folders of markdown instructions an agent loads on demand to change how it performs a task, rather than what it knows.
Agentic AI
AI systems that plan, call tools, evaluate the results, and decide what to do next, rather than producing a single response and stopping.
Agentic Memory System
A comprehensive memory framework for AI agents that maintains episodic, semantic, and procedural memory to enable learning and continuous improvement.
AI Access Control
Systems determining who can use AI models, which data they can access, and what they're allowed to do.
AI Agent
An autonomous system that perceives its environment, makes decisions, and takes actions to achieve specific goals without direct human intervention.
AI Auditability
The ability to create a complete record of what an AI system did, why it did it, and what inputs influenced its outputs.
AI Bias
Systematic skew in model outputs that favors or disadvantages certain groups or answers, caused by patterns in training data, context, or system design.
AI Cost Model
The framework for understanding and predicting how much your AI system will cost to operate at different scales.
AI Data Governance
Policies and systems controlling what data goes into AI models, how it's used, and who can access it.
AI Plugins
Packaged extensions that give an AI assistant new abilities at runtime, spanning the original ChatGPT plugins through to MCP servers, agent skills, and apps.
AI Traceability
The technical capability to follow an input through every transformation until it produces output, showing what influenced the result.
AI Vendor Lock-In Risk
The danger of becoming dependent on a specific AI provider's models or infrastructure, making it costly to switch.
Alignment
Ensuring AI systems behave in accordance with human values, goals, and constraints
Alignment Evals
Testing whether AI system behavior aligns with specified goals, values, and constraints
API (Application Programming Interface)
A defined contract that lets one system call a specific function on another, with agreed inputs, outputs, and errors.
Attention Mechanism
The operation that lets a model weigh every token against every other token, deciding what in the input matters for predicting what comes next.
Audit Log
Comprehensive records of AI system decisions, actions, and state changes for accountability and compliance
Autoresearch
Agents that run the research loop themselves, planning, searching, reading, synthesizing, and verifying until a question is actually answered
B
Backpropagation
The algorithm that computes how much each parameter contributed to a model error, by applying the chain rule backwards through the network.
Behavioral Evals
Testing how AI systems behave under various conditions, focusing on consistency and patterns rather than single outputs
Benchmark
A standardized public test set used to compare models against each other, which is a genuinely different job from evaluating whether your system works.
C
Chain-of-Thought (CoT)
A prompting technique that makes LLMs show their reasoning step-by-step, improving accuracy especially on complex reasoning tasks.
Chunking
The process of dividing long documents into smaller pieces for RAG systems to store and retrieve efficiently.
Code Agent
AI agents specialized in writing, analyzing, and executing code to solve problems programmatically
Compliance
Ensuring AI systems adhere to applicable laws, regulations, industry standards, and ethical guidelines.
Context Compression
Reducing the token size of context information while preserving critical details and meaning
Context Engineering
The discipline of deciding what information goes into a model's context window, in what order, and in what form, drawn from a much larger pool of available information.
Context Eviction
Removing or deprioritizing old information from the AI's active context to make room for new data
Context Management
Strategically selecting what information an AI system should consider in each interaction
Context Retrieval
Fetching relevant past information or memories to include in current AI processing
Context Rot
Degradation of memory quality and accuracy as stored context becomes outdated or semantically disconnected
Context Window
The maximum amount of text an LLM can consider at once, measured in tokens.
Coordination Protocol
The rules and standards enabling multiple AI agents to work together, share information, and synchronize actions.
Cost-to-Completion
The total cost, in money or tokens, required to accomplish a task using AI, from initial attempt to satisfactory result.
Cross-Encoder Scoring
Using transformer models to score query-document pairs directly rather than encoding them separately
Customization
Tailoring AI systems to specific organizational needs, preferences, and constraints without rebuilding from scratch.
D
Data Catalog
An inventory of the datasets an organization holds, recording what exists, who owns it, where it lives, and what it contains.
Data Lineage
The record of where data came from and how it was transformed on the way to where it is now.
Data Observability
Continuous monitoring of whether data is healthy right now: freshness, volume, schema drift, distribution shifts, and anomalies.
Data Product
Data packaged with a contract: defined schema, named owner, service levels, access policy, and documentation, so consumers can depend on it.
Data Sovereignty (AI Context)
The principle that data, especially when used in AI systems, should remain under the control and jurisdiction of its origin country or organization.
Dataset
A collection of raw or lightly processed data, defined by its contents rather than by any contract about quality, ownership, or support.
Delegation
Agents assigning subtasks to other agents or systems, breaking complex problems into manageable pieces
Dense Retrieval
Using learned embeddings to retrieve information based on semantic similarity rather than keyword matching
Deployment
The process of taking a trained AI model or application from development into production where it serves real users.
Developer Agents
AI agents designed to autonomously write, test, debug, and optimize code, assisting software engineers in development tasks.
Distributed Systems
Computing architectures where AI systems are spread across multiple machines or locations, enabling scale, reliability, and geographic distribution.
Document Ranking
Sorting retrieved documents by relevance to the query using scoring or learning-to-rank models
E
Embedding Drift
Changes in embedding model output distributions or quality over time, degrading retrieval performance
Embeddings
Numerical representations of text that capture semantic meaning, enabling AI systems to understand similarity and relationships.
Emergent Behaviors
Complex system behaviors that arise unexpectedly from simpler components interacting, not explicitly programmed
End-to-End Eval
Evaluating complete AI system performance across entire workflows rather than isolated components
Enterprise Agents
AI agents deployed in organizations to autonomously execute business processes and complete multi-step tasks under organizational control.
Enterprise AI Stack
The complete set of components an enterprise organization needs to build, deploy, and manage AI systems in production.
Enterprise Framing
How to position and communicate AI capabilities to enterprise organizations by emphasizing control, governance, and business value.
Enterprise Governance
The organizational frameworks, policies, and oversight mechanisms that ensure AI systems are used appropriately and comply with requirements.
Enterprise Memory & AI Systems
Persistent memory infrastructure that lets AI systems learn from past interactions and deliver personalized, context-aware experiences at scale.
Enterprise Metrics
The suite of quantitative measurements organizations use to assess whether AI systems are delivering business value and operating as intended.
Enterprise Procurement
The organizational and contractual processes large companies use to evaluate, approve, and purchase AI systems and services.
Enterprise Workflows
Structured, automated processes within organizations that incorporate AI to automate decision-making, task routing, and multi-step operations.
Episodic Memory (AI)
AI systems storing specific experiences and interactions in chronological context with sensory/contextual details
Evals (Evaluation Systems)
Systematic testing frameworks that measure AI system quality across multiple dimensions like accuracy, safety, and efficiency
Event Loop (Agent Runtime)
The core execution mechanism that cycles through agent decision-making, tool execution, and state updates
Explainability
Making AI decisions and outputs interpretable to humans, showing why the system generated specific responses
F
Failure Modes
Systematic ways AI systems malfunction or produce wrong outputs, including hallucinations, biases, and reasoning errors
Feedback Loop (Agentic)
Mechanisms where AI agents observe outcomes of their actions and adjust subsequent behavior based on results
Fine-Tuning
Training a pre-trained LLM on domain-specific data to improve performance on specialized tasks without rebuilding from scratch.
Formal Verification
Mathematically proving that a system satisfies a specification for every possible input, rather than testing it on the inputs someone thought of.
G
Generative AI
AI that produces new artifacts (text, images, code, audio) in response to a prompt, rather than classifying or predicting from existing data.
Governance
Systems and processes that establish rules, make decisions about how something should be run, and ensure compliance with those rules.
Gradient Descent
The optimization algorithm behind nearly every modern AI model, nudging parameters downhill along the error surface one small step at a time.
Graph Engineering
Designing and maintaining the entity-and-relationship layer that AI systems traverse, covering schema, extraction, resolution, provenance, and pruning
Graph RAG
Using knowledge graphs to structure and retrieve information instead of flat vector databases
Groundedness Evals
Testing whether AI outputs are factually supported by source materials and not hallucinated
Grounding
The property of a model output being verifiably tied to a trusted source, rather than generated from parametric memory alone.
Guardrails
Safety mechanisms that constrain AI output to acceptable ranges and prevent harmful or out-of-scope responses
H
Hallucination
When an LLM generates plausible-sounding but false, misleading, or fabricated information with high confidence.
Hallucination Mitigation via Retrieval
Using external knowledge sources and retrieval-augmented generation to ground AI outputs in factual information
Hallucination Rate
Quantitative measurement of how often an AI system produces factually incorrect or unfounded outputs
Harness
The code wrapped around a model, the loop, the tools, the context assembly, the permissions, and the verification, that turns raw inference into a working agent
Harness Engineering
The discipline of designing, measuring, and tuning the scaffolding around a model, treating the agent loop as software rather than as a prompt
Human-in-the-Loop (HITL)
Systems where humans and AI collaborate, with humans reviewing and correcting AI decisions to improve quality
Hybrid Search
Combining semantic similarity search with keyword/lexical matching for more robust information retrieval
I
Inference
The process of running a trained model to generate outputs from input data, as opposed to training which creates the model.
Integrations
Connections between AI systems and external services, data sources, and business tools that enable the AI to access information and take actions.
K
Knowledge Graph
A structured representation of entities, relationships, and attributes that captures domain knowledge as interconnected nodes and edges.
Knowledge Storage
Infrastructure for persisting and organizing information that AI systems retrieve and reason about
Knowledge Systems
Infrastructure that stores, organizes, and retrieves structured and unstructured information to support AI reasoning and decision-making.
Knowledge Work
Jobs centered on creating, analyzing, and applying information rather than physical tasks
KV Cache
The stored attention keys and values from tokens a model has already processed, so it can generate each new token without recomputing the entire prompt.
L
Latency Optimization
Reducing response time for AI systems through caching, batching, model optimization, and infrastructure tuning
Lifecycle (Model Lifecycle)
The complete journey of an AI model from conception through development, testing, deployment, monitoring, retraining, and eventual retirement.
LLM (Large Language Model)
A neural network trained on massive amounts of text data to predict and generate human-like text, often used as the reasoning engine for AI applications.
LLM-as-a-Judge
Using a language model to score another model output against a rubric, for the many qualities that have no automatic metric and too much volume for human graders.
Long-Term Memory (AI)
Persistent storage of information that an AI system can access across sessions and conversations, enabling it to learn from and recall past interactions.
LoRA (Low-Rank Adaptation)
A fine-tuning method that freezes the base model and trains a small pair of low-rank matrices per layer, cutting trainable parameters by orders of magnitude.
Loss Function
The single number a model is trained to minimize, which makes it the real specification of what the model is optimizing for.
M
Maintenance
The ongoing operations and updates required to keep AI systems running effectively, including monitoring, bug fixes, updates, and performance optimization.
MCP (Model Context Protocol)
An open standard for exposing tools, data, and context to AI models, so any compliant model can discover and use any compliant integration.
Mechanistic Interpretability
Reverse-engineering the internal computations of a neural network into human-readable algorithms, features, and circuits
Memory & Optimization
Strategies for managing and persisting information about users, interactions, and context to improve AI performance while maintaining efficiency and privacy.
Memory & Personalization
Tailoring AI responses and memory retrieval based on individual user preferences, history, and behavior patterns
Memory Consolidation
Processing and integrating new experiences into organized long-term memory structures for persistent learning
Model Routing
Systems that intelligently direct requests to different AI models based on criteria like cost, latency, accuracy, or specialization.
Multi-Agent Systems
Architectures where multiple AI agents work together, often with different roles or specializations, to solve complex problems collaboratively.
Multimodal AI
AI systems that can process multiple types of input (text, images, audio, video) and reason across them within a single model.
O
Observability
Monitoring and understanding AI system behavior through logs, metrics, and traces to detect problems
Orchestration
Coordinating multiple AI models, tools, and systems to work together in complex workflows
Orchestration Layer
The infrastructure that coordinates and manages multiple AI models, services, data sources, and tools within an AI system.
P
Personalization Engine
Systems that customize AI outputs, recommendations, and experiences based on individual user preferences, behavior, and characteristics.
Policy Engine
Systems that enforce organizational rules and constraints on AI behavior, including access control, content filtering, and decision approval.
Prompt Caching
Reusing a model provider stored attention state for a repeated prompt prefix, cutting latency and input cost on every request that shares it.
Prompt Engineering
The practice of crafting specific input text (prompts) to guide LLMs toward producing desired outputs with improved quality and consistency.
Prompt Injection
A security vulnerability where user input or untrusted data manipulates an AI model's behavior by injecting instructions into the prompt.
Prompt Template
Reusable prompt structures with placeholders that enable consistent, parameterized interactions with AI models across different inputs.
R
RAG (Retrieval-Augmented Generation)
A technique that retrieves relevant information before generating responses, making LLMs more accurate and factual.
RAG Pipeline
The complete workflow of retrieving relevant documents or data and providing them to an AI model to ground its responses in external knowledge.
Rate Limiting
Mechanisms that restrict how frequently users or systems can call AI APIs or services to prevent overload, control costs, and ensure fair usage.
Red Teaming
Adversarially attacking your own AI system to find the failures that ordinary testing and ordinary users will never surface.
Reranking
Reordering search results using more sophisticated models or signals after initial retrieval
Retrieval Pipeline
The technical infrastructure that searches, ranks, and retrieves relevant information from knowledge bases or documents to support AI systems.
RLHF (Reinforcement Learning from Human Feedback)
The post-training stage that turns a raw next-token predictor into a usable assistant by optimizing against human preferences between outputs.
S
Safety Filters
Systems that detect and prevent AI models from producing harmful, unethical, or inappropriate content before it reaches users.
Sandboxing
Running agent-generated code and tool calls inside an isolated environment that has no more access than the specific task requires.
Semantic Layer
A shared definition layer that turns raw tables into consistent business concepts, metrics, and relationships that anyone can query.
Semantic Search
Search that understands meaning rather than matching keywords, retrieving results based on conceptual similarity rather than exact word matches.
Session Management
Systems that maintain and manage conversation context, user state, and history across multiple interactions with an AI system.
SKILL.md
The markdown file at the heart of an Agent Skill: YAML frontmatter the agent always sees, and a body it loads only when it decides the skill applies.
Sparse Retrieval
Retrieval methods that use explicit keywords and term matching to find relevant documents, contrasting with semantic similarity-based approaches.
State Management (Agent)
Systems that track and maintain the current status, progress, and internal variables of AI agents as they work through multi-step tasks.
Structured Output
Constraining AI model outputs to specific, machine-parseable formats (JSON, XML, etc.) instead of free-form text.
System Prompt
Initial instructions provided to an AI model that define its role, behavior, constraints, and how it should respond to users.
T
Temperature (LLM)
A parameter controlling randomness in model outputs, from deterministic (0) to highly creative/random (1+), with different optimal values for different tasks.
Token Budget
The total allocation of tokens (units of text cost) available for an AI system, used to constrain spending and optimize resource allocation.
Tokenization
The process of breaking text into small pieces (tokens) that LLMs process, where tokens are not always whole words.
Tool Use (Function Calling)
Enabling AI models to call external functions or APIs to access information and take actions
Transformer Architecture
The underlying neural network structure used by modern large language models, based on self-attention mechanisms for processing sequential data.
W
Warm-Up (Model)
Pre-loading and initializing AI models before serving requests to reduce latency and improve response times for users.
Workflow Automation
Using AI to automatically execute multi-step business processes, reducing manual work and enabling faster, more consistent operations.