Bias is systematic error. That word is doing the work: a model that's wrong randomly is inaccurate, while a model that's wrong in the same direction every time for the same kind of input is biased. The distinction matters operationally, because random error averages out across many decisions and systematic error compounds. Ten thousand resume screenings with random noise is roughly fair. Ten thousand with a two-point skew against one group is a pattern with legal consequences.
It's worth separating this cleanly from hallucination, since the two get lumped together as "AI is unreliable." A hallucination is a fabrication with no basis, and it's usually detectable by checking the claim against a source. Bias is often factually defensible output that is nonetheless skewed in aggregate, and it's invisible in any single response. You cannot find it by reading outputs one at a time, which is exactly why it survives review processes that catch hallucinations easily.
The sources stack up at every layer. Training data reflects the world that produced it, including its historical inequities, and internet-scale corpora overrepresent some languages, regions, and demographics dramatically. Human feedback during alignment encodes the preferences of whoever did the labeling. The system prompt encodes the assumptions of whoever wrote it. And retrieval introduces its own: if your knowledge base is 80% documentation from one product line, the model will over-apply that product's conventions everywhere, which is bias arising purely from context assembly rather than from anything in the weights.
That last source is the one teams building on top of foundation models actually control, and it's underappreciated. You may not be able to retrain the model, but you chose the documents, the retrieval strategy, the examples in the prompt, and which past interactions get remembered. A few-shot prompt with five examples drawn from one customer segment will skew behavior for every other segment. Memory systems that preferentially recall recent or frequent interactions will drift toward the majority use case and get quietly worse at the long tail.
Detection requires deliberate measurement, because nothing surfaces it automatically. The workable approach is paired testing: construct inputs identical except for the attribute you're worried about, run both at volume, and compare outcome distributions rather than individual responses. Then segment your production evals so quality is reported per group instead of as a single aggregate, since an overall 92% accuracy can easily hide 97% for one segment and 74% for another.
Mitigation is real but partial, and honesty about that matters. You can rebalance training and retrieval corpora, diversify few-shot examples, add explicit instructions about relevant attributes, apply post-hoc fairness constraints, and keep humans in the loop for consequential decisions. What you cannot do is declare a system unbiased. Fairness has multiple mathematical definitions that provably cannot all hold at once, so the practical goal is to state which definition you're optimizing for, measure against it continuously, and document the tradeoff you accepted.
Why It Matters
Bias is where AI deployment meets regulation and liability. Hiring, lending, insurance, healthcare, and housing decisions are governed by rules that don't care whether a human or a model produced the disparity. Beyond compliance, bias degrades products in ways that show up as unexplained quality gaps: a support system that works well for one customer segment and poorly for another looks like a mysterious accuracy problem until someone thinks to segment the metrics.
Example
A company uses an LLM to summarize performance reviews. Aggregate quality scores look fine. A paired test (same review text, only the employee name changed between two demographically-associated sets) finds the summaries systematically use more achievement-oriented framing for one set and more effort-oriented framing for the other. No single summary is factually wrong, none would fail a hallucination check, and the pattern is only visible across a few hundred paired samples. The fix combines an explicit instruction to summarize only stated outcomes, a redaction step that removes names before summarization, and an ongoing paired eval in CI.