Overfitting and Generalization
Intuition
Overfitting resembles memorizing answer keys: almost every training question is right, but a differently worded question fails. Generalization learns a pattern that transfers to new examples. A common signal is that training score continues rising while validation score plateaus or falls.
Why overfitting happens
- Few examples and many features give a model opportunities to memorize accidental details.
- Duplicate or near-duplicate examples make training performance appear excellent.
- Model capacity is too large relative to the task.
- A team can “overfit the validation set” by repeatedly tuning for the same one.
- Labels contain noise, so the model begins fitting wrong or inconsistent answers.
Common controls
- More reliable data: expand coverage, deduplicate, and resolve rule conflicts. This is often more fundamental than changing models.
- Regularization: impose a cost on excessive weights or complex structures.
- Reduce capacity: decrease tree depth, feature count, parameters, or training rounds.
- Early stopping: stop when validation performance no longer improves.
- Strict evaluation: group by user/document/time and retain a genuinely unseen test set.
Bias–variance is a thinking framework: a model that is too simple has large systematic error (high bias); a model overly sensitive to small training-data changes is unstable (high variance). In reality, also account for data drift and irreducible label ambiguity.
Special forms in Agent/RAG work
- Different chunks from the same document enter both training and test.
- An evaluation question is generated by a model from an answer, carrying answer phrasing into the question.
- Prompt tuning watches one fixed set of sample questions for too long, so the result works only on them.
- A routing model treats customer name as an intent shortcut and collapses on a new customer.
Error-investigation order
- First check duplicates, leakage, and split unit.
- Compare training and validation scores, including subgroup slices.
- Inspect whether error examples have inconsistent labels.
- Only then try regularization, feature changes, or model-complexity changes.
- Confirm with an independent test set and out-of-time data.
Reconfirm generalization after deployment
Training/validation/test boundaries answer questions about one historical snapshot only. After deployment, input-feature distributions, class proportions, label definitions, human processes, or available fields can all change, making an old score unrepresentative. You can monitor input, predictions, delayed actual outcomes, and human-escalation rate by time window, source, business setting, and error cost. An alert is a starting point for investigation, not automatic proof that “the model got worse” or “retraining must begin now.” First investigate data quality, rule/label changes, product workflow, and population change, then decide whether to add data, adjust workflow, reevaluate, or train a new version.
Exercise
Write an anti-overfitting plan for repeatedly tuning prompts against a fixed set of 100 Agent evaluation questions. It must include a development set, sealed test set, version record, and a policy for new failure examples.
Then complete two comparisons:
- Sketch training/validation scores versus model complexity, marking underfitting, a suitable range, and overfitting.
- Explain separately what “add high-quality independent examples,” “duplicate existing examples,” and “increase regularization” change. Do not treat the three as equivalent operations.
Mastery checklist
- I can identify an overfitting candidate from a train/validation gap, but do not conclude from one curve alone.
- I investigate duplicates, leakage, grouping unit, and label noise before changing models.
- I know what regularization, early stopping, and reduced capacity each change.
- I protect a sealed test set and avoid team-level overfitting to a fixed validation set.
Next: Metrics, Baselines, and Error Analysis.
References
Review date: 2026-07-22.