Overfitting and regularization

Choose your learning level

BeginnerIntermediateAdvanced

Start with plain-language explanations. Switch levels at any time.

Beginner explanation

Overfitting happens when a model learns quirks of its training examples that do not carry over to new data. Memorizing practice answers is not the same as learning the subject. Separate evaluation examples help reveal the difference.

Intermediate explanation

Watch both training and validation performance. Regularization, data augmentation, reduced model capacity and early stopping can help, depending on the task. Validation data guides choices; the final test set should remain untouched until those choices are fixed.

Advanced explanation

L2 regularization penalizes large parameter norms, while dropout randomly masks selected activations during training. These change optimization and effective capacity; neither repairs leaked labels or an unrepresentative test set. Repeated test-set tuning contaminates the evaluation.

PUT IT IN CONTEXT

A worked example

A model gets 99% training accuracy but 65% on new cases. Adding more epochs may widen that gap. Check leakage, labels and split quality, then compare a simpler model and a validation-based stopping point.

TRY IT YOURSELF

Practice the idea

You used the test set to choose among 40 model variants. Can you still present its best score as an untouched final evaluation?

Your working notes (not saved or sent)

Reveal the answer guide

No. The test set influenced selection. Treat it as part of development and use a fresh, appropriate held-out evaluation set if possible; otherwise disclose the limitation.

Go further: advanced challenge

Explain why lower training accuracy after regularization can accompany higher validation accuracy. Include the role of model capacity.

An open-ended discussion task. Explain your assumptions and evidence; there may be more than one defensible answer.

Continue the conversation

What part of this idea would you like to test or question? Compare your reasoning with the community.

Find related forum discussions ↗

Opens a topic search in the existing AstraCruz forum. Results depend on available posts; signing in may be required. No discussion or activity is fabricated.

Browse all forums