Lesson 7 of 8
Picking the right metric
In plain words: The metric should match what a mistake costs in real life.
- Classification: precision when false alarms are costly, recall when misses are costly, F1 score for a balance.
- Imbalanced classes: avoid plain Accuracy; use ROC-AUC or F1.
- Numbers: MAE is easy to explain; RMSE punishes big misses.
- Forecasts: MAPE or MAE with backtesting.
- Chatbots / RAG: answer correctness and groundedness on a golden set.
Tip Always report your metric next to the baseline's, so people can see the improvement.