MAE vs RMSE: which error metric should you use?

One example, two answers

A model predicts delivery times for five orders. Its errors are 2, 3, 2, 4 and 24 minutes: four small misses and one large one.

• MAE (mean absolute error) averages the size of the errors: (2 + 3 + 2 + 4 + 24) ÷ 5 = 7 minutes.
• RMSE (root mean squared error) squares the errors, averages them, then takes the square root: √((4 + 9 + 4 + 16 + 576) ÷ 5) ≈ 11 minutes.

Squaring makes the 24-minute miss dominate. RMSE is always at least as large as MAE, and the gap between them tells you something: a large gap means a few big misses; a small gap means the errors are similar in size.

import numpy as np
from sklearn.metrics import mean_absolute_error, mean_squared_error

actual = np.array([30, 25, 40, 35, 20])
predicted = np.array([32, 28, 38, 31, 44])   # errors of 2, 3, 2, 4 and 24 minutes

print("MAE: ", mean_absolute_error(actual, predicted))             # 7.0
print("RMSE:", round(np.sqrt(mean_squared_error(actual, predicted)), 1))   # 11.0

Which one to use

• Use MAE when every unit of error costs about the same, such as minutes of waiting or dollars of difference. It’s also the easiest to explain: “on average, we’re off by 7 minutes.”
• Use RMSE when big misses are much worse than small ones, such as a stock forecast where running out costs far more than being slightly over. It pushes the model to avoid large errors.
• Report both when you’re not sure. If RMSE is much larger than MAE, look at the biggest misses: they’re often unusual cases, data errors or a group the model handles badly.

Two cautions

First, both are in the units of what you predict, so they can’t be compared across problems: an MAE of 7 is excellent for house prices in dollars and terrible for exam scores out of 10. Compare against a baseline on the same data instead, such as always predicting the median.

Second, when the values range widely, such as products that sell 2 units a week next to ones that sell 2,000, a single MAE is dominated by the large ones. A percentage-based measure such as MAPE or WAPE, or reporting the error per group, gives a fairer picture.