The easiest way to read it
Pick one positive case and one negative case at random, such as a customer who left and one who stayed. ROC-AUC is the chance that the model gives the positive case the higher score. An AUC of 0.85 means the model ranks them the right way round 85% of the time.
That’s why AUC doesn’t depend on a threshold: it only looks at the order of the scores. It answers “can this model tell the two groups apart?” before you decide where to draw the line.
What the curve shows
The ROC curve plots two rates as the threshold moves from strict to lenient: the true positive rate (recall: how many real positives are caught) against the false positive rate (how many negatives are wrongly flagged). A useless model follows the diagonal: catching more positives costs just as many false alarms. A good model rises steeply first, catching most positives before many false alarms appear. The area under the curve (AUC) sums that up in one number.
How good is good?
- 0.5: no better than random ordering.
- 0.6 to 0.7: some signal, often useful for ranking a long list.
- 0.7 to 0.85: a solid model for most business problems.
- Above 0.95 on a hard problem: check for data leakage before celebrating.
These are rough guides. What matters is whether the model beats a simple baseline on your data, and whether its ranking helps the decision you’re making.
When AUC can mislead
When positives are very rare, such as 0.2% fraud, AUC can look excellent while the model still produces many false alarms for every real case. That’s because the false positive rate is divided by a huge number of negatives. For rare events, also look at average precision (the area under the precision-recall curve), which focuses on how clean the top of the list is.
CodeROC-AUC and average precision with scikit-learn
from sklearn.metrics import roc_auc_score, average_precision_score
y_true = [0, 0, 1, 0, 1, 0, 0, 1, 0, 0]
scores = [0.1, 0.3, 0.8, 0.2, 0.65, 0.7, 0.05, 0.9, 0.4, 0.15]
print("ROC-AUC:", round(roc_auc_score(y_true, scores), 3))
print("Average precision:", round(average_precision_score(y_true, scores), 3))