Step 1: Decide what the predictions are for, and what they’re not for
A prediction that someone might resign is sensitive. Used well, it shows where pay, workload or management need attention. Used badly, it gets people passed over for projects or promotions because a model guessed they might go, which makes them more likely to leave. Agree on the uses before writing any code:
- Yes: finding the teams and conditions linked to people leaving, planning hiring, and offering stay conversations to everyone in a group at risk.
- No: decisions about one person’s pay, promotion, projects or performance. Keep individual scores to a small HR group, and tell employees what the model uses and why.
Then define what the model predicts: resigned in the 12 months after a snapshot date. Leave out retirements, layoffs and contracts that ended; they have different causes and different fixes. Before you start, check with your HR and legal teams which records you may use: privacy law in many places requires telling employees how their data is used.
Step 2: Get one row per employee as of a date
Ask HR for a snapshot of everyone on the payroll on one date, with what was known about them that day: team, level, years in the company, months since their last promotion, pay compared with the middle of their pay band, last rating, overtime, manager changes, commute and their latest engagement survey answer. Then add the date each person resigned, if they did. Two snapshots a year apart let you learn from one year and test on the next.
No data to hand? This script makes two sample snapshots, on January 1, 2024 and January 1, 2025, of a company of about 3,000 people in five departments. Hidden in it are common reasons people leave: no promotion for a long time, pay below the band (worst for strong performers), long hours, new managers, low engagement and the first two years in a job.
CodeOptional: make sample employees.csv
# pip install pandas numpy scikit-learn
import numpy as np
import pandas as pd
rng = np.random.default_rng(5)
DEPARTMENTS = ["engineering", "sales", "support", "operations", "finance"]
def snapshot(date, n, start_id):
"""Everyone on the payroll on one date, as HR records them, and when anyone resigned afterwards."""
e = pd.DataFrame({
"employee_id": np.arange(start_id, start_id + n),
"snapshot_date": pd.Timestamp(date),
"department": rng.choice(DEPARTMENTS, n, p=[0.30, 0.25, 0.20, 0.15, 0.10]),
"gender": rng.choice(["woman", "man", "not stated"], n, p=[0.46, 0.50, 0.04]),
"level": rng.choice([1, 2, 3, 4], n, p=[0.40, 0.32, 0.20, 0.08]),
"tenure_years": rng.exponential(3.5, n).round(1),
"months_since_promotion": rng.integers(0, 60, n),
"pay_vs_band": rng.normal(1.0, 0.08, n), # salary divided by the middle of the role's pay band
"rating": rng.choice([2, 3, 4, 5], n, p=[0.08, 0.52, 0.30, 0.10]),
"overtime_hours": rng.gamma(1.5, 5, n), # a month, on average
"manager_changes_2y": rng.poisson(0.6, n),
"commute_km": rng.gamma(2, 8, n).round(0),
"engagement": rng.choice([1, 2, 3, 4, 5], n, p=[0.06, 0.14, 0.32, 0.34, 0.14]).astype(float),
})
# Teams differ: support works the most overtime and is paid lowest in its bands
e["overtime_hours"] = (e.overtime_hours + e.department.map({"support": 6, "operations": 3}).fillna(0)).round(0)
e["pay_vs_band"] = (e.pay_vs_band - 0.04 * (e.department == "support")).round(2)
e["months_since_promotion"] = np.minimum(e.months_since_promotion, (e.tenure_years * 12).astype(int))
# Hidden reasons people leave: stuck without a promotion, paid below the band (worst for strong performers),
# long hours, new managers, low engagement, and the first two years in a job.
logit = (-3.0 + 0.9 * (e.months_since_promotion > 30) + 6 * (1 - e.pay_vs_band).clip(lower=0)
+ 1.1 * ((e.rating >= 4) & (e.pay_vs_band < 0.95)) + 0.05 * (e.overtime_hours - 8)
+ 0.45 * e.manager_changes_2y - 0.45 * (e.engagement - 3) + 0.6 * (e.tenure_years < 2)
+ 0.012 * e.commute_km + 0.35 * (e.department == "sales"))
skipped = rng.random(n) < 0.12 + 0.10 * (logit > -1.5) # people about to leave skip the survey more often
e.loc[skipped, "engagement"] = np.nan
leaves = rng.random(n) < 1 / (1 + np.exp(-logit))
e["resignation_date"] = pd.NaT
e.loc[leaves, "resignation_date"] = e.snapshot_date[leaves] + pd.to_timedelta(rng.integers(1, 366, leaves.sum()), unit="D")
return e
employees = pd.concat([snapshot("2024-01-01", 3_000, 1), snapshot("2025-01-01", 3_100, 5_001)])
employees.to_csv("employees.csv", index=False)
print(employees.groupby("snapshot_date").resignation_date.count())What you’ll see
snapshot_date 2024-01-01 541 2025-01-01 546 Name: resignation_date, dtype: int64
CodeLoad the snapshots and mark who resigned
import numpy as np
import pandas as pd
pd.set_option("display.width", 120, "display.max_columns", 20)
df = pd.read_csv("employees.csv", parse_dates=["snapshot_date", "resignation_date"])
# Left = resigned in the 12 months after the snapshot
df["left"] = ((df.resignation_date > df.snapshot_date) &
(df.resignation_date <= df.snapshot_date + pd.DateOffset(years=1))).astype(int)
print(df.groupby("snapshot_date").left.agg(people="size", share_left="mean").round(3))What you’ll see
people share_left snapshot_date 2024-01-01 3000 0.180 2025-01-01 3100 0.176
About 18% of people resigned within a year of each snapshot. That’s the rate any list has to beat.
Step 3: Choose the columns the model may use
Two columns stay out. Gender isn’t a reason anyone resigns, and a model that uses it, or something that stands in for it, can treat people differently for who they are; keep it only to check the model in Step 9. The resignation date is the answer itself. Leave out anything else recorded after the snapshot too, such as notice periods or exit interviews: that’s data leakage, and it makes a model look excellent in testing and useless in real use.
CodeThe model’s columns
FEATURES = ["department", "level", "tenure_years", "months_since_promotion", "pay_vs_band", "rating",
"overtime_hours", "manager_changes_2y", "commute_km", "engagement"]
# Not features: gender (kept only to check fairness in Step 9) and resignation_date (it's the answer)
df["department"] = df.department.astype("category")
print(df[FEATURES].isna().mean().loc[lambda s: s > 0].round(2)) # who skipped the engagement surveyWhat you’ll see
engagement 0.15 dtype: float64
15% of people have no engagement answer because they skipped the survey. Don’t drop them: people who are thinking of leaving are often the ones who don’t answer. Step 6 fills the gap and adds a column that says the answer was missing, so the model can learn whether skipping matters.
Step 4: Learn from one year, test on the next
Train on the 2024 snapshot and who resigned during 2024, and test on the 2025 snapshot and who resigned during 2025. That’s how the model will be used: trained on the past, then applied to the people on the payroll today. A random split would mix the two years and hide what changes from one year to the next.
CodeSplit by snapshot
train = df[df.snapshot_date == "2024-01-01"] # learn from who left during 2024
test = df[df.snapshot_date == "2025-01-01"] # test on who left during 2025
X_train, y_train, X_test, y_test = train[FEATURES], train.left, test[FEATURES], test.left
print(len(train), "people to learn from,", len(test), "to test on")What you’ll see
3000 people to learn from, 3100 to test on
Step 5: Measure two simple rules first
Measure two things for every approach. The ROC-AUC says how well it ranks people overall, from 0.5 (no better than chance) to 1 (perfect). The top 10% list is what HR can act on: of the 10% of people an approach ranks riskiest, how many really resigned? Then try two rules HR teams often use as a baseline: the longest without a promotion, and the lowest paid compared with their band.
CodeTwo simple rules
from sklearn.metrics import roc_auc_score
def report(name, risk, share=0.1):
"""ROC-AUC, and who a list of the riskiest 10% would hold: the share of it who left, and the share of all leavers on it."""
top = np.argsort(-np.asarray(risk))[: int(len(risk) * share)]
left = y_test.to_numpy()[top]
print(f"{name:<24} ROC-AUC {roc_auc_score(y_test, risk):.3f} "
f"{left.mean():.0%} of the list left it holds {left.sum() / y_test.sum():.0%} of all leavers")
print(f"Left within the year: {y_test.mean():.0%} of everyone")
report("Months since promotion", X_test.months_since_promotion)
report("Paid below the band", -X_test.pay_vs_band)What you’ll see
Left within the year: 18% of everyone Months since promotion ROC-AUC 0.527 24% of the list left it holds 14% of all leavers Paid below the band ROC-AUC 0.623 32% of the list left it holds 18% of all leavers
The longest-without-a-promotion rule barely beats chance: a ROC-AUC of 0.527, and 24% of its list resigned. Ranking by pay does better, at 0.623, and 32% of its list resigned.
Step 6: Train two models and compare
logistic regression adds up each column’s effect, so its results are easy to explain. Gradient boosting builds many small decision trees, and can find combinations such as strong performers who are paid below their band. Train both on 2024 and test both on 2025.
CodeLogistic regression and gradient boosting
from sklearn.compose import make_column_transformer
from sklearn.ensemble import HistGradientBoostingClassifier
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import OneHotEncoder, StandardScaler
numbers = [f for f in FEATURES if f != "department"]
logreg = make_pipeline(
make_column_transformer(
(OneHotEncoder(handle_unknown="ignore"), ["department"]),
# a missing survey answer gets the median, plus a column that says it was missing
(make_pipeline(SimpleImputer(strategy="median", add_indicator=True), StandardScaler()), numbers),
verbose_feature_names_out=False),
LogisticRegression(max_iter=1000))
logreg.fit(X_train, y_train)
report("Logistic regression", logreg.predict_proba(X_test)[:, 1])
boost = HistGradientBoostingClassifier(max_iter=300, learning_rate=0.03, max_leaf_nodes=15, min_samples_leaf=40,
l2_regularization=1.0, categorical_features=["department"], random_state=0)
boost.fit(X_train, y_train) # handles the missing answers on its own
report("Gradient boosting", boost.predict_proba(X_test)[:, 1])What you’ll see
Logistic regression ROC-AUC 0.728 45% of the list left it holds 26% of all leavers Gradient boosting ROC-AUC 0.722 43% of the list left it holds 24% of all leavers
Logistic regression ranks best, with a ROC-AUC of 0.728: 45% of its top 10% resigned, two and a half times the average, and the list holds 26% of everyone who left. Gradient boosting comes close, at 0.722 with 43% of its list resigning, but it isn’t better. When a simple model ties a complex one, use the simple one: it’s easier to explain to HR and to the people it’s about. The rest of this guide uses logistic regression.
The numbers also say something important: three in four people who resign aren’t on the list. People leave for reasons no HR record holds, such as a partner’s new job or an offer out of the blue. That’s why this model is for finding patterns across teams, not for judging one person.
Step 7: Check that the chances are honest
HR will read a 30% chance as “3 in 10 people like this resign”. Check that it’s true: group the test employees by their predicted chance and compare it with how many actually resigned.
CodePredicted against actual, by group
test = test.assign(risk=logreg.predict_proba(X_test)[:, 1])
band = pd.cut(test.risk, [0, 0.1, 0.2, 0.3, 0.5, 1], labels=["under 10%", "10–20%", "20–30%", "30–50%", "over 50%"])
print(test.groupby(band, observed=True).agg(people=("left", "size"), predicted=("risk", "mean"), actually_left=("left", "mean")).round(2))What you’ll see
people predicted actually_left risk under 10% 1020 0.06 0.06 10–20% 1033 0.14 0.15 20–30% 515 0.25 0.22 30–50% 431 0.38 0.39 over 50% 101 0.60 0.50
The two columns match closely: 6% predicted and 6% resigned in the lowest group, 38% predicted and 39% resigned in the 30–50% group. Only the small top group is a little high, at 60% predicted and 50% resigned. Honest chances matter because Step 10 adds them up into a count of expected leavers per team.
Step 8: Find what drives the risk
Shuffle one column at a time in the test year and see how much worse the ranking gets: the bigger the drop, the more the model relies on that column. Then the model’s weights show which way each column pushes.
CodeWhat the model relies on
from sklearn.inspection import permutation_importance
# How much worse the ranking gets when one column is shuffled: the bigger the drop, the more the model leans on it
drop = permutation_importance(logreg, X_test, y_test, scoring="roc_auc", n_repeats=10, random_state=0)
print(pd.Series(drop.importances_mean, index=FEATURES).sort_values(ascending=False).round(3))
coefs = pd.Series(logreg[-1].coef_[0], index=logreg[0].get_feature_names_out())
print(coefs.sort_values().round(2))What you’ll see
pay_vs_band 0.070 engagement 0.063 manager_changes_2y 0.033 overtime_hours 0.026 months_since_promotion 0.015 rating 0.013 commute_km 0.005 tenure_years 0.004 department 0.003 level 0.001 dtype: float64 engagement -0.56 pay_vs_band -0.51 tenure_years -0.14 department_finance -0.09 department_operations -0.02 department_engineering -0.01 missingindicator_engagement 0.01 department_support 0.02 level 0.04 department_sales 0.12 commute_km 0.13 rating 0.22 months_since_promotion 0.22 overtime_hours 0.30 manager_changes_2y 0.35 dtype: float64
Pay compared with the band and engagement matter most, then manager changes and overtime. The weights point the way you’d expect: higher pay and engagement go with staying; more manager changes, more overtime and longer without a promotion go with leaving. Higher ratings go with leaving too, because strong performers have options. These are all things a company can change, which makes them more useful than any one person’s score.
Step 9: Check that the model works as well for every group
Use the gender column you kept out of the model to compare groups: the share of each group on the top 10% list, how many of those resigned, and how much of each group’s leavers the list holds. Large gaps mean the model works better for some people than for others, often because of a column that stands in for the group.
CodeCompare groups
flagged = test.risk >= test.risk.quantile(0.9)
fair = test.assign(flagged=flagged).groupby("gender").apply(lambda g: pd.Series({
"people": len(g), "share_flagged": g.flagged.mean(), "share_left": g.left.mean(),
"flagged_who_left": g.left[g.flagged].mean(), "leavers_flagged": g.flagged[g.left == 1].mean()}), include_groups=False)
print(fair.round(2))What you’ll see
people share_flagged share_left flagged_who_left leavers_flagged gender man 1554.0 0.10 0.18 0.50 0.27 not stated 128.0 0.12 0.21 0.40 0.22 woman 1418.0 0.10 0.17 0.41 0.24
Men and women are on the list at the same rate (10%), and it holds a similar share of each group’s leavers (27% and 24%). Of the people on the list, more of the men resigned (50% against 41%). With about 150 people of each on the list, part of that gap is chance: check it again every time you retrain, and if it lasts, look for the column behind it. The “not stated” group, at 128 people, is too small to read much into.
Step 10: Report by team, not by name
Before using the model, retrain it on both years. Then, each quarter, score the latest snapshot and add up the chances by team. Because the chances are honest (Step 7), the sum is the number of people each team can expect to lose in the next 12 months. Here the 2025 snapshot stands in for today’s.
CodeExpected leavers by team
final = logreg.fit(pd.concat([X_train, X_test]), pd.concat([y_train, y_test])) # learn from both years
def team_report(snapshot):
"""Expected leavers per team over the next 12 months, with the conditions the model leans on most."""
s = snapshot.assign(risk=final.predict_proba(snapshot[FEATURES])[:, 1])
return s.groupby("department", observed=True).agg(
people=("risk", "size"), expected_leavers=("risk", "sum"), average_risk=("risk", "mean"),
overtime_hours=("overtime_hours", "mean"), pay_vs_band=("pay_vs_band", "mean"),
skipped_survey=("engagement", lambda x: x.isna().mean())).sort_values("average_risk", ascending=False)
print(team_report(df[df.snapshot_date == "2025-01-01"]).round(2))What you’ll see
people expected_leavers average_risk overtime_hours pay_vs_band skipped_survey department support 619 138.60 0.22 13.48 0.96 0.16 sales 781 143.30 0.18 7.62 1.00 0.16 operations 469 84.90 0.18 10.69 1.00 0.17 finance 304 47.32 0.16 7.45 1.00 0.13 engineering 927 139.46 0.15 7.04 1.00 0.14
Support stands out: the highest average risk (22%), the most overtime (13.5 hours a month, against 7 to 11 elsewhere) and the lowest pay in its bands. That points to a fix for the team, such as more staff or a pay review, rather than a list of names. Run the same report every quarter: if a change works, the team’s expected leavers should fall, and then its actual ones.
- Retrain once a year, when the next year’s resignations are complete, and repeat Steps 7 and 9 each time.
- Watch for change. A reorganization, a new pay policy or a move to a new office changes why people leave. That’s data drift: if expected and actual leavers per team drift apart, retrain.
- Measure the fixes, not the model. Compare teams that got a change with similar teams that haven’t yet, over the following year.
Common pitfalls
- Using the score for decisions about one person, which can push out the very people it flags.
- Training on columns recorded after the snapshot, such as notice periods or exit interviews.
- Dropping people who skipped the survey, who are often the ones most likely to leave.
- Reading the chances as exact without checking them against what really happened.
- Checking the overall score but never comparing how the model does for different groups of people.