Step 1: Decide what to predict, and what good enough means
Predict the minutes from the moment the customer presses “Order” to the moment the food arrives. Measure it with the MAE: how many minutes the estimate is off on average. It’s easy to explain (“off by 4 minutes”), and it’s what customers feel.
Also decide how you’ll show it. A single number is always a little wrong, and customers mind a late order much more than an early one. So the goal here is a window, like “35–45 min”, whose end is met for about 9 orders in 10.
Step 2: Use only what you know at checkout
You need one row per past order with the real delivery time, plus everything you knew when the order was placed: the distance, the number of items, the restaurant, whether it was raining, how many couriers were online and how many orders came in over the last 30 minutes.
Leave out anything that only becomes known later, such as when the courier picked the food up. It makes the model look much better in testing (on the sample data the error drops from 3.6 to 2.1 minutes), but it can’t be used at checkout, so the real estimates would be worse than the test promised. That’s data leakage.
No data to hand? This script makes realistic sample data: 40,000 orders from 60 restaurants over six months, with lunch and dinner rushes, rainy weeks, and a few orders held up by a wrong address or a closed road.
CodeOptional: make a sample orders.csv
# pip install pandas numpy
import numpy as np
import pandas as pd
rng = np.random.default_rng(3)
n = 40_000
# Orders over six months, busiest at lunch and dinner.
day = rng.integers(0, 181, n)
hour = rng.choice(np.arange(10, 23), n, p=np.array([2, 6, 9, 6, 3, 2, 3, 6, 10, 9, 6, 3, 2]) / 67)
ordered_at = pd.Timestamp("2025-01-01") + pd.to_timedelta(day, unit="D") + pd.to_timedelta(hour * 60 + rng.integers(0, 60, n), unit="min")
restaurant = rng.integers(0, 60, n)
kitchen_speed = rng.normal(14, 4, 60).clip(6, 25) # each restaurant has its own typical prep time
distance_km = rng.gamma(2.2, 1.3, n).clip(0.3, 12).round(1)
items = rng.integers(1, 8, n)
rush = np.isin(hour, [12, 13, 18, 19, 20])
raining = rng.random(n) < np.where((day // 7) % 5 == 0, 0.45, 0.12) # some weeks are wetter than others
weekend = ordered_at.dayofweek.values >= 5
couriers_online = (rng.normal(30, 5, n) * np.where(rush, 1.3, 1.0) * np.where(weekend, 0.85, 1.0)).round().astype(int)
orders_last_30min = (rng.poisson(np.where(rush, 55, 22) * np.where(raining, 1.4, 1.0) * np.where(weekend, 1.15, 1.0))).astype(int)
prep = kitchen_speed[restaurant] + 0.9 * items + 0.08 * orders_last_30min + rng.normal(0, 2.5, n)
# Waiting for a courier: grows quickly once orders outnumber couriers.
courier_wait = np.maximum(0, 3 * (orders_last_30min / couriers_online - 1)) * 6 + rng.exponential(2, n)
speed_kmh = 18 * np.where(rush, 0.8, 1.0) * np.where(raining, 0.75, 1.0)
travel = distance_km / speed_kmh * 60 + rng.normal(0, 2, n)
delays = rng.random(n) < 0.03 # wrong address, a closed road...
minutes = prep + courier_wait + travel + np.where(delays, rng.exponential(15, n), 0) + 4
orders = pd.DataFrame({
"order_id": np.arange(n), "ordered_at": ordered_at, "restaurant_id": restaurant,
"distance_km": distance_km, "items": items, "raining": raining.astype(int),
"couriers_online": couriers_online, "orders_last_30min": orders_last_30min,
"picked_up_after_min": (prep + courier_wait).round(1), # only known once the courier collects the food
"delivery_minutes": minutes.clip(8).round(1),
}).sort_values("ordered_at")
orders.to_csv("orders.csv", index=False)
print(len(orders), "orders; median delivery", orders.delivery_minutes.median(), "minutes")Step 3: Split by time
Learn from January to April, choose settings on May, and test once on June. Splitting by date, instead of at random, checks the model the way it will be used: on orders that come after the ones it learned from.
CodeFeatures, target and a split by month
import numpy as np
import pandas as pd
orders = pd.read_csv("orders.csv", parse_dates=["ordered_at"])
orders["hour"] = orders.ordered_at.dt.hour
orders["weekday"] = orders.ordered_at.dt.dayofweek
# Only use what you know when the customer presses "Order". The pickup time comes later, so it's left out.
FEATURES = ["distance_km", "items", "raining", "couriers_online", "orders_last_30min", "hour", "weekday", "restaurant_id"]
TARGET = "delivery_minutes"
# Learn from January to April, choose settings on May, test once on June.
train = orders[orders.ordered_at < "2025-05-01"]
val = orders[(orders.ordered_at >= "2025-05-01") & (orders.ordered_at < "2025-06-01")]
test = orders[orders.ordered_at >= "2025-06-01"]
print(f"{len(train)} training, {len(val)} validation and {len(test)} test orders")Step 4: Measure the simple estimates first
Many apps show the same estimate to everyone, or one based on distance alone. They’re the baseline: a model has to beat them clearly to be worth running.
CodeTwo baselines
# pip install scikit-learn
from sklearn.metrics import mean_absolute_error
def report(name, data, predicted):
print(f"{name}: off by {mean_absolute_error(data[TARGET], predicted):.1f} minutes on average")
# Baseline 1: promise everyone the typical delivery time.
report("Same estimate for everyone", val, np.full(len(val), train[TARGET].median()))
# Baseline 2: the typical time for orders of a similar distance.
bands = pd.cut(orders.distance_km, [0, 1, 2, 3, 4, 6, 8, 12])
by_distance = train.groupby(bands[train.index], observed=True)[TARGET].median()
report("Typical time for the distance", val, by_distance.loc[bands[val.index]].to_numpy())On the sample data, the same estimate for everyone is off by 12.6 minutes on average, and the distance-based one by 11.1. Distance matters, but how busy the kitchen and the couriers are matters just as much.
Step 5: Try a linear regression
A linear regression adds up a weight for each feature: so many minutes per kilometer, per item, per order in the last half hour. It’s quick, and easy to read.
CodeLinear regression
from sklearn.linear_model import LinearRegression
linear = LinearRegression().fit(train[FEATURES], train[TARGET])
report("Linear regression", val, linear.predict(val[FEATURES]))It’s off by 5.4 minutes, less than half the baseline’s error. But some effects aren’t straight lines: waiting for a courier barely matters on a quiet afternoon, and then grows fast once orders outnumber couriers.
Step 6: Train gradient-boosted trees
Gradient boosting builds many small decision trees, each one fixing the mistakes of the ones before, so it picks up effects like that one by itself. LightGBM is a fast, popular choice. How many trees to use is a setting to choose on the validation month: too few and the model misses patterns, too many and it starts learning noise.
CodeLightGBM, with the number of trees chosen on validation
# pip install lightgbm
import lightgbm as lgb
params = dict(n_estimators=1500, learning_rate=0.03, num_leaves=31, min_child_samples=40, verbose=-1)
model = lgb.LGBMRegressor(objective="l1", **params) # "l1" aims straight at the average error in minutes
model.fit(train[FEATURES], train[TARGET], categorical_feature=["restaurant_id"])
# How many trees? Too few underfits, too many overfits: choose on the validation month.
scores = {k: mean_absolute_error(val[TARGET], model.predict(val[FEATURES], num_iteration=k)) for k in range(100, 1501, 100)}
trees = min(scores, key=scores.get)
report(f"LightGBM with {trees} trees", val, model.predict(val[FEATURES], num_iteration=trees))
# Which features it leans on most:
importance = pd.Series(model.booster_.feature_importance("gain", iteration=trees), index=FEATURES)
print((importance / importance.sum()).sort_values(ascending=False).round(2).to_dict())This brings the error down to 3.6 minutes. The model leans most on how busy things are (orders in the last 30 minutes, then couriers online), the distance and the restaurant. Rain matters less on its own, because rain already shows up as more orders.
Step 7: Test once, and look at the hard cases
Run the test month once, with every setting already chosen. Then break the error down by situation: an average can hide a model that’s fine on quiet afternoons and poor in the dinner rush.
CodeThe test month
# The test month, once.
predicted = model.predict(test[FEATURES], num_iteration=trees)
report("LightGBM on the test month", test, predicted)
error = (test[TARGET] - predicted).abs()
rush = test.hour.isin([12, 13, 18, 19, 20])
print(f" quiet hours {error[~rush].mean():.1f} min · rush hours {error[rush].mean():.1f} min · "
f"dry {error[test.raining == 0].mean():.1f} min · rain {error[test.raining == 1].mean():.1f} min")On the test month it’s still off by 3.6 minutes on average: 3.5 in quiet hours, 3.7 in rush hours and 4.0 in the rain. No situation is badly off, so it’s ready to try with real customers.
Step 8: Promise a window customers can trust
Train a second model that predicts a time 90% of orders beat (LightGBM’s quantile objective does this), and show a prediction interval from the typical time to that one. Check the promise on the validation month, and add a small margin if it’s met less often than 9 times in 10.
CodeA time that 9 orders in 10 beat
from sklearn.metrics import mean_pinball_loss
# Customers mind a late order far more than an early one. Predict a time that 90% of orders beat.
late = lgb.LGBMRegressor(objective="quantile", alpha=0.9, **params)
late.fit(train[FEATURES], train[TARGET], categorical_feature=["restaurant_id"])
losses = {k: mean_pinball_loss(val[TARGET], late.predict(val[FEATURES], num_iteration=k), alpha=0.9) for k in range(100, 1501, 100)}
late_trees = min(losses, key=losses.get)
# Check the promise on the validation month and add a small margin if it's met less than 90% of the time.
miss = val[TARGET] - late.predict(val[FEATURES], num_iteration=late_trees)
margin = max(0.0, np.quantile(miss, 0.9))
print(f"Margin added from validation: {margin:.1f} minutes")
promise = late.predict(test[FEATURES], num_iteration=late_trees) + margin
typical = model.predict(test[FEATURES], num_iteration=trees)
print(f"On the test month, the promised time was met for {(test[TARGET] <= promise).mean():.0%} of orders; "
f"it's {np.mean(promise - typical):.0f} minutes after the typical time on average")On the test month, the promised time was met for 90% of orders, as intended. On average, the window ends about 6 minutes after the typical time.
Step 9: Estimate at checkout, and keep it honest
At checkout, look up the live numbers (couriers online, recent orders, the weather), and turn the two predictions into a rounded window.
CodeAn estimate for a new order
import joblib
joblib.dump({"typical": model, "trees": trees, "late": late, "late_trees": late_trees,
"margin": margin, "features": FEATURES}, "delivery_model.joblib")
def estimate(order):
"""At checkout: a window like "35–45 min", from what's known when the customer presses Order."""
row = pd.DataFrame([order])[FEATURES]
low = model.predict(row, num_iteration=trees)[0]
high = late.predict(row, num_iteration=late_trees)[0] + margin
return f"{5 * np.floor(low / 5):.0f}–{5 * np.ceil(high / 5):.0f} min"
quiet = {"distance_km": 2.5, "items": 3, "raining": 0, "couriers_online": 32, "orders_last_30min": 25,
"hour": 15, "weekday": 2, "restaurant_id": 7}
print("Wednesday afternoon:", estimate(quiet))
print("Rainy Saturday evening:", estimate({**quiet, "raining": 1, "orders_last_30min": 80, "hour": 19, "weekday": 5}))The same order gets “35–45 min” on a quiet Wednesday afternoon and “70–80 min” on a rainy Saturday evening.
Once it’s live, track two numbers every week: the average error, and the share of orders that arrive by the end of the window. If either slips, something has changed: new restaurants, a new area, more couriers (data drift). Retrain every month on the latest orders, and test each new model on the most recent month before switching.
Common pitfalls
- Using information that only exists after the order, like the pickup time, so testing looks far better than real life.
- Splitting orders at random instead of by date.
- Showing one exact number when a window would be more honest and more useful.
- Judging the model on its average error alone, and missing that it’s poor at rush hour or in the rain.
- Never retraining as the business grows and changes.