How to predict house prices

Step 1: Decide what the estimate is for

The same model can serve very different purposes, and each one sets a different bar:

• A starting point for a listing price: an agent adjusts it, so being within 10% most of the time is useful.
• Spotting homes priced far above or below the market: you care about the big misses, and about explaining why a home looks cheap.
• An automatic valuation, such as for a loan: errors cost real money, so you need a tight error and a way to say “I don’t know.”

Write down your target in words a buyer would understand, for example: half of the estimates within 8% of the sale price. A percentage works better than dollars here, because a $20,000 miss is small on a $600,000 home and large on a $90,000 one.

Step 2: Get past sales

You need homes that sold, their details at the time of sale, and the price. To learn, use the data from Kaggle’s “House Prices” competition: 1,460 sales in Ames, Iowa, from 2006 to 2010, each with 79 details and a SalePrice. This guide uses 14 of those details, the ones a buyer would ask about first.

# pip install pandas scikit-learn
import numpy as np
import pandas as pd

homes = pd.read_csv("train.csv")   # Kaggle "House Prices": one row per sale
numeric = ["GrLivArea", "TotalBsmtSF", "LotArea", "OverallQual", "OverallCond", "YearBuilt",
           "YearRemodAdd", "GarageCars", "FullBath", "Fireplaces"]
categorical = ["Neighborhood", "KitchenQual", "GarageType", "CentralAir"]
X, y = homes[numeric + categorical], homes.SalePrice
print(f"{len(homes):,} sales, median price ${y.median():,.0f}, mean ${y.mean():,.0f}")

Notice the mean is higher than the median. A few expensive homes pull it up: prices are skewed, which matters in Step 5.

Step 3: Use cross-validation on a small dataset

With 1,460 sales, a single 80/20 split leaves under 300 homes for testing, and the score depends a lot on which homes land there. Cross-validation is fairer: split the data into five parts, train on four and test on the fifth, and repeat so every home is tested exactly once. The helper below reports two numbers for every model: the MAE in dollars, and the median percentage error, which matches the target from Step 1.

from sklearn.model_selection import KFold, cross_val_predict

cv = KFold(n_splits=5, shuffle=True, random_state=0)

def report(name, model):
    pred = cross_val_predict(model, X, y, cv=cv)
    pct = np.abs(pred - y) / y
    print(f"{name:<24} MAE ${np.mean(np.abs(pred - y)):>9,.0f}   median error {np.median(pct):.1%}")
    return pred

Step 4: Measure two simple baselines

Before any real model, see how far two simple estimates get you. The first guesses the median price for every home. The second uses one number, the living area, with a straight-line linear regression. Any model you build later has to beat both, or the extra effort isn’t worth it.

from sklearn.compose import ColumnTransformer
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression
from sklearn.pipeline import make_pipeline

report("Median price", DummyRegressor(strategy="median"))
size_only = make_pipeline(ColumnTransformer([("size", "passthrough", ["GrLivArea"])]), LinearRegression())
report("Living area only", size_only)

Step 5: Predict the log of the price

A linear model adds up dollar amounts: so many dollars per square foot, so many for a garage. But a renovated kitchen adds more to a large home in a good neighborhood than to a small one elsewhere. Predicting the logarithm of the price turns those additions into percentages, which fits how prices behave and tames the skew from Step 2. Scikit-learn converts back to dollars for you.

This step also handles the categories and the gaps. Neighborhoods become yes/no columns. A missing GarageType means the home has no garage, so it becomes its own category, “None,” rather than being guessed.

from sklearn.compose import TransformedTargetRegressor
from sklearn.impute import SimpleImputer
from sklearn.linear_model import RidgeCV
from sklearn.preprocessing import OneHotEncoder, StandardScaler

prep = ColumnTransformer([
    ("num", make_pipeline(SimpleImputer(strategy="median"), StandardScaler()), numeric),
    ("cat", make_pipeline(SimpleImputer(strategy="constant", fill_value="None"),
                          OneHotEncoder(handle_unknown="ignore")), categorical),
])
linear = TransformedTargetRegressor(make_pipeline(prep, RidgeCV(alphas=np.logspace(-2, 3, 20))),
                                    func=np.log, inverse_func=np.exp)
report("Linear, log price", linear)

RidgeCV is linear regression with a penalty that keeps the weights small, and it picks the strength of that penalty itself. It helps when, as here, some details overlap, such as the year built and the year of the last remodel.

Step 6: Train gradient-boosted trees

Trees find interactions on their own: a fireplace matters more in some neighborhoods than others, and quality matters more for larger homes. Scikit-learn’s histogram-based gradient boosting is fast, handles missing values and reads categories directly, so there’s nothing new to install.

from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.preprocessing import OrdinalEncoder

tree_prep = ColumnTransformer([
    ("num", "passthrough", numeric),
    ("cat", OrdinalEncoder(handle_unknown="use_encoded_value", unknown_value=-1, encoded_missing_value=-1), categorical),
])
cat_columns = list(range(len(numeric), len(numeric) + len(categorical)))
boosted = TransformedTargetRegressor(
    make_pipeline(tree_prep, HistGradientBoostingRegressor(max_iter=500, learning_rate=0.05,
                                                            categorical_features=cat_columns, random_state=0)),
    func=np.log, inverse_func=np.exp)
pred = report("Gradient boosting", boosted)

With this little data, the linear model and the trees are often close. If the gain from trees is small, prefer the linear model: its weights are easy to explain to a buyer or an agent.

Step 7: Find the homes it can’t price

Sort the homes by how far off the estimate was. The biggest misses teach you more than any score.

homes["predicted"] = pred.round(-3)
homes["error_pct"] = ((pred - y) / y * 100).round(1)
worst = homes.error_pct.abs().nlargest(10).index
print(homes.loc[worst, ["GrLivArea", "Neighborhood", "SaleCondition", "SalePrice", "predicted", "error_pct"]])

Check the SaleCondition column for each miss. This dataset has a well-known example: two very large homes that sold for far less than their size suggests, both partial sales of new homes that weren’t finished. Abnormal sales, such as foreclosures or sales between family members, often show up among the misses too. The model may be right about what the home is worth; the sale just wasn’t a normal market sale.

Decide what your model is for, and match the data to it. For market estimates, train only on normal sales (SaleCondition == "Normal") and say clearly that the estimate doesn’t apply to other kinds of sale.

Step 8: Explain an estimate

People trust an estimate they can understand. Permutation importance shuffles one detail at a time and measures how much worse the estimates get, which shows what the model relies on overall. Here it’s computed on the data the model learned from, so treat it as a rough guide.

from sklearn.inspection import permutation_importance

boosted.fit(X, y)   # the final model learns from every sale
imp = permutation_importance(boosted, X, y, n_repeats=5, random_state=0, scoring="neg_mean_absolute_error")
print(pd.Series(imp.importances_mean, index=X.columns).sort_values(ascending=False).round(0).head(8))

new_home = pd.DataFrame([{"GrLivArea": 1800, "TotalBsmtSF": 900, "LotArea": 9000, "OverallQual": 7,
                          "OverallCond": 5, "YearBuilt": 2004, "YearRemodAdd": 2004, "GarageCars": 2,
                          "FullBath": 2, "Fireplaces": 1, "Neighborhood": "CollgCr", "KitchenQual": "Gd",
                          "GarageType": "Attchd", "CentralAir": "Y"}])
print(f"Estimated price: ${boosted.predict(new_home)[0]:,.0f}")

Expect overall quality and living area at the top. Show the estimate with its typical error from Step 3, for example “about $210,000, usually within 8%,” rather than a single exact number.

Step 9: Keep up with the market

House prices move with interest rates and local demand, so a model trained on last year’s sales drifts out of date; that’s data drift. Each month, compare the estimates with the prices homes actually sold for. If the median error rises, or the estimates are consistently too high or too low, retrain on the latest sales. Weighting recent sales more, or adding the sale month as a detail, helps the model follow the market.

Common pitfalls

• Using details only known after the sale, such as days on the market or the final sale terms.
• Trusting one train/test split on a small dataset. Cross-validation gives a steadier answer.
• Mixing foreclosures and family sales with normal sales, then wondering why big homes are priced badly.
• Filling in “no garage” or “no basement” as if the value were simply unknown.
• Quoting a single exact price instead of an estimate with its typical error.