Interpreting a model
A coefficient is a conditional statement about the model, not about the world. Read it in its units, check it against a held-out set, and never mistake importance for cause.
A coefficient is conditional
A coefficient, the weight on one feature, says how the prediction moves. Hold every other feature fixed. Raise this feature by one unit. The prediction then changes by the coefficient. That clause about the other features is the key part. A coefficient describes the fitted model, not a law about the world.
Units decide the number. A coefficient per year looks small next to one per thousand dollars, even when the two features matter equally. Scaling the features first removes the units. Each coefficient then measures a change of one standard deviation, which puts them on a similar footing without changing the model.
Correlated features make individual weights unstable. Two columns that carry almost the same information can be traded one against the other. The signs can flip between two samples even when both features are useful. The prediction stays stable while the account of it moves. That is a warning about reading a single weight in isolation.
Log-odds and odds ratios
Logistic regression predicts a log-odds, the log of the odds. A coefficient b means that a one-unit rise adds b to the log-odds of the positive class. Turning the log back into odds gives the odds ratio. A ratio of 1.0 means no change. A ratio of 2.0 doubles the odds, and 0.5 halves them. That is a multiplicative statement about odds, not about chance.
Odds and chance move apart as the baseline moves. A coefficient can triple the odds while moving a predicted chance from 0.02 to 0.06. The same coefficient can move a chance from 0.40 to 0.67. Convert to predicted chances at typical feature values before you explain the effect to anyone who will act on it.
The intercept is the log-odds when every feature is zero. That is often a row that cannot exist: zero age, zero income, no prior purchases. Centring the features makes the intercept the log-odds at the mean. That is at least a plausible reference point. It does not change any slope.
Importance on held-out data
from sklearn.inspection import permutation_importance
result = permutation_importance(
model, X_test, y_test, n_repeats=20, random_state=0, scoring="roc_auc"
)
for name, mean, std in zip(X_test.columns, result.importances_mean, result.importances_std):
print(f"{name:12} {mean:+.3f} +/- {std:.3f}")Coefficients in their units
Fit the logistic model, compare the standardised odds ratios with the per-year and per-thousand ones, then add a copy of age and watch the coefficients split.
Global views and their blind spots
Permutation importance shuffles one column and measures the drop in a held-out metric. It repeats that a few times. The method works for any model. It answers a useful question: how much does the fitted model rely on this column for these rows? Correlated columns share the credit. Shuffling can also create rows that never occur in reality.
Impurity importance from a tree is computed on training data. It favours features with many different values, because they offer more places to split. It is cheap, and it is a poor guide to generalisation. Prefer permutation importance on data the model did not fit. Report the spread across repeats, not a single ranking.
A partial dependence plot shows the average prediction as one feature changes. The other features are averaged out. It hides interactions, and it can extrapolate into empty regions of the feature space. Individual conditional expectation curves show one line per row. There a real interaction becomes visible as lines that do not move together.
Importance is not causation. A feature can rank highly because it is a proxy for another column, or because it encodes the label by accident, and neither fact makes it a lever you can pull.
Local explanations and honesty
A local explanation credits one prediction to its features. SHAP values come from a cooperative game and add up to the prediction. LIME fits a small stand-in model around the row. A counterfactual names the nearest point with a different prediction. Each answers a different question. Mixing them in one report produces a story that is hard to check.
Attribution is not causation here either. A feature can be important because it stands in for a protected attribute. It can also matter because the training window happened to link it with the label. The explanation describes the model’s reasoning. That is exactly what you want when the model is wrong. It is exactly what you must not present as a claim about the world.
Report direction, units, uncertainty and the group the numbers describe. Bootstrap the weights or the importance scores, and show the interval. A ranking without a spread is a story. Check the explanation against a known case before trusting it on a new one. Write down the version of the model and the data it was computed on. An explanation of yesterday’s weights is a historical document.
Importance by shuffling
Run the leave-one-out accuracy, compare the importance of x0 before and after the duplicate is added, then replace the duplicate with a column of pure noise.
Check what you learned
One question at a time. A wrong answer explains itself — take another go.
Part 1 of 5
What does a linear coefficient describe?