1. Promise
One-sentence promise
After this lecture, you can choose an appropriate level of interpretability, avoid common feature-importance traps in time series, and communicate uncertainty in a way that supports safe decisions.
End tag line
Explainability is safety, not marketing.
2. Learning objectives
After reading this lecture, you will be able to:
- Explain the interpretability–performance–complexity trade-off.
- Identify why feature importance can be misleading in time series.
- Use time-consistent diagnostics for “what drives the model”.
- Communicate uncertainty as ranges, scenarios, and decision thresholds.
- Write a short explanation that a risk/control stakeholder accepts.
3. Conceptual core
3.1 Interpretability trade-offs
Interpretability is not binary. It has levels.
Level 0 — Transparent - linear regression, ridge/lasso, - simple decision rules.
Pros: - easy to validate, - stable explanations, - fast.
Cons: - limited nonlinearity.
Level 1 — Explainable black box - tree/boosting with careful constraints, - monotonic constraints (where appropriate), - calibrated probabilities.
Pros: - strong performance in many tabular tasks.
Cons: - explanations are post-hoc, can be fragile.
Level 2 — Opaque - deep learning, complex ensembles.
Pros: - may help when structure is genuinely complex and data is rich.
Cons: - difficult governance, - hard debugging, - failure modes are harder to bound.
Engineering rule
Match interpretability to: - decision criticality, - regulatory needs, - latency, - ability to monitor.
Validator sentence
A slightly worse but interpretable model can be the only deployable model.
3.2 Feature importance pitfalls in time series
Feature importance is not “truth”. In time series, it is particularly easy to lie to yourself.
Pitfall 1: Leakage makes “importance” meaningless
If a feature contains future information, it will look dominant.
Control: - strict time-split validation, - causal feature construction tests.
Pitfall 3: Non-stationarity changes importance over time
A feature can be important in one regime and useless in another.
Control: - compute importance by fold / subperiod, - report variability.
Pitfall 4: Model-based importance measures what the model used, not what is causal
Importance is conditional on the model class. It is not a causal claim.
Control: - use language: “predictive driver” not “cause”.
Pitfall 5: Permutation importance can break time dependence
Shuffling a feature destroys temporal structure. Permutation scores can be misinterpreted.
Control: - use block permutation (shuffle in blocks), - or compute importance fold-by-fold with caution.
Validator sentence
In time series, importance must be reported with stability, not with confidence.
3.3 Communicating uncertainty
Uncertainty is not a weakness. It is a specification.
Three practical types:
(1) Aleatoric uncertainty (noise)
Markets are noisy. Your model can be correct and still wrong on any given day.
(2) Epistemic uncertainty (model uncertainty)
Limited data, regime changes, misspecification.
(3) Decision uncertainty
Even with a forecast, the decision can be ambiguous near thresholds.
Practical communication tools
- Prediction intervals:
- Scenario buckets:
- base / adverse / stress.
- Confidence gating:
- “act only when signal-to-noise exceeds threshold.”
Stakeholder language
- “The point forecast is X.”
- “The uncertainty band is ±Y.”
- “We only act when X is large relative to Y.”
- “In high-vol regimes, Y increases, so we scale down.”
Validator sentence
The safest model is often the one that knows when not to act.
4. What “good” explainability looks like (deliverables)
A good explainability pack includes:
- A one-page explanation of the model type and intended use.
- Fold-wise importance stability (not one number).
- A regime-conditioned story: “drivers differ in high vs low vol”.
- An uncertainty policy: intervals, gating, and actions.
5. Control lens
Explainability is observability: - can you see what drives the controller output? - can you diagnose failure?
Uncertainty is noise modelling: - control gains should depend on noise intensity. Finance decisions should too.
6. Python lab (importance stability + time-series-safe permutation + uncertainty bands)
This lab demonstrates:
- a simple linear model (interpretable baseline),
- coefficient stability across folds,
- block-permutation importance (time-consistent-ish),
- rolling uncertainty bands from residual volatility.
No external data is required.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.model_selection import TimeSeriesSplit
from sklearn.linear_model import Ridge
rng = np.random.default_rng(0)
# ---------------------------------------------------------
# 1) Synthetic time series with regimes
# ---------------------------------------------------------
T = 600
idx = pd.date_range("2024-01-01", periods=T, freq="B")
t = np.arange(T)
vol = np.ones(T) * 0.8
vol[T//2:] = 1.6
# latent signal with seasonality
signal = 0.3*np.sin(2*np.pi*t/5) + 0.2*np.sin(2*np.pi*t/20)
y = np.zeros(T)
for k in range(1, T):
y[k] = 0.2*y[k-1] + signal[k] + vol[k]*rng.normal()
df = pd.DataFrame({"y": y}, index=idx)
# Features: lag block + vol proxy
df["r1"] = df["y"].diff()
df["y_target"] = df["r1"].shift(-1)
df["lag1"] = df["r1"].shift(1)
df["lag2"] = df["r1"].shift(2)
df["lag5"] = df["r1"].shift(5)
df["vol_20"] = df["r1"].rolling(20).std()
df["dow"] = df.index.dayofweek
feat_cols = ["lag1", "lag2", "lag5", "vol_20", "dow"]
d = df.dropna().copy()
X = d[feat_cols].values
y = d["y_target"].values
# ---------------------------------------------------------
# 2) Fit model by fold and collect coefficient stability
# ---------------------------------------------------------
tscv = TimeSeriesSplit(n_splits=5)
coefs = []
pred = np.full_like(y, np.nan, dtype=float)
resid = np.full_like(y, np.nan, dtype=float)
for k, (tr, te) in enumerate(tscv.split(X), start=1):
model = Ridge(alpha=1.0)
model.fit(X[tr], y[tr])
yhat = model.predict(X[te])
pred[te] = yhat
resid[te] = y[te] - yhat
coefs.append(pd.Series(model.coef_, index=feat_cols, name=f"fold{k}"))
coef_df = pd.concat(coefs, axis=1)
print("=== Coefficients by fold (stability view) ===")
print(coef_df)
# Plot coefficient variability
plt.figure(figsize=(8, 3.8))
for c in feat_cols:
plt.plot(coef_df.columns, coef_df.loc[c].values, marker="o", label=c)
plt.title("Coefficient stability across TimeSeriesSplit folds (Ridge)")
plt.xlabel("fold")
plt.ylabel("coefficient")
plt.legend()
plt.tight_layout()
plt.show()
# ---------------------------------------------------------
# 3) Block permutation importance (simple, time-aware-ish)
# ---------------------------------------------------------
def rmse(y_true, y_hat):
return float(np.sqrt(np.mean((y_true - y_hat)**2)))
def block_permute(x, block=20, rng=None):
rng = rng or np.random.default_rng(0)
n = len(x)
blocks = [x[i:i+block] for i in range(0, n, block)]
order = rng.permutation(len(blocks))
return np.concatenate([blocks[i] for i in order])[:n]
# Evaluate importance on the LAST fold only (simple, fast)
# In practice: do this fold-by-fold and report distribution.
last_tr, last_te = list(tscv.split(X))[-1]
base_model = Ridge(alpha=1.0).fit(X[last_tr], y[last_tr])
base_pred = base_model.predict(X[last_te])
base_rmse = rmse(y[last_te], base_pred)
imp_rows = []
for j, name in enumerate(feat_cols):
Xp = X[last_te].copy()
Xp[:, j] = block_permute(Xp[:, j], block=30, rng=rng)
p = base_model.predict(Xp)
imp = rmse(y[last_te], p) - base_rmse
imp_rows.append({"feature": name, "rmse_increase": float(imp)})
imp_df = pd.DataFrame(imp_rows).sort_values("rmse_increase", ascending=False)
print("\n=== Block permutation importance (last fold) ===")
print(imp_df.to_string(index=False))
plt.figure(figsize=(7.5, 3.5))
plt.bar(imp_df["feature"], imp_df["rmse_increase"])
plt.title("Block permutation importance (RMSE increase)")
plt.xlabel("feature")
plt.ylabel("ΔRMSE")
plt.xticks(rotation=30, ha="right")
plt.tight_layout()
plt.show()
# ---------------------------------------------------------
# 4) Communicating uncertainty: rolling residual std and bands
# ---------------------------------------------------------
s = pd.Series(resid, index=d.index).dropna()
band = s.rolling(40).std()
pred_s = pd.Series(pred, index=d.index)
y_s = pd.Series(y, index=d.index)
# show last 120 points
ix = d.index[-120:]
plt.figure(figsize=(8, 3.8))
plt.plot(ix, y_s.loc[ix], label="realised y")
plt.plot(ix, pred_s.loc[ix], label="prediction")
upper = pred_s.loc[ix] + 2.0*band.loc[ix]
lower = pred_s.loc[ix] - 2.0*band.loc[ix]
plt.fill_between(ix, lower.values, upper.values, alpha=0.2, label="±2σ band (rolling)")
plt.title("Prediction with uncertainty band (rolling residual volatility)")
plt.xlabel("date")
plt.ylabel("next-day return")
plt.legend()
plt.tight_layout()
plt.show()
print("\nBase RMSE (last fold):", base_rmse)7. How to explain the lab results (short, credible)
- “We use a simple ridge model to keep explanations stable.”
- “Coefficients vary by fold, so drivers are not constant across time.”
- “We use block permutation to avoid destroying all time structure.”
- “Uncertainty bands widen when residual volatility rises.”
- “We gate decisions: act only when forecast is large relative to its uncertainty.”
8. Mini-exercises
- Repeat block permutation importance for each fold and report mean ± std.
- Replace ridge with a tree model and compare stability of explanations.
- Create an uncertainty-based rule: trade only if .
9. Summary
- Interpretability is a deployability choice, not a philosophical choice.
- Feature importance in time series must be checked for stability and leakage.
- Uncertainty should be communicated as bands and decision gating.
- Explainability reduces misuse and supports governance approval.