Module 102 — Data definition for quantitative models — Quant Model Playbook

1. Promise

One-sentence promise

After this lecture, you can define a universe, design time-consistent splits, and run a leakage checklist that prevents look-ahead, target leakage, survivorship bias, and peeking through normalisation.

End tag line

Clean data is not enough. It must be time-correct.

2. Learning objectives

After reading this lecture, you will be able to:

Define a trading or credit universe and formalise inclusion rules.
Apply filters (liquidity, trading hours, corporate actions, data quality) without biasing evaluation.
Design train/validation/test splits for time series (walk-forward and embargo).
Run a leakage checklist covering look-ahead, label leakage, survivorship, and normalisation peeking.
Explain why each leakage type inflates performance and how to prevent it.

3. Prerequisites and setup

Required maths and prior knowledge

Basic probability and expectation.
Understanding of time series indexing.
Familiarity with regression/classification framing.

Python environment assumptions

Python 3.9 or newer.
Jupyter notebook or similar.

Packages used

numpy, pandas, matplotlib

4. Conceptual core

4.1 Universe definition and filters

Notation

Assets indexed by $i \in \{1,\dots,N\}$ .
Time indexed by $t$ .
Universe at time $t$ : $\mathcal{U}_t \subseteq \{1,\dots,N\}$ .

A universe definition is a rule that maps time to a set: $\mathcal{U}_t = \{ i : \text{InclusionRule}(i,t)=1 \}.$

Key principle

Universe membership must be decided using information available at $t$ , not at the end of the sample.

Typical inclusion rules (examples)

Listing venue and instrument type (e.g., Swiss equities, cash bonds, FX spot).
Availability of valid prices and corporate actions.
Minimum liquidity: ADV, turnover, bid–ask spread thresholds.
Trading calendar: remove holidays, auction periods, illiquid sessions.

Filters: what to state explicitly

Liquidity filter: e.g., include $i$ if ADV $(i,t;L)\ge a_{\min}$ , computed over a lookback $L$ that ends at $t$ .
Price filter: e.g., exclude penny stocks or near-zero prices to avoid microstructure artefacts.
Corporate actions handling: adjust prices for splits/dividends; define how delistings are handled.
Data quality filter: remove assets with stale quotes or broken timestamps.

Validator lens

Every filter changes the distribution of outcomes.
A filter is acceptable if it matches what could be traded or decided at time $t$ .
If you filter using future outcomes (delisting, future liquidity), you introduce survivorship or look-ahead.

Slide-ready sentence

A universe is a time-dependent set $\mathcal{U}_t$ . If you define it with future knowledge, the test is invalid.

4.2 Train / validation / test split design

Time series split requirement

For financial data, iid random splits are usually wrong because temporal dependence and regime structure matter.

Let the dataset be $\{(X_t, Y_{t+h})\}_{t=1}^{T-h}$ .

A time-correct split ensures:

Training uses only times $\le t_{\text{train end}}$ .
Validation uses future times.
Test uses later future times not used in model selection.

Common split designs

Single hold-out by time

Train: $[1, t_1]$
Validation: $(t_1, t_2]$
Test: $(t_2, T]$

Walk-forward (rolling / expanding)

Fit on a window, validate on next block, roll forward.
Expanding window: training set grows over time.

Embargo / purge for overlapping labels If $Y_{t+h}$ overlaps with $Y_{t+h+1}$ (e.g., using returns over windows), observations are not independent.

You must purge overlapping samples and add an embargo between train and test: - Purge: remove training points whose label window overlaps the test window. - Embargo: remove a buffer of size $e$ after the test start to prevent leakage through overlap.

Validator lens

Model selection must use validation only.
Test must be used once, at the end, to estimate out-of-sample performance.

Slide-ready sentence

In time series, the split is part of the model. Random splits leak regime information.

4.3 Leakage checklist

Leakage is any path where information from the future enters the model at time $t$ .

(1) Look-ahead bias

Definition

A feature uses information not available at decision time: $X_t \leftarrow g(\text{data up to } t+\Delta),\quad \Delta>0.$

Examples: - Using close-to-close returns when your decision is at open. - Using end-of-day volume to trade intraday.

Fix

Enforce feature availability timestamps and only use data with time $\le t$ .

(2) Target leakage

Definition

Features include the target or a proxy: $X_t \text{ contains } Y_{t+h} \text{ or function of it}.$

Examples: - Using realised P&L fields when predicting next-day returns. - Using “days past due” that already encodes default event timing.

Fix

Audit each feature: could it be computed without knowing the label?

(3) Survivorship bias

Definition

Universe is defined using only assets that survive to the end date: $\mathcal{U}_t = \{i : i \text{ exists at } T\},$ which excludes bankrupt/delisted names that were tradable earlier.

Effect

Inflates performance because failures are removed.

Fix

Use point-in-time membership and include delisted assets up to delisting date.

(4) Peeking via normalisation (scaling leakage)

Definition

Normalisation uses statistics computed on the full sample: $\mu = \frac{1}{T}\sum_{t=1}^T X_t,\quad \sigma^2 = \frac{1}{T}\sum_{t=1}^T (X_t-\mu)^2,$ then applies them to training and test.

This uses future information because $\mu,\sigma$ depend on test periods.

Fix

Fit scalers on training only, then apply to validation/test.

5. Control lens

5.1 Analogue and failure mode

Control analogue

In system identification, you cannot use future outputs to build the regressor at time $t$ . You also cannot estimate noise statistics using data that includes the test trajectory.

Failure mode

Your identified model looks excellent offline. It fails when deployed because the identification used information unavailable online.

6. Worked examples (short, realistic)

6.1 Universe and filters (toy example)

You define $\mathcal{U}_t$ as “assets with ADV over last 20 days above threshold”. ADV must be computed using days $[t-19, t]$ , not $[t-19, t+1]$ .

This single off-by-one error is a look-ahead leak.

6.2 Split design (walk-forward)

Train on 2018–2021, validate on 2022, test on 2023. If labels use a 10-day forward return, purge 10 days at split boundaries.

7. Python lab (leakage demos + plots)

This lab produces three figures:

Look-ahead feature creates artificial predictability.
Normalisation leakage inflates test correlation.
Survivorship bias inflates average return.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

rng = np.random.default_rng(1)

# -----------------------------
# Part A: Look-ahead leakage demo
# -----------------------------
T = 600
eps = rng.normal(0, 1.0, T)

# True process: y_{t+1} is mostly noise, weak persistence
y = np.zeros(T)
for t in range(1, T):
    y[t] = 0.1 * y[t-1] + eps[t]

# Correct feature at time t: uses only past info
x_ok = pd.Series(y).shift(1).to_numpy()

# Leaky feature at time t: accidentally uses future y_{t+1}
x_leak = pd.Series(y).shift(-1).to_numpy()

df = pd.DataFrame({"y": y, "x_ok": x_ok, "x_leak": x_leak}).dropna().reset_index(drop=True)

# Train/test split by time
split = int(0.7 * len(df))
train = df.iloc[:split]
test  = df.iloc[split:]

def corr(a, b):
    a = np.asarray(a); b = np.asarray(b)
    return float(np.corrcoef(a, b)[0, 1])

print("=== Look-ahead demo: correlation with future target ===")
print(f"Correct feature corr(x_ok, y):   train={corr(train.x_ok, train.y):.3f}, test={corr(test.x_ok, test.y):.3f}")
print(f"Leaky feature corr(x_leak, y):   train={corr(train.x_leak, train.y):.3f}, test={corr(test.x_leak, test.y):.3f}")

plt.figure(figsize=(7, 3.6))
plt.plot(test.y.values[:120], label="y (test)")
plt.plot(test.x_ok.values[:120], label="x_ok (lagged)")
plt.plot(test.x_leak.values[:120], label="x_leak (future-shifted)")
plt.title("Look-ahead: a future-shifted feature tracks the target")
plt.xlabel("Test time index")
plt.ylabel("Value")
plt.legend()
plt.tight_layout()
plt.show()

# -----------------------------
# Part B: Normalisation peeking demo
# -----------------------------
T = 800
x = rng.normal(0, 1.0, T)

# Non-stationary shift in the second half (common in finance)
x[T//2:] += 2.0

# Target is unrelated noise (no true predictability)
y = rng.normal(0, 1.0, T)

split = int(0.7 * T)
x_train, x_test = x[:split], x[split:]
y_train, y_test = y[:split], y[split:]

# BAD: scaler fitted on full sample (peeking)
mu_all, sd_all = x.mean(), x.std()
x_all_scaled = (x - mu_all) / (sd_all + 1e-12)
x_test_bad = x_all_scaled[split:]

# GOOD: scaler fitted on train only
mu_tr, sd_tr = x_train.mean(), x_train.std()
x_test_good = (x_test - mu_tr) / (sd_tr + 1e-12)

print("\n=== Normalisation peeking demo ===")
print(f"Test mean after BAD scaling:  {x_test_bad.mean():.3f}")
print(f"Test mean after GOOD scaling: {x_test_good.mean():.3f}")

plt.figure(figsize=(7, 3.6))
plt.hist(x_test_bad, bins=40, alpha=0.6, label="BAD: scaled with full-sample stats")
plt.hist(x_test_good, bins=40, alpha=0.6, label="GOOD: scaled with train-only stats")
plt.title("Peeking via normalisation changes the test distribution")
plt.xlabel("Scaled feature value (test)")
plt.ylabel("Count")
plt.legend()
plt.tight_layout()
plt.show()

# -----------------------------
# Part C: Survivorship bias demo
# -----------------------------
# Simulate 200 assets with returns; some "die" (delist) after bad performance
n_assets = 200
T = 250

# Base returns: small positive drift + noise
R = rng.normal(0.0005, 0.02, size=(T, n_assets))

# Death rule: if cumulative return drops below threshold, asset delists
alive = np.ones((T, n_assets), dtype=bool)
cum = np.zeros(n_assets)

death_threshold = -0.35  # -35% cumulative
for t in range(T):
    cum += R[t]
    if t > 20:
        alive[t, cum < death_threshold] = False
        # once dead, stay dead
        if t > 0:
            alive[t] = alive[t] & alive[t-1]

# True universe: include assets while alive (point-in-time)
pit_mean = np.array([R[t, alive[t]].mean() if alive[t].any() else np.nan for t in range(T)])

# Survivorship-biased universe: keep only those alive at the end
survivors = alive[-1]
surv_mean = R[:, survivors].mean(axis=1)

print("\n=== Survivorship bias demo ===")
print(f"Final survivors: {survivors.sum()} / {n_assets}")

plt.figure(figsize=(7, 3.6))
plt.plot(np.cumsum(pit_mean), label="Point-in-time universe mean (cum)")
plt.plot(np.cumsum(surv_mean), label="Survivors-only mean (cum)")
plt.title("Survivorship bias inflates performance by removing failures")
plt.xlabel("Day")
plt.ylabel("Cumulative mean return (toy units)")
plt.legend()
plt.tight_layout()
plt.show()

8. Diagnostics and failure modes (validator checklist)

8.1 Universe and filters

Do you have a written rule for $\mathcal{U}_t$ ?
Are liquidity filters computed using only past data up to $t$ ?
Do you include delisted assets up to delisting date?
Are corporate actions handled consistently and point-in-time?

8.2 Split design

Is the split purely by time (no shuffling)?
Is validation used for model selection and test used once?
Are labels overlapping across splits (need purge/embargo)?
Are time zones and market calendars consistent?

8.3 Leakage checks

Look-ahead: any feature uses $t+\Delta$ info?
Target leakage: any feature encodes the label directly?
Survivorship: universe defined using end-of-sample membership?
Normalisation: are scalers fit on training only?
Feature engineering: rolling windows end at $t$ , not $t+h$ .

9. Mini-exercises

Exercise 1

You filter on “top 500 by market cap”. Market cap is updated daily. What is the correct time-consistent rule?

Solution Define $\mathcal{U}_t$ using market cap available at $t$ . Recompute membership point-in-time. Do not use end-of-year membership for all past dates.

Exercise 2 (trap)

You standardise features using mean and std from the entire dataset. Is this leakage in a time series split?

Solution Yes. The scaler uses test-period statistics. Fit scalers on training only.

Exercise 3

Your target is 10-day forward return. You split train ending at day 200 and test starting at day 201. What must you do?

Solution Purge training points whose label window overlaps the test window. Add an embargo of at least 10 days around the boundary.

10. Summary

Universe is a time-dependent set $\mathcal{U}_t$ , not a fixed list.
Filters must be computed with information available at time $t$ .
Splits must respect time and overlapping label windows.
Leakage comes in four common forms: look-ahead, target leakage, survivorship, normalisation peeking.
A leakage checklist is part of the modelling standard, not an optional hygiene step.

11. Out of scope

Detailed corporate action adjustment algorithms.
Full production data lineage and metadata systems.
Advanced cross-validation for dependent data beyond walk-forward basics.

1. Promise

2. Learning objectives

3. Prerequisites and setup

4. Conceptual core

4.1 Universe definition and filters

Notation

Typical inclusion rules (examples)

Filters: what to state explicitly

4.2 Train / validation / test split design

Time series split requirement

Common split designs

4.3 Leakage checklist

(1) Look-ahead bias

(2) Target leakage

(3) Survivorship bias

(4) Peeking via normalisation (scaling leakage)

5. Control lens

5.1 Analogue and failure mode

6. Worked examples (short, realistic)

6.1 Universe and filters (toy example)

6.2 Split design (walk-forward)

7. Python lab (leakage demos + plots)

8. Diagnostics and failure modes (validator checklist)

8.1 Universe and filters

8.2 Split design

8.3 Leakage checks

9. Mini-exercises

Exercise 1

Exercise 2 (trap)

Exercise 3

10. Summary

11. Out of scope

Follow new writing