1. Promise

One-sentence promise

After this lecture, you can recognise autocorrelation, seasonality, and volatility clustering, and you can explain how they shape feature design, evaluation, and risk controls.

End tag line

Structure is signal. Structure is also a trap.

2. Learning objectives

After reading this lecture, you will be able to:

  • Explain what autocorrelation is and what it implies for baselines.
  • Use partial autocorrelation as a diagnostic for “direct” dependence.
  • Identify daily and weekly seasonality and avoid false alpha.
  • Describe heteroskedasticity and volatility clustering and why finance returns are not iid.
  • Translate these patterns into modelling actions (features, splits, stability checks).

3. Prerequisites and setup

  • Basic understanding of time-indexed data.
  • No econometrics formulas required.

4. Conceptual core

4.1 Autocorrelation (ACF) — the intuition

What it means

Autocorrelation asks: > Does the past of the series help predict its future?

If a series is positively autocorrelated, it tends to persist. If it is negatively autocorrelated, it tends to revert.

Finance intuition

  • Prices often look persistent (random-walk-like). That does not mean returns are predictable.
  • Returns are often close to zero autocorrelation at daily horizons.
  • Volatility is strongly autocorrelated (high vol follows high vol).

What to do with it

  • If autocorrelation is strong, a last-value or lag baseline will be hard to beat.
  • If autocorrelation is weak, focus on other structure (seasonality, cross-sectional signals, volatility state).

Common trap

Autocorrelation in levels (prices) can trick you into thinking you have predictability. That is often non-stationarity, not signal.

Slide-ready sentence

If the series has memory, baselines become strong and leakage becomes easier.

4.2 Partial autocorrelation (PACF) — the intuition

Why it exists

ACF shows overall correlation with lagged values. But correlation can be indirect.

PACF asks: > After controlling for shorter lags, does lag kk still have a direct effect?

Mental model

  • ACF is “total influence”.
  • PACF is “direct influence not explained by nearer past”.

How to use it (without formulas)

  • If PACF cuts off after lag 1 or 2, short-lag models (AR-like) may be sufficient.
  • If PACF has spikes at specific lags (e.g., 5), that may indicate weekly seasonality or periodic structure.

Finance reality

PACF is more useful for: - volatility proxies, - spreads, - volumes, than for daily returns.

Slide-ready sentence

PACF tells you which lags add information after you already used the recent past.

4.3 Seasonality (daily / weekly)

Seasonality is predictable structure tied to calendar time.

Daily / intraday seasonality

  • Volume and volatility often follow a U-shape across the trading day.
  • Bid–ask spreads widen at open/close and around news.

Weekly seasonality

  • Day-of-week patterns can appear in volume, flows, and sometimes returns.
  • Many “alphas” are just calendar effects that vanish after transaction costs or regime changes.

What seasonality does to modelling

  • Add calendar features:
    • day-of-week,
    • hour bucket,
    • holiday indicator.
  • Use seasonal baselines:
    • “seasonal naive” (use same time last week).
  • Evaluate stability:
    • seasonality can disappear when microstructure or client behaviour changes.

Validation trap

If you shuffle time, you leak seasonality patterns. Time splits are mandatory.

Slide-ready sentence

Seasonality is the easiest structure to exploit and the easiest to overclaim.

4.4 Heteroskedasticity and volatility clustering (finance reality)

What it means

Heteroskedasticity means: > Variance is not constant over time.

Volatility clustering means: > High-volatility periods cluster, and low-volatility periods cluster.

So returns are not iid even when their mean is nearly zero.

Intuition - Calm markets: small daily moves. - Stress markets: large moves, jumps, gaps, liquidity changes.

Why it matters

  • Your model errors are regime-dependent.
  • Risk controls must adapt to volatility state.
  • Scaling and thresholds can break when variance shifts.

Practical modelling consequences

  • Use volatility state as a feature (rolling std, ATR-like proxies).
  • Evaluate by regime:
    • low-vol vs high-vol buckets.
  • Consider robust losses:
    • MAE can be more stable than RMSE under jumps.
  • Add constraints:
    • position sizing rules that shrink exposure in high vol.

Slide-ready sentence

In finance, the mean is hard. The variance moves.

5. Control lens

5.1 Analogue

Autocorrelation is “state persistence”. Seasonality is “periodic forcing”. Volatility clustering is “time-varying noise intensity”.

A control engineer would never assume constant noise variance if it clearly changes with operating mode.

Finance is the same.

6. Practical checklist (what to do tomorrow)

  1. Plot ACF/PACF for:
    • returns,
    • absolute returns or squared returns,
    • volume,
    • spread.
  2. Add seasonal features and seasonal naive baseline.
  3. Include volatility regime in evaluation.
  4. Report metrics by regime and worst-window.
  5. Decide whether your strategy is allowed to operate in high-vol regimes.

7. Diagnostics and failure modes

  1. You model prices and discover strong autocorrelation
    • Likely non-stationarity.
    • Transform to returns or differences.
  2. You see spikes at lag 5
    • Often weekly seasonality.
    • Add weekday features or seasonal baseline.
  3. Your model fails only in crisis weeks
    • Likely volatility clustering / regime shift.
    • Add volatility state, restrict usage, or redesign objective.
  4. Your residuals get wider over time
    • Heteroskedasticity.
    • Scale, bucket by regime, or use robust objectives.

8. Mini-exercises

Exercise 1

If daily returns have near-zero ACF but squared returns have strong ACF, what does it imply?

Solution Mean is hard to predict. Volatility clusters. Model the volatility state and manage risk accordingly.

Exercise 2

You discover a strong pattern at lag 5 for volume. What is the most plausible explanation?

Solution Weekly seasonality (trading week structure).

Exercise 3 (trap)

If ACF shows predictability, you have alpha. Agree or disagree.

Solution Disagree. ACF can reflect non-stationarity, seasonality, or microstructure artefacts. You need decision-level validation and stability checks.

9. Summary

  • ACF shows memory; PACF hints which lags add direct information.
  • Seasonality is predictable calendar structure; exploit it carefully.
  • Finance time series are heteroskedastic; volatility clusters.
  • Dependence structure should drive features, baselines, and evaluation discipline.

10. Out of scope (for this sub-lecture)

  • Formal ARMA/ARIMA estimation details.
  • GARCH modelling and full volatility forecasting theory.
  • Cointegration and multi-series dependence.

Python lab (ACF/PACF intuition, seasonality, volatility clustering)


import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

rng = np.random.default_rng(0)

# ---------------------------------------------------------
# 1) Synthetic series that contains:
#    - weak mean autocorrelation in returns
#    - weekly seasonality (period=5)
#    - volatility clustering via regime switching
# ---------------------------------------------------------
T = 600
t = np.arange(T)

# Weekly seasonality component (period 5)
season = 0.3 * np.sin(2 * np.pi * t / 5)

# Volatility regime switching: low vol then high vol then low vol
sigma = np.ones(T) * 0.8
sigma[200:400] = 2.0

# Innovations with time-varying volatility
eps = rng.normal(0, 1, T) * sigma

# Returns: weak AR(1) in the mean + seasonality + heteroskedastic noise
r = np.zeros(T)
phi = 0.08  # weak mean dependence (often close to 0 in real daily returns)
for k in range(1, T):
    r[k] = phi * r[k-1] + season[k] + eps[k]

# A "volume-like" series with strong weekly structure (more seasonal than returns)
vol = 100 + 20*np.sin(2*np.pi*t/5) + 10*rng.normal(0, 1, T)

df = pd.DataFrame({"r": r, "abs_r": np.abs(r), "r2": r**2, "vol": vol})

# ---------------------------------------------------------
# 2) ACF (manual): correlation at lag k
# ---------------------------------------------------------
def acf(x, max_lag=40):
    x = np.asarray(x, dtype=float)
    x = x - np.mean(x)
    denom = np.sum(x**2) + 1e-12
    out = []
    for k in range(max_lag + 1):
        num = np.sum(x[k:] * x[:len(x)-k])
        out.append(num / denom)
    return np.array(out)

# ---------------------------------------------------------
# 3) PACF intuition (simple residualisation method)
#    For each lag k:
#      regress x_t on lags 1..k-1, take residuals
#      regress x_{t-k} on lags 1..k-1, take residuals
#      corr(residuals) = partial corr
# This is not a full textbook PACF estimator, but it teaches the concept cleanly.
# ---------------------------------------------------------
def pacf_residualised(x, max_lag=20):
    x = np.asarray(x, dtype=float)
    x = x - np.mean(x)
    out = [1.0]
    for k in range(1, max_lag + 1):
        # Align series: y = x_t, z = x_{t-k}
        y = x[k:]
        z = x[:-k]

        if k == 1:
            # No shorter lags to control for
            out.append(np.corrcoef(y, z)[0, 1])
            continue

        # Build control matrix with lags 1..k-1 for time t
        # Controls for y: [x_{t-1},...,x_{t-(k-1)}]
        Y_controls = np.column_stack([x[k-j:len(x)-j] for j in range(1, k)])

        # Controls for z: [x_{t-k-1},...,x_{t-k-(k-1)}] => align with z index
        Z_controls = np.column_stack([x[:-k-j] for j in range(1, k)])

        # Match lengths (they should already match)
        n = min(len(y), len(z), len(Y_controls), len(Z_controls))
        y = y[:n]; z = z[:n]
        Y_controls = Y_controls[:n]
        Z_controls = Z_controls[:n]

        # Residualise y
        Y = np.column_stack([np.ones(n), Y_controls])
        beta_y = np.linalg.lstsq(Y, y, rcond=None)[0]
        y_res = y - Y @ beta_y

        # Residualise z
        Z = np.column_stack([np.ones(n), Z_controls])
        beta_z = np.linalg.lstsq(Z, z, rcond=None)[0]
        z_res = z - Z @ beta_z

        # Partial correlation
        pc = np.corrcoef(y_res, z_res)[0, 1]
        out.append(pc)

    return np.array(out)

# Compute ACFs
max_lag = 30
acf_r = acf(df["r"], max_lag=max_lag)
acf_abs = acf(df["abs_r"], max_lag=max_lag)
acf_r2 = acf(df["r2"], max_lag=max_lag)
acf_vol = acf(df["vol"], max_lag=max_lag)

# Compute PACF (residualised)
max_pacf = 20
pacf_r = pacf_residualised(df["r"], max_lag=max_pacf)
pacf_vol = pacf_residualised(df["vol"], max_lag=max_pacf)

# ---------------------------------------------------------
# 4) Plots: ACF and PACF
# ---------------------------------------------------------
def stem_plot(vals, title):
    plt.figure(figsize=(8, 3.6))
    xs = np.arange(len(vals))
    plt.stem(xs, vals, basefmt=" ")
    plt.axhline(0, linestyle="--")
    plt.title(title)
    plt.xlabel("Lag")
    plt.ylabel("Correlation")
    plt.tight_layout()
    plt.show()

stem_plot(acf_r, "ACF of returns r_t (often weak at daily horizons)")
stem_plot(pacf_r, "PACF of returns r_t (direct lag effects after controlling shorter lags)")

stem_plot(acf_vol, "ACF of volume-like series (strong seasonality shows up clearly)")
stem_plot(pacf_vol, "PACF of volume-like series (direct periodic structure often remains)")

# ---------------------------------------------------------
# 5) Seasonality demo: lag-5 effect (weekly)
# ---------------------------------------------------------
plt.figure(figsize=(8, 3.6))
plt.plot(df["vol"].values[:120], label="volume-like series (first 120)")
plt.title("Weekly seasonality example (period=5) in a volume-like series")
plt.xlabel("t")
plt.ylabel("value")
plt.legend()
plt.tight_layout()
plt.show()

print("ACF at lag 5 (returns):", float(acf_r[5]))
print("ACF at lag 5 (volume-like):", float(acf_vol[5]))

# ---------------------------------------------------------
# 6) Volatility clustering demo:
#    Compare ACF of r_t vs ACF of |r_t| and r_t^2
# ---------------------------------------------------------
plt.figure(figsize=(8, 3.6))
plt.plot(acf_r, label="ACF(r)")
plt.plot(acf_abs, label="ACF(|r|)")
plt.plot(acf_r2, label="ACF(r^2)")
plt.axhline(0, linestyle="--")
plt.title("Volatility clustering: |r| and r^2 have strong dependence even if r does not")
plt.xlabel("Lag")
plt.ylabel("ACF")
plt.legend()
plt.tight_layout()
plt.show()

# ---------------------------------------------------------
# 7) Mini-demo: random split leaks seasonality
#    We predict vol_t using a naive seasonal rule: vol_{t-5}
#    Compare error when using proper time split vs random split.
# ---------------------------------------------------------
df2 = df.copy()
df2["vol_lag5"] = df2["vol"].shift(5)
df2 = df2.dropna().reset_index(drop=True)

y_true = df2["vol"].values
y_hat = df2["vol_lag5"].values  # seasonal naive predictor

def mse(a, b):
    return float(np.mean((a - b) ** 2))

# Proper time split
split = int(0.7 * len(df2))
mse_time = mse(y_true[split:], y_hat[split:])

# Random split (WRONG for time series)
idx = np.arange(len(df2))
rng.shuffle(idx)
test_idx = idx[:len(df2)-split]
mse_random = mse(y_true[test_idx], y_hat[test_idx])

print("\n=== Seasonality leakage demo ===")
print("MSE with proper time split:", mse_time)
print("MSE with random split (leaky):", mse_random)
print("Note: random split mixes regimes/seasonality and can exaggerate performance.")