Numerical Tolerances for Regression Tests
Tolerances must reflect trade scale, numerical method, and noise sources, balancing stability against necessary change.
Explanation
Absolute tolerances are suitable for small-magnitude quantities, while relative tolerances scale with the size of the value.
A common rule is |new − old| ≤ abs_tol + rel_tol·|old|, with parameters tailored to product type and currency.
Deterministic methods such as PDE, lattice, or closed-form pricers justify tight tolerances dominated by floating-point limits.
Monte Carlo tolerances should be related to estimated standard errors so tests do not fail on expected sampling variation alone.
toleranceregressionnumericsstability
Interactive visualisation
Numerical tolerances for regression tests
Choose tolerances to balance stability against necessary change.
Tolerance rule
Pass if |new − old| ≤ abs_tol + rel_tol·|old|
abs_tol
0.020
rel_tol
0.0010
Engine and change
Suite failures
0 / 5
Balanced: tests are sensitive but not fragile.
Per-case deviation versus tolerance
Bar shows |new − old|. Band shows allowed tolerance.
FX option (small premium)
old: 0.3800 · new: 0.3800
|diff|: 0.00000 · tol: 0.02038
PASS
0|diff| / tol0.00×
Equity call (normal)
old: 2.5400 · new: 2.5400
|diff|: 0.00000 · tol: 0.02254
PASS
0|diff| / tol0.00×
Rates swap PV (large)
old: 145.2000 · new: 145.2000
|diff|: 0.00000 · tol: 0.16520
PASS
0|diff| / tol0.00×
Commodity spread (mid)
old: 5.8000 · new: 5.8000
|diff|: 0.00000 · tol: 0.02580
PASS
0|diff| / tol0.00×
Credit index PV (large)
old: 980.0000 · new: 980.0000
|diff|: 0.00000 · tol: 1.00000
PASS
0|diff| / tol0.00×
Interpretation
- Absolute tolerance protects small values. Relative tolerance scales for large PVs.
- Deterministic methods justify tight tolerances. Monte Carlo needs SE-aware tolerances.
- Too tight → false fails. Too loose → behaviour drifts without being noticed.