Model Validation

Out-of-sample. Walk-forward. No excuses.

Walk-forward means the model trains only on historical data prior to the prediction date — no future information leaks in. All backtest results shown below are out-of-sample. Go-live requires four independent thresholds: ROI > 3%, CLV ≥ 55%, calibration ±2%, and ≥ 300 paper trade plays.

Go-Live Requirements — All Four Required, No Exceptions

>3%

Backtest ROI

≥55%

CLV Rate

±2%

Calibration

300+

Paper Trade Plays

Backtest Results by Sport

✅ Backtest Complete

🤾 Handball — Efficiency-Flow Model v1.1.0

Brier Score	0.1936 (22.6% above random baseline of 0.25)
Directional Accuracy	70.3%
Simulated ROI	+39%
Sample Size	973 matches
Competitions	EHF CL + Bundesliga
Data Period	2019–2025
Teams Covered	48 teams, 6 seasons

Platt scaling calibration infrastructure added (v1.1.0). PLATT_ENABLED=False pending additional data — overconfidence identified at low-probability end (0–30%), underconfidence at high end (90%+). Live signals are issued without shrinkage until calibration refit threshold (500+ predictions) is reached.

✅ Backtest Complete

🏏 Cricket International — Resource-Value Model v0.1.0

Brier Score (National)	0.2330 post-ELO separation
Brier Score (Franchise/IPL)	0.2552 post-ELO separation
National Sample	3,111 matches, 107 countries
Franchise Sample	1,117 matches (IPL), 18 teams
Data Period	2008–2025

ELO pools separated in v0.5.1 — franchise teams (IPL) and national teams (ICC) now maintain independent ELO ratings. Prior conflation was suppressing predictive accuracy for international matches.

🟡 Paper Trading Active

🏒 NHL — Team ML + Pythagorean Model

Model	Pythagorean Win% (exp = 2.0) + ELO blend
ELO Source	ESPN standings API
Odds Source	ESPN Core API + FanDuel player props
Signals Issued (Mar 21)	12 team ML/totals · 369 player prop lines
Pythagorean RMSE	In calibration (paper trade phase)

NHL team model uses Pythagorean expected win% derived from Goals For / Goals Against. ELO initialized from ESPN power rankings. Three confirmed SNIPE (T1) totals on Mar 21 with 14–23% edge: DAL@MIN Over 5.5, NYI@MTL Over 5.5, BOS@DET Over 5.5.

🟡 Paper Trading Active

⚾ NCAA Baseball — ELO + ISR Model

ELO Source	Warren Nolan (warrennolan.com) — weekly
ISR Source	Boyd's World (boydsbets.com) — weekly
ELO / ISR Blend	60% ELO + 40% ISR
SP ERA Adjustment	±3–4pp per dominant vs. shaky SP matchup
Live Signals Since	March 21, 2026
Signals Issued (Mar 21)	7 plays — 3 T1 FILTHY · 2 T2 WHEELHOUSE · 2 T3 SCOUT

Closing line value (CLV) tracking begins from first live signal date. 30-day paper trade window in progress. Go-live target: April 20, 2026 (pending all 4 thresholds).

Walk-Forward Backtest Protocol

Methodology Transparency — How Every Backtest Is Run

All backtests use walk-forward splits — model trained on data prior to prediction date only. Zero lookahead contamination.
10¢ slippage applied to every simulated entry.
Minimum 3 seasons required before trusting any team ML backtest result.
Calibration bins must be within ±2% across all probability ranges (0–10%, 10–20%, … 90–100%).
Closing line (no-vig) = CLV benchmark. Pinnacle closing line is the gold standard reference.
Paper trading minimum: 30 days before real money on any new model.
All results logged publicly in the Results tab — wins and losses, no cherry-picking.

Data Sources Used in Validation

Primary Data Sources by Sport

Boyd's World— boydsbets.com · ISR ratings, NCAA Baseball, updated weekly
Warren Nolan— warrennolan.com · ELO ratings, NCAA Baseball, updated weekly
Cricsheet— cricsheet.org · T20 ball-by-ball data, Cricket international + IPL
API Sports— EHF / Bundesliga handball fixtures · ICC / IPL cricket fixtures and results
MoneyPuck— moneypuck.com · NHL shot quality, xG data, 2007–present
football-data.co.uk— Soccer results and odds, 6 top leagues, 1993–present
Club Elo— clubelo.com · Soccer ELO ratings, 1939–present