Hockey Scout Data Lab

Destiny accuracy · labs

Stress Testing

Every major FlameWire Calc / NHLe adjustment we ran on 6-tier draft-day destiny, how predictability moved, and what we shipped. Hard path champion: fw_calc_v0_9_1 PDA + nhle_v1_sched · 37.3% exact / 86.8% within-1 (n=365).

Regressions → Methodologies

Live hard path (drafted)

FlameWire PDA (live)

Method fw_calc_v0_9_4 · weights P0.4/D0.3/A0.3

Star≥96 · Top-6≥74 · Middle-6≥62 · Bottom-6≥41 · Bubble≥26

86.8% Within-1 tier
37.3% Exact tier
0.58 Spearman destiny
0.47 Spearman 7yr GP

Ship rule: Change hard score/cuts only if CV within-1 improves without a large exact collapse (or a clear joint win). Floors: majority-class exact ≈27.0% · random 6-way ≈17.0%.

Accuracy progression

How draft-day destiny hit rates moved as the formula matured. Filled markers = steps that shipped (or match the live champ). Dashed teal line = current within-1 champion (86.8%).

Destiny accuracy progression 20%40%60%80%100% Champ w1 86.8% Random 6-wayMajority classNHLe → role onlyearly (pre-v0.6)v0.6 PDA retunev0.9.1 + soft/…NHLe schedule-…v0.9.4 pre-dra… ● Exact % ● Within-1 % Filled = shipped step
# Step Date Exact Within-1 Shipped? Notes
0 Random 6-way — 16.7% 44.0% No Uniform guess across 6 destiny tiers.
1 Majority class — 27.0% 55.0% No Always predict most common destiny band (~bottom-6 / middle mass).
2 NHLe → role only early 24.1% 68.0% No Map NHLe role band alone (no draft capital / age PDA). Weak on destiny.
3 Draft order only — — — No Pick ranking Spearman vs 7yr GP ≈0.45 — capital is real but not a full role model.
4 FW early (pre-v0.6) 2026-08 34.0% 84.0% No Earlier weight/cut set before draft-day destiny retune (approx).
5 FW v0.6 PDA retune 2026-08 37.0% 86.8% Yes P0.40/D0.30/A0.30 + agree + D-boost + destiny cuts. Core champ formula.
6 FW v0.9.1 + soft/hurdle side 2026-08-12 37.0% 86.8% Yes Hard path unchanged; soft probs + hurdle attached as side channels.
7 NHLe schedule-norm (shipped) 2026-08-13 37.3% 86.8% Yes nhle_v1_sched sample weights. Exact +0.3pp; w1 held. Live drafted champ.
8 v0.9.4 pre-draft elite (undrafted) 2026-08-13 37.3% 86.8% Yes Drafted CV unchanged. Pre-pick board only (DuPont-class capital).

Experiment space (draft-day)

Each point is a lab configuration on the drafted destiny set (n≈365 unless noted). Upper-right is better (higher exact + higher within-1). Green = shipped; grey = tried and not shipped; teal crosshair = live champ. Many “higher exact” models sit left/down on within-1 — that fails the ship rule.

Experiment space: exact vs within-1 20253035405060708090 Random 6-way: 16.7% / 44.0%Majority class: 27.0% / 55.0%NHLe role map only: 24.1% / 68.0%FW PDA v0.9.1: 37.0% / 86.8%FW + schedule-norm NHLe: 37.3% / 86.8%F/D separate PDA weights: 36.5% / 86.2%Active relative → +P: 35.8% / 84.5%CS blend into drafted D: 37.0% / 86.5%Age×rate / 1st major-jr boost: 35.6% / 84.1%Size residual: 36.8% / 86.5%Model-tree pick bands: 36.7% / 86.3%PCS cohorts: 35.2% / 83.8%Ridge / kNN / hand → p82 → role: 38.4% / 83.0%Learned league factors → role: 38.0% / 83.0%Stack FW+ridge+learned (year-LO): 38.9% / 84.7%Expand min GP=20: 36.1% / 84.9%Full proto (rel+P + broad ceil): 35.9% / 83.3%Tight p98 + top3/5/10 ceil: 36.4% / 83.0%Narrow u17-p98 + top3 ceil+8: 37.5% / 84.9%Ceiling-only top3+ultra +11: 37.5% / 85.8%Hurdle hard tiers (5-fold): 32.3% / 81.1%Hurdle hard (year holdout): 34.5% / 82.7%Hard-apply kNN residual: 35.1% / 85.8%Tightened elite capital (drafted CV): 37.3% / 86.8% Exact tier % → Within-1 % → ● Shipped ● Lab only ◎ Live champ

All combinations & labs

Catalog of formula, feature, model, and undrafted-path experiments. Undrafted CS rows use a different sample (first-NHL undrafted NHLers) — do not mix with draft-day %.

Draft-day destiny (n≈365)

Family Experiment Exact W1 n Ship Δ w1 vs champ Notes
Baseline Random 6-way 16.7% 44.0% 365 Lab -42.8pp Chance floor.
Baseline Majority class 27.0% 55.0% 365 Lab -31.8pp Always most common tier.
Baseline NHLe role map only 24.1% 68.0% 365 Lab -18.8pp No PDA / draft capital.
Champ FW PDA v0.9.1 37.0% 86.8% 365 Ship +0.0pp Core hard path after destiny retune.
Champ FW + schedule-norm NHLe 37.3% 86.8% 365 Ship +0.0pp Live drafted champion.
Formula lab F/D separate PDA weights 36.5% 86.2% 365 Lab -0.6pp ≤ baseline joint.
Formula lab Active relative → +P 35.8% 84.5% 365 Lab -2.3pp Usually hurt within-1.
Formula lab CS blend into drafted D 37.0% 86.5% 365 Lab -0.3pp Noise (~0.3–0.9pp); collinear with pick.
Formula lab Age×rate / 1st major-jr boost 35.6% 84.1% 365 Lab -2.7pp Hurt both metrics.
Formula lab Size residual 36.8% 86.5% 365 Lab -0.3pp No clear win.
Formula lab Model-tree pick bands 36.7% 86.3% 365 Lab -0.5pp ≤ baseline.
Formula lab PCS cohorts 35.2% 83.8% 365 Lab -3.0pp Hurt w1.
Non-NHLe / ML Ridge / kNN / hand → p82 → role 38.4% 83.0% 365 Lab -3.8pp Best exact lab; lost within-1 (ship rule fail).
Non-NHLe / ML Learned league factors → role 38.0% 83.0% 365 Lab -3.8pp Exact up, w1 down.
Non-NHLe / ML Stack FW+ridge+learned (year-LO) 38.9% 84.7% 365 Lab -2.1pp Higher exact; still loses joint w1 vs champ.
Sample Expand min GP=20 36.1% 84.9% 416 Lab -1.9pp Noisier destiny labels.
Proto 1+2 Full proto (rel+P + broad ceil) 35.9% 83.3% 365 Lab -3.5pp MacKinnon→Star case win; CV fail (false Stars).
Proto 1+2 Tight p98 + top3/5/10 ceil 36.4% 83.0% 365 Lab -3.8pp Still too broad.
Proto 1+2 Narrow u17-p98 + top3 ceil+8 37.5% 84.9% 365 Lab -1.9pp Exact up; w1 still below champ.
Proto 1+2 Ceiling-only top3+ultra +11 37.5% 85.8% 365 Lab -1.0pp Best proto joint; still −1pp w1.
Hurdle Hurdle hard tiers (5-fold) 32.3% 81.1% 365 Lab -5.7pp Lost hard-tier CV. Kept as P(regular) side channel.
Hurdle Hurdle hard (year holdout) 34.5% 82.7% 365 Lab -4.1pp Paper Spearman competitive; tiers not.
Historical comps Hard-apply kNN residual 35.1% 85.8% 365 Lab -1.0pp Worse than PDA. Side channel only.
Pre-draft elite Tightened elite capital (drafted CV) 37.3% 86.8% 365 Ship +0.0pp Undrafted-only; drafted CV identical to schedule-norm champ.

Undrafted path (first-NHL destiny, n=137)

Family Experiment Exact W1 Ship Notes
Undrafted CS Ladder only 32.8% 81.8% Lab Undrafted NHLers first-NHL as-of (not draft-day drafted set).
Undrafted CS max(ladder, CS) 35.0% 83.9% Ship Shipped for undrafted D. CS-only slice 44.2 / 88.5.
Undrafted CS blend_v2 + mid→final 34.3% 83.2% Lab Lost ~2pp on CS slice vs max.

Correlations & paper metrics

Ranking quality beyond tier hits. FW beats draft order slightly on 7yr GP Spearman; hurdle year-holdout is competitive on paper but loses hard 6-tier within-1.

Ranking / correlation metrics FW score vs destiny ordinal0.582FW score vs 7yr NHL GP0.467Draft order vs 7yr NHL GP0.446Hurdle OOS vs 7yr GP (5-fold)0.465Hurdle year-holdout vs 7yr GP0.472CS final rank vs destiny (drafted set)0.540NHLe vs actual P/82 (draft-day)0.593
Metric Value n Kind Notes
FW score vs destiny ordinal 0.582 365 spearman Draft-day FW vs observed tier rank.
FW score vs 7yr NHL GP 0.467 867 spearman Paper-style; draft ≤2019 window.
Draft order vs 7yr NHL GP 0.446 867 spearman Pick ranking baseline.
Hurdle OOS vs 7yr GP (5-fold) 0.465 867 spearman Matches FW; does not beat tier w1.
Hurdle year-holdout vs 7yr GP 0.472 867 spearman Slight edge on paper metric only.
CS final rank vs destiny (drafted set) 0.540 365 spearman ≈ pick (~0.54); collinear — not shipped into drafted D.
NHLe vs actual P/82 (draft-day) 0.593 365 pearson Rate signal present; role tiers need capital + age.

Shipped scoring path

Hard destiny path (drafted)

PDA weights 0.40/0.30/0.30, destiny cuts, agree term, defense boost +6, young-elite ceiling only when real pick capital exists, schedule-normalized NHLe sample weights. Live: ~37.3% exact / 86.8% within-1 on n=365.

Side channels (not hard score)

Soft midpoint tier probabilities, hurdle P(regular), historical comps neighbors / p_star. Useful for UI and paper ranking; hard-applying them hurt within-1.

Undrafted / pre-draft only

CS max(ladder, rank score); tightened pre-draft elite capital from age-adj dominance + NHLe; 2nd/3rd-year re-entry keeps 52%/28% of capital above the ladder. Does not change drafted draft-day CV.

Explicitly not shipped to hard path

Proto 1+2 extreme rel→P / bigger top-3 ceiling, hurdle hard tiers, comps residual on hard score, CS into drafted D, blend_v2 CS residual, ridge/stack models that trade within-1 for exact.

Ship decision board

Decision Item Why
Ship FW PDA v0.6–v0.9.1 hard tiers Best joint within-1 then exact on draft-day destiny.
Ship Schedule-normalized NHLe Exact +0.3pp, within-1 held.
Ship (side) Soft probs + hurdle + historical comps UI / paper signal without hard-tier regression.
Ship (undrafted) CS max + pre-draft elite capital Board realism for pre-pick elites; drafted CV untouched.
Do not ship Proto 1+2 / hard comps / hurdle hard / ridge stacks Within-1 fell or exact-only wins violated ship rule.

Takeaway

Across baselines, feature labs, proto ceiling/rel boosts, hurdle hard tiers, comps residual, and non-NHLe stacks, no configuration beat FlameWire PDA on joint within-1 then exact on the draft-day 6-tier destiny set. Models that lifted exact% usually gave back within-1 — the metric we optimize first. Live methodology is therefore the best-supported hard path we can ship today: 37.3% exact / 86.8% within-1 (n=365), with undrafted-only capital priors that leave this CV unchanged.

Stance: this is the champ to beat, not proof of a global ceiling. Next gains need signals orthogonal to junior box + pick capital (scouting text, play-driving, D role labels, mature-label slices). Full lab notes: DESTINY_ACCURACY.md. Live resample: Regressions.