Models, first layer: artist × DR channel × week panel with IFPI chart history; NB and logit direct h-step models, Kalman artist form, persistence/popularity baselines; walk-forward backtest (docs/BACKTEST.md) and the ranking (forecasts table)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-15 02:08:48 +00:00
co-authored by Claude Opus 5.5
parent 6272fbd75d
commit eededc7571
12 changed files with 1318 additions and 1 deletions
+148
View File
@@ -0,0 +1,148 @@
# Backtest: DR artist × channel × week
Panel: 2329 artists × 14 channels × 51 complete weeks (2025-11-17 to 2026-11-02). Forecast origins from week 27; regressions refitted every 1 origin(s).
"In rotation" = at least τ plays in the week: p3 7, p4kbh 5, p4sjaelland 5, p4fyn 5, p4syd 5, p4trekanten 5, p4vest 5, p4aarhus 5, p4nord 5, p4bornholm 5, p4esbjerg 5, p5 4, p6beat 5, p8jazz 3.
## h = 1 week(s) ahead (24 forecast weeks)
| Model | Log-loss, all | Log-loss, adds | Brier, adds | PR-AUC, adds | MAE plays | Spearman | NDCG@50 |
|---|---|---|---|---|---|---|---|
| persistence | 0.0683 | 0.0488 | 0.0085 | 0.013 | 0.28 | 0.829 | 0.967 |
| popularity | 0.0470 | 0.0287 | 0.0080 | 0.256 | 0.35 | 0.864 | 0.888 |
| kalman | 0.0385 | 0.0256 | 0.0065 | 0.377 | 0.32 | 0.874 | 0.957 |
| nb | 0.0389 | 0.0260 | 0.0066 | 0.355 | 0.28 | 0.859 | 0.965 |
| logit | 0.0363 | 0.0241 | 0.0064 | 0.367 | 0.28 | 0.859 | 0.965 |
Skill on adds (1 − log-loss ÷ baseline log-loss; 90% block-bootstrap interval over weeks):
| Model | vs persistence | vs popularity |
|---|---|---|
| kalman | 0.475 (0.451 to 0.496) | 0.109 (0.063 to 0.141) |
| nb | 0.466 (0.445 to 0.486) | 0.094 (0.051 to 0.123) |
| logit | 0.505 (0.489 to 0.520) | 0.160 (0.129 to 0.184) |
Calibration of the NB's add probabilities (rows not in rotation at the origin):
| Predicted | Rows | Mean predicted | Observed |
|---|---|---|---|
| 0.0–0.1 | 716676 | 0.003 | 0.002 |
| 0.1–0.2 | 15028 | 0.143 | 0.088 |
| 0.2–0.3 | 6884 | 0.244 | 0.219 |
| 0.3–0.4 | 3000 | 0.343 | 0.391 |
| 0.4–0.5 | 1355 | 0.441 | 0.578 |
| 0.5–0.6 | 521 | 0.540 | 0.551 |
| 0.6–0.7 | 197 | 0.650 | 0.482 |
| 0.7–0.8 | 108 | 0.748 | 0.509 |
| 0.8–0.9 | 95 | 0.847 | 0.211 |
| 0.9–1.0 | 31 | 0.941 | 0.258 |
## h = 4 week(s) ahead (21 forecast weeks)
| Model | Log-loss, all | Log-loss, adds | Brier, adds | PR-AUC, adds | MAE plays | Spearman | NDCG@50 |
|---|---|---|---|---|---|---|---|
| persistence | 0.0773 | 0.0560 | 0.0100 | 0.014 | 0.36 | 0.810 | 0.908 |
| popularity | 0.0563 | 0.0350 | 0.0091 | 0.246 | 0.41 | 0.841 | 0.834 |
| kalman | 0.0536 | 0.0346 | 0.0081 | 0.347 | 0.43 | 0.849 | 0.895 |
| nb | 0.0543 | 0.0371 | 0.0081 | 0.319 | 0.37 | 0.835 | 0.909 |
| logit | 0.0484 | 0.0319 | 0.0079 | 0.325 | 0.37 | 0.835 | 0.909 |
Skill on adds (1 − log-loss ÷ baseline log-loss; 90% block-bootstrap interval over weeks):
| Model | vs persistence | vs popularity |
|---|---|---|
| kalman | 0.383 (0.333 to 0.430) | 0.012 (-0.045 to 0.048) |
| nb | 0.337 (0.254 to 0.407) | -0.061 (-0.165 to 0.010) |
| logit | 0.430 (0.398 to 0.462) | 0.088 (0.056 to 0.102) |
Calibration of the NB's add probabilities (rows not in rotation at the origin):
| Predicted | Rows | Mean predicted | Observed |
|---|---|---|---|
| 0.0–0.1 | 622067 | 0.003 | 0.002 |
| 0.1–0.2 | 15752 | 0.144 | 0.083 |
| 0.2–0.3 | 7411 | 0.245 | 0.203 |
| 0.3–0.4 | 3454 | 0.343 | 0.359 |
| 0.4–0.5 | 1395 | 0.442 | 0.463 |
| 0.5–0.6 | 487 | 0.545 | 0.591 |
| 0.6–0.7 | 212 | 0.646 | 0.538 |
| 0.7–0.8 | 125 | 0.747 | 0.208 |
| 0.8–0.9 | 76 | 0.844 | 0.579 |
| 0.9–1.0 | 21 | 0.933 | 0.667 |
## h = 8 week(s) ahead (17 forecast weeks)
| Model | Log-loss, all | Log-loss, adds | Brier, adds | PR-AUC, adds | MAE plays | Spearman | NDCG@50 |
|---|---|---|---|---|---|---|---|
| persistence | 0.0874 | 0.0648 | 0.0119 | 0.017 | 0.45 | 0.779 | 0.837 |
| popularity | 0.0679 | 0.0445 | 0.0107 | 0.236 | 0.46 | 0.815 | 0.784 |
| kalman | 0.0703 | 0.0473 | 0.0105 | 0.293 | 0.57 | 0.815 | 0.825 |
| nb | 0.0774 | 0.0570 | 0.0103 | 0.270 | 0.48 | 0.797 | 0.828 |
| logit | 0.0634 | 0.0432 | 0.0101 | 0.262 | 0.48 | 0.797 | 0.828 |
Skill on adds (1 − log-loss ÷ baseline log-loss; 90% block-bootstrap interval over weeks):
| Model | vs persistence | vs popularity |
|---|---|---|
| kalman | 0.271 (0.219 to 0.305) | -0.064 (-0.100 to -0.046) |
| nb | 0.120 (-0.012 to 0.209) | -0.283 (-0.418 to -0.196) |
| logit | 0.334 (0.289 to 0.366) | 0.028 (-0.001 to 0.043) |
Calibration of the NB's add probabilities (rows not in rotation at the origin):
| Predicted | Rows | Mean predicted | Observed |
|---|---|---|---|
| 0.0–0.1 | 497574 | 0.004 | 0.004 |
| 0.1–0.2 | 15374 | 0.143 | 0.071 |
| 0.2–0.3 | 7616 | 0.244 | 0.160 |
| 0.3–0.4 | 4057 | 0.343 | 0.263 |
| 0.4–0.5 | 1630 | 0.441 | 0.426 |
| 0.5–0.6 | 493 | 0.544 | 0.519 |
| 0.6–0.7 | 264 | 0.637 | 0.523 |
| 0.7–0.8 | 79 | 0.751 | 0.456 |
| 0.8–0.9 | 63 | 0.837 | 0.476 |
| 0.9–1.0 | 15 | 0.922 | 0.733 |
## NB coefficients, h = 4, all weeks (alpha = 0.116)
| Feature | Coefficient |
|---|---|
| intercept | -0.694 |
| log plays t | +0.380 |
| log plays t-1 | +0.140 |
| log mean plays 4w | +0.447 |
| log mean plays 12w | -0.190 |
| log max plays 12w | +0.420 |
| log other channels t | -0.024 |
| log other channels 4w | +0.045 |
| log P3 t | +0.020 |
| weeks since played (/12) | -3.147 |
| Track Top-40 score | -0.018 |
| Airplay Top-20 score | -0.397 |
| chart entry recency 6w | +0.425 |
| chart entry recency 26w | -0.275 |
| log Track Top-40 hits | +0.029 |
| log Airplay Top-20 hits | +0.019 |
| Danish | -0.013 |
| group | -0.091 |
| female | +0.022 |
| career years (/50) | -0.269 |
| debut unknown | -0.080 |
| Christmas target week | +0.000 |
| Danish x P3 | +0.007 |
| Danish x P4 | +0.039 |
| Danish x P6 | +0.020 |
| ch p4kbh | -0.043 |
| ch p4sjaelland | -0.039 |
| ch p4fyn | -0.046 |
| ch p4syd | -0.040 |
| ch p4trekanten | -0.044 |
| ch p4vest | -0.041 |
| ch p4aarhus | -0.040 |
| ch p4nord | -0.037 |
| ch p4bornholm | -0.043 |
| ch p4esbjerg | -0.041 |
| ch p5 | +0.092 |
| ch p6beat | +0.044 |
| ch p8jazz | +0.085 |