Files

148 lines
5.5 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Backtest: DR artist × channel × week
Panel: 3246 artists × 14 channels × 51 complete weeks (2025-11-17 to 2026-11-02). Forecast origins from week 27; regressions refitted every 2 origin(s).
"In rotation" = at least τ plays in the week: p3 23, p4kbh 15, p4sjaelland 14, p4fyn 14, p4syd 14, p4trekanten 14, p4vest 13, p4aarhus 14, p4nord 15, p4bornholm 14, p4esbjerg 14, p5 10, p6beat 16, p8jazz 7.
## h = 1 week(s) ahead (24 forecast weeks)
| Model | Log-loss, all | Log-loss, adds | Brier, adds | PR-AUC, adds | MAE plays | Spearman | NDCG@50 |
|---|---|---|---|---|---|---|---|
| persistence | 0.0217 | 0.0159 | 0.0023 | 0.003 | 0.27 | 0.826 | 0.967 |
| popularity | 0.0216 | 0.0110 | 0.0028 | 0.170 | 0.31 | 0.864 | 0.886 |
| kalman | 0.0156 | 0.0088 | 0.0019 | 0.316 | 0.30 | 0.872 | 0.951 |
| nb | 0.0131 | 0.0080 | 0.0017 | 0.330 | 0.27 | 0.856 | 0.960 |
| logit | 0.0113 | 0.0066 | 0.0017 | 0.359 | 0.27 | 0.856 | 0.960 |
Skill on adds (1 − log-loss ÷ baseline log-loss; 90% block-bootstrap interval over weeks):
| Model | vs persistence | vs popularity |
|---|---|---|
| kalman | 0.447 (0.376 to 0.510) | 0.204 (0.136 to 0.266) |
| nb | 0.497 (0.439 to 0.550) | 0.274 (0.204 to 0.337) |
| logit | 0.582 (0.554 to 0.614) | 0.398 (0.368 to 0.436) |
Calibration of the NB's add probabilities (rows not in rotation at the origin):
| Predicted | Rows | Mean predicted | Observed |
|---|---|---|---|
| 0.0–0.1 | 1067535 | 0.000 | 0.001 |
| 0.1–0.2 | 2694 | 0.146 | 0.129 |
| 0.2–0.3 | 1559 | 0.247 | 0.246 |
| 0.3–0.4 | 1113 | 0.347 | 0.340 |
| 0.4–0.5 | 697 | 0.449 | 0.420 |
| 0.5–0.6 | 444 | 0.543 | 0.482 |
| 0.6–0.7 | 212 | 0.649 | 0.448 |
| 0.7–0.8 | 96 | 0.740 | 0.417 |
| 0.8–0.9 | 63 | 0.840 | 0.508 |
| 0.9–1.0 | 19 | 0.950 | 0.421 |
## h = 4 week(s) ahead (21 forecast weeks)
| Model | Log-loss, all | Log-loss, adds | Brier, adds | PR-AUC, adds | MAE plays | Spearman | NDCG@50 |
|---|---|---|---|---|---|---|---|
| persistence | 0.0318 | 0.0242 | 0.0037 | 0.005 | 0.33 | 0.811 | 0.907 |
| popularity | 0.0284 | 0.0168 | 0.0039 | 0.145 | 0.36 | 0.845 | 0.829 |
| kalman | 0.0283 | 0.0196 | 0.0033 | 0.227 | 0.39 | 0.850 | 0.887 |
| nb | 0.0257 | 0.0186 | 0.0031 | 0.251 | 0.34 | 0.835 | 0.898 |
| logit | 0.0198 | 0.0127 | 0.0029 | 0.298 | 0.34 | 0.835 | 0.898 |
Skill on adds (1 − log-loss ÷ baseline log-loss; 90% block-bootstrap interval over weeks):
| Model | vs persistence | vs popularity |
|---|---|---|
| kalman | 0.191 (0.046 to 0.325) | -0.170 (-0.329 to -0.038) |
| nb | 0.235 (0.093 to 0.362) | -0.107 (-0.283 to 0.030) |
| logit | 0.477 (0.420 to 0.536) | 0.244 (0.175 to 0.293) |
Calibration of the NB's add probabilities (rows not in rotation at the origin):
| Predicted | Rows | Mean predicted | Observed |
|---|---|---|---|
| 0.0–0.1 | 931278 | 0.001 | 0.002 |
| 0.1–0.2 | 3415 | 0.143 | 0.094 |
| 0.2–0.3 | 2051 | 0.247 | 0.207 |
| 0.3–0.4 | 1566 | 0.348 | 0.287 |
| 0.4–0.5 | 1010 | 0.444 | 0.413 |
| 0.5–0.6 | 528 | 0.545 | 0.443 |
| 0.6–0.7 | 235 | 0.641 | 0.306 |
| 0.7–0.8 | 94 | 0.748 | 0.521 |
| 0.8–0.9 | 35 | 0.839 | 0.457 |
| 0.9–1.0 | 3 | 0.927 | 0.000 |
## h = 8 week(s) ahead (17 forecast weeks)
| Model | Log-loss, all | Log-loss, adds | Brier, adds | PR-AUC, adds | MAE plays | Spearman | NDCG@50 |
|---|---|---|---|---|---|---|---|
| persistence | 0.0433 | 0.0344 | 0.0055 | 0.007 | 0.40 | 0.783 | 0.833 |
| popularity | 0.0359 | 0.0241 | 0.0052 | 0.152 | 0.40 | 0.823 | 0.777 |
| kalman | 0.0427 | 0.0330 | 0.0050 | 0.195 | 0.51 | 0.821 | 0.816 |
| nb | 0.0418 | 0.0330 | 0.0048 | 0.233 | 0.43 | 0.804 | 0.812 |
| logit | 0.0301 | 0.0208 | 0.0046 | 0.280 | 0.43 | 0.804 | 0.812 |
Skill on adds (1 − log-loss ÷ baseline log-loss; 90% block-bootstrap interval over weeks):
| Model | vs persistence | vs popularity |
|---|---|---|
| kalman | 0.040 (-0.078 to 0.167) | -0.367 (-0.490 to -0.223) |
| nb | 0.042 (-0.097 to 0.159) | -0.365 (-0.522 to -0.228) |
| logit | 0.394 (0.348 to 0.433) | 0.137 (0.089 to 0.173) |
Calibration of the NB's add probabilities (rows not in rotation at the origin):
| Predicted | Rows | Mean predicted | Observed |
|---|---|---|---|
| 0.0–0.1 | 751167 | 0.001 | 0.003 |
| 0.1–0.2 | 4090 | 0.145 | 0.084 |
| 0.2–0.3 | 2546 | 0.248 | 0.209 |
| 0.3–0.4 | 1673 | 0.348 | 0.289 |
| 0.4–0.5 | 976 | 0.443 | 0.416 |
| 0.5–0.6 | 429 | 0.544 | 0.445 |
| 0.6–0.7 | 208 | 0.646 | 0.409 |
| 0.7–0.8 | 76 | 0.740 | 0.434 |
| 0.8–0.9 | 25 | 0.828 | 0.880 |
## NB coefficients, h = 4, all weeks (alpha = 0.134)
| Feature | Coefficient |
|---|---|
| intercept | -0.787 |
| log plays t | +0.313 |
| log plays t-1 | +0.140 |
| log mean plays 4w | +0.480 |
| log mean plays 12w | -0.057 |
| log max plays 12w | +0.368 |
| log other channels t | -0.018 |
| log other channels 4w | +0.044 |
| log P3 t | +0.014 |
| weeks since played (/12) | -3.164 |
| Track Top-40 score | -0.021 |
| Airplay Top-20 score | -0.418 |
| chart entry recency 6w | +0.539 |
| chart entry recency 26w | -0.383 |
| log Track Top-40 hits | +0.032 |
| log Airplay Top-20 hits | +0.005 |
| Danish | -0.007 |
| group | -0.053 |
| female | +0.021 |
| career years (/50) | -0.229 |
| debut unknown | -0.059 |
| Christmas target week | +0.000 |
| Danish x P3 | -0.001 |
| Danish x P4 | +0.020 |
| Danish x P6 | +0.007 |
| ch p4kbh | -0.013 |
| ch p4sjaelland | -0.009 |
| ch p4fyn | -0.015 |
| ch p4syd | -0.010 |
| ch p4trekanten | -0.012 |
| ch p4vest | -0.010 |
| ch p4aarhus | -0.010 |
| ch p4nord | -0.007 |
| ch p4bornholm | -0.013 |
| ch p4esbjerg | -0.011 |
| ch p5 | +0.118 |
| ch p6beat | +0.066 |
| ch p8jazz | +0.103 |