Files

5.5 KiB
Raw Permalink Blame History

Backtest: DR artist × channel × week

Panel: 3246 artists × 14 channels × 51 complete weeks (2025-11-17 to 2026-11-02). Forecast origins from week 27; regressions refitted every 2 origin(s).

"In rotation" = at least τ plays in the week: p3 23, p4kbh 15, p4sjaelland 14, p4fyn 14, p4syd 14, p4trekanten 14, p4vest 13, p4aarhus 14, p4nord 15, p4bornholm 14, p4esbjerg 14, p5 10, p6beat 16, p8jazz 7.

h = 1 week(s) ahead (24 forecast weeks)

Model Log-loss, all Log-loss, adds Brier, adds PR-AUC, adds MAE plays Spearman NDCG@50
persistence 0.0217 0.0159 0.0023 0.003 0.27 0.826 0.967
popularity 0.0216 0.0110 0.0028 0.170 0.31 0.864 0.886
kalman 0.0156 0.0088 0.0019 0.316 0.30 0.872 0.951
nb 0.0131 0.0080 0.0017 0.330 0.27 0.856 0.960
logit 0.0113 0.0066 0.0017 0.359 0.27 0.856 0.960

Skill on adds (1 − log-loss ÷ baseline log-loss; 90% block-bootstrap interval over weeks):

Model vs persistence vs popularity
kalman 0.447 (0.376 to 0.510) 0.204 (0.136 to 0.266)
nb 0.497 (0.439 to 0.550) 0.274 (0.204 to 0.337)
logit 0.582 (0.554 to 0.614) 0.398 (0.368 to 0.436)

Calibration of the NB's add probabilities (rows not in rotation at the origin):

Predicted Rows Mean predicted Observed
0.0–0.1 1067535 0.000 0.001
0.1–0.2 2694 0.146 0.129
0.2–0.3 1559 0.247 0.246
0.3–0.4 1113 0.347 0.340
0.4–0.5 697 0.449 0.420
0.5–0.6 444 0.543 0.482
0.6–0.7 212 0.649 0.448
0.7–0.8 96 0.740 0.417
0.8–0.9 63 0.840 0.508
0.9–1.0 19 0.950 0.421

h = 4 week(s) ahead (21 forecast weeks)

Model Log-loss, all Log-loss, adds Brier, adds PR-AUC, adds MAE plays Spearman NDCG@50
persistence 0.0318 0.0242 0.0037 0.005 0.33 0.811 0.907
popularity 0.0284 0.0168 0.0039 0.145 0.36 0.845 0.829
kalman 0.0283 0.0196 0.0033 0.227 0.39 0.850 0.887
nb 0.0257 0.0186 0.0031 0.251 0.34 0.835 0.898
logit 0.0198 0.0127 0.0029 0.298 0.34 0.835 0.898

Skill on adds (1 − log-loss ÷ baseline log-loss; 90% block-bootstrap interval over weeks):

Model vs persistence vs popularity
kalman 0.191 (0.046 to 0.325) -0.170 (-0.329 to -0.038)
nb 0.235 (0.093 to 0.362) -0.107 (-0.283 to 0.030)
logit 0.477 (0.420 to 0.536) 0.244 (0.175 to 0.293)

Calibration of the NB's add probabilities (rows not in rotation at the origin):

Predicted Rows Mean predicted Observed
0.0–0.1 931278 0.001 0.002
0.1–0.2 3415 0.143 0.094
0.2–0.3 2051 0.247 0.207
0.3–0.4 1566 0.348 0.287
0.4–0.5 1010 0.444 0.413
0.5–0.6 528 0.545 0.443
0.6–0.7 235 0.641 0.306
0.7–0.8 94 0.748 0.521
0.8–0.9 35 0.839 0.457
0.9–1.0 3 0.927 0.000

h = 8 week(s) ahead (17 forecast weeks)

Model Log-loss, all Log-loss, adds Brier, adds PR-AUC, adds MAE plays Spearman NDCG@50
persistence 0.0433 0.0344 0.0055 0.007 0.40 0.783 0.833
popularity 0.0359 0.0241 0.0052 0.152 0.40 0.823 0.777
kalman 0.0427 0.0330 0.0050 0.195 0.51 0.821 0.816
nb 0.0418 0.0330 0.0048 0.233 0.43 0.804 0.812
logit 0.0301 0.0208 0.0046 0.280 0.43 0.804 0.812

Skill on adds (1 − log-loss ÷ baseline log-loss; 90% block-bootstrap interval over weeks):

Model vs persistence vs popularity
kalman 0.040 (-0.078 to 0.167) -0.367 (-0.490 to -0.223)
nb 0.042 (-0.097 to 0.159) -0.365 (-0.522 to -0.228)
logit 0.394 (0.348 to 0.433) 0.137 (0.089 to 0.173)

Calibration of the NB's add probabilities (rows not in rotation at the origin):

Predicted Rows Mean predicted Observed
0.0–0.1 751167 0.001 0.003
0.1–0.2 4090 0.145 0.084
0.2–0.3 2546 0.248 0.209
0.3–0.4 1673 0.348 0.289
0.4–0.5 976 0.443 0.416
0.5–0.6 429 0.544 0.445
0.6–0.7 208 0.646 0.409
0.7–0.8 76 0.740 0.434
0.8–0.9 25 0.828 0.880

NB coefficients, h = 4, all weeks (alpha = 0.134)

Feature Coefficient
intercept -0.787
log plays t +0.313
log plays t-1 +0.140
log mean plays 4w +0.480
log mean plays 12w -0.057
log max plays 12w +0.368
log other channels t -0.018
log other channels 4w +0.044
log P3 t +0.014
weeks since played (/12) -3.164
Track Top-40 score -0.021
Airplay Top-20 score -0.418
chart entry recency 6w +0.539
chart entry recency 26w -0.383
log Track Top-40 hits +0.032
log Airplay Top-20 hits +0.005
Danish -0.007
group -0.053
female +0.021
career years (/50) -0.229
debut unknown -0.059
Christmas target week +0.000
Danish x P3 -0.001
Danish x P4 +0.020
Danish x P6 +0.007
ch p4kbh -0.013
ch p4sjaelland -0.009
ch p4fyn -0.015
ch p4syd -0.010
ch p4trekanten -0.012
ch p4vest -0.010
ch p4aarhus -0.010
ch p4nord -0.007
ch p4bornholm -0.013
ch p4esbjerg -0.011
ch p5 +0.118
ch p6beat +0.066
ch p8jazz +0.103