Backtest: new adds
An add = the artist enters rotation (plays ≥ τ) after at least 4 weeks out of rotation, or their plays jump well above their own 8-week baseline and stay up; per channel, and on any DR channel. Target: an add within the 8 weeks that start h weeks after the forecast origin. Origins from week 27; each fit uses only windows closed (a week before) its origin; only artists knowable at the origin. Pre-release rows blend the window's NMF placement from the artist's history; release timing in the window is a scenario input.
Window starting h = 0 weeks ahead (16 forecast origins)
Base rates: per channel 0.46% of artist-channels get an add in the window; on any channel 2.5% of artists.
| Model |
Log-loss, per channel |
PR-AUC, per channel |
Log-loss, any channel |
PR-AUC, any channel |
| rate by in-rotation-now |
0.0271 |
0.032 |
0.1111 |
0.069 |
| rate by last-12-weeks level |
0.0188 |
0.148 |
0.0956 |
0.134 |
| release timing only |
0.0274 |
0.010 |
0.1144 |
0.040 |
| model, calibrated on the last 6 origins (pre-release) |
0.0156 |
0.263 |
0.0837 |
0.254 |
| model, raw (pre-release) |
0.0155 |
0.263 |
0.0854 |
0.246 |
| model + pooled Platt (pre-release) |
0.0156 |
0.262 |
0.0855 |
0.246 |
| model + pooled isotonic (pre-release) |
0.0157 |
0.254 |
0.0859 |
0.244 |
| model, NMF known (post-release) |
0.0153 |
0.270 |
0.0836 |
0.263 |
Skill (1 − log-loss ÷ baseline log-loss; 90% interval from a bootstrap over blocks of 8 forecast weeks):
| Model |
Per channel vs last-12-weeks |
Per channel vs release timing |
Any channel vs last-12-weeks |
Any channel vs release timing |
| model, calibrated on the last 6 origins (pre-release) |
+0.174 (+0.167 to +0.176) |
+0.432 (+0.412 to +0.445) |
+0.124 (+0.123 to +0.133) |
+0.269 (+0.242 to +0.289) |
| model + pooled Platt (pre-release) |
+0.172 (+0.164 to +0.172) |
+0.431 (+0.409 to +0.444) |
+0.105 (+0.091 to +0.112) |
+0.253 (+0.213 to +0.272) |
| model + pooled isotonic (pre-release) |
+0.168 (+0.160 to +0.168) |
+0.428 (+0.405 to +0.444) |
+0.102 (+0.086 to +0.112) |
+0.250 (+0.209 to +0.272) |
Calibration, any channel:
| Predicted |
model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed |
model + pooled Platt (pre-release): rows / mean predicted / observed |
model + pooled isotonic (pre-release): rows / mean predicted / observed |
| 0.0–0.1 |
112229 / 0.010 / 0.013 |
111004 / 0.012 / 0.012 |
111041 / 0.011 / 0.012 |
| 0.1–0.2 |
2985 / 0.142 / 0.196 |
3319 / 0.142 / 0.155 |
2508 / 0.146 / 0.149 |
| 0.2–0.3 |
1360 / 0.245 / 0.263 |
1636 / 0.245 / 0.249 |
2058 / 0.249 / 0.218 |
| 0.3–0.4 |
799 / 0.346 / 0.328 |
998 / 0.347 / 0.305 |
1765 / 0.339 / 0.301 |
| 0.4–0.5 |
450 / 0.447 / 0.360 |
626 / 0.445 / 0.321 |
489 / 0.442 / 0.309 |
| 0.5–0.6 |
251 / 0.546 / 0.347 |
355 / 0.548 / 0.293 |
247 / 0.546 / 0.352 |
| 0.6–0.7 |
162 / 0.647 / 0.407 |
215 / 0.646 / 0.344 |
152 / 0.646 / 0.421 |
| 0.7–0.8 |
93 / 0.746 / 0.505 |
149 / 0.740 / 0.483 |
73 / 0.740 / 0.479 |
| 0.8–0.9 |
52 / 0.841 / 0.596 |
74 / 0.837 / 0.554 |
43 / 0.837 / 0.558 |
| 0.9–1.0 |
11 / 0.924 / 0.182 |
16 / 0.924 / 0.438 |
16 / 0.932 / 0.500 |
Calibration, per channel:
| Predicted |
model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed |
model + pooled Platt (pre-release): rows / mean predicted / observed |
model + pooled isotonic (pre-release): rows / mean predicted / observed |
| 0.0–0.1 |
1638964 / 0.002 / 0.002 |
1639990 / 0.002 / 0.002 |
1637740 / 0.002 / 0.002 |
| 0.1–0.2 |
8541 / 0.142 / 0.165 |
7975 / 0.141 / 0.176 |
9301 / 0.135 / 0.149 |
| 0.2–0.3 |
3740 / 0.248 / 0.229 |
3586 / 0.247 / 0.231 |
4805 / 0.251 / 0.229 |
| 0.3–0.4 |
2586 / 0.344 / 0.216 |
2464 / 0.345 / 0.211 |
3432 / 0.338 / 0.297 |
| 0.4–0.5 |
1570 / 0.449 / 0.336 |
1465 / 0.449 / 0.374 |
939 / 0.448 / 0.376 |
| 0.5–0.6 |
984 / 0.545 / 0.450 |
966 / 0.547 / 0.443 |
588 / 0.548 / 0.371 |
| 0.6–0.7 |
621 / 0.645 / 0.473 |
549 / 0.643 / 0.461 |
328 / 0.644 / 0.424 |
| 0.7–0.8 |
351 / 0.751 / 0.464 |
358 / 0.751 / 0.455 |
207 / 0.756 / 0.560 |
| 0.8–0.9 |
91 / 0.835 / 0.352 |
90 / 0.834 / 0.489 |
148 / 0.853 / 0.507 |
| 0.9–1.0 |
40 / 0.914 / 0.950 |
45 / 0.928 / 0.689 |
— |
Window starting h = 4 weeks ahead (12 forecast origins)
Base rates: per channel 0.47% of artist-channels get an add in the window; on any channel 2.5% of artists.
| Model |
Log-loss, per channel |
PR-AUC, per channel |
Log-loss, any channel |
PR-AUC, any channel |
| rate by in-rotation-now |
0.0277 |
0.033 |
0.1102 |
0.084 |
| rate by last-12-weeks level |
0.0204 |
0.117 |
0.0984 |
0.137 |
| release timing only |
0.0286 |
0.010 |
0.1146 |
0.043 |
| model, calibrated on the last 6 origins (pre-release) |
0.0186 |
0.166 |
0.0898 |
0.208 |
| model, raw (pre-release) |
0.0186 |
0.166 |
0.0908 |
0.200 |
| model + pooled Platt (pre-release) |
0.0186 |
0.166 |
0.0908 |
0.200 |
| model + pooled isotonic (pre-release) |
0.0186 |
0.166 |
0.0908 |
0.200 |
| model, NMF known (post-release) |
0.0183 |
0.174 |
0.0896 |
0.219 |
Skill (1 − log-loss ÷ baseline log-loss; 90% interval from a bootstrap over blocks of 8 forecast weeks):
| Model |
Per channel vs last-12-weeks |
Per channel vs release timing |
Any channel vs last-12-weeks |
Any channel vs release timing |
| model, calibrated on the last 6 origins (pre-release) |
+0.086 (+0.065 to +0.080) |
+0.349 (+0.334 to +0.345) |
+0.088 (+0.084 to +0.095) |
+0.217 (+0.207 to +0.221) |
| model + pooled Platt (pre-release) |
+0.086 (+0.065 to +0.080) |
+0.349 (+0.334 to +0.345) |
+0.077 (+0.072 to +0.080) |
+0.208 (+0.195 to +0.208) |
| model + pooled isotonic (pre-release) |
+0.086 (+0.065 to +0.080) |
+0.349 (+0.334 to +0.345) |
+0.077 (+0.072 to +0.080) |
+0.208 (+0.195 to +0.208) |
Calibration, any channel:
| Predicted |
model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed |
model + pooled Platt (pre-release): rows / mean predicted / observed |
model + pooled isotonic (pre-release): rows / mean predicted / observed |
| 0.0–0.1 |
84532 / 0.011 / 0.015 |
83207 / 0.013 / 0.013 |
83207 / 0.013 / 0.013 |
| 0.1–0.2 |
2353 / 0.140 / 0.178 |
2764 / 0.142 / 0.156 |
2764 / 0.142 / 0.156 |
| 0.2–0.3 |
904 / 0.242 / 0.304 |
1271 / 0.245 / 0.238 |
1271 / 0.245 / 0.238 |
| 0.3–0.4 |
479 / 0.347 / 0.311 |
717 / 0.346 / 0.262 |
717 / 0.346 / 0.262 |
| 0.4–0.5 |
221 / 0.446 / 0.385 |
359 / 0.445 / 0.304 |
359 / 0.445 / 0.304 |
| 0.5–0.6 |
124 / 0.544 / 0.306 |
188 / 0.547 / 0.372 |
188 / 0.547 / 0.372 |
| 0.6–0.7 |
83 / 0.640 / 0.325 |
142 / 0.641 / 0.345 |
142 / 0.641 / 0.345 |
| 0.7–0.8 |
30 / 0.750 / 0.333 |
66 / 0.749 / 0.273 |
66 / 0.749 / 0.273 |
| 0.8–0.9 |
4 / 0.830 / 0.000 |
16 / 0.835 / 0.312 |
16 / 0.835 / 0.312 |
Calibration, per channel:
| Predicted |
model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed |
model + pooled Platt (pre-release): rows / mean predicted / observed |
model + pooled isotonic (pre-release): rows / mean predicted / observed |
| 0.0–0.1 |
1228298 / 0.002 / 0.003 |
1228298 / 0.002 / 0.003 |
1228298 / 0.002 / 0.003 |
| 0.1–0.2 |
6683 / 0.141 / 0.129 |
6683 / 0.141 / 0.129 |
6683 / 0.141 / 0.129 |
| 0.2–0.3 |
3587 / 0.247 / 0.188 |
3587 / 0.247 / 0.188 |
3587 / 0.247 / 0.188 |
| 0.3–0.4 |
1731 / 0.345 / 0.207 |
1731 / 0.345 / 0.207 |
1731 / 0.345 / 0.207 |
| 0.4–0.5 |
956 / 0.443 / 0.285 |
956 / 0.443 / 0.285 |
956 / 0.443 / 0.285 |
| 0.5–0.6 |
571 / 0.548 / 0.256 |
571 / 0.548 / 0.256 |
571 / 0.548 / 0.256 |
| 0.6–0.7 |
333 / 0.647 / 0.441 |
333 / 0.647 / 0.441 |
333 / 0.647 / 0.441 |
| 0.7–0.8 |
55 / 0.734 / 0.255 |
55 / 0.734 / 0.255 |
55 / 0.734 / 0.255 |
| 0.8–0.9 |
5 / 0.820 / 0.400 |
5 / 0.820 / 0.400 |
5 / 0.820 / 0.400 |
| 0.9–1.0 |
1 / 0.905 / 1.000 |
1 / 0.905 / 1.000 |
1 / 0.905 / 1.000 |
Coefficients, window starting now (h = 0), all closed windows
| Any-channel feature |
Coefficient |
| intercept |
-4.141 |
| log DR plays t |
+0.882 |
| log DR plays 4w |
+0.253 |
| log DR plays 12w |
-0.255 |
| log max DR plays 12w |
+0.583 |
| in rotation anywhere now |
-1.155 |
| station groups in rotation now (/5) |
+0.320 |
| current run anywhere (/12) |
-1.215 |
| weeks since in rotation anywhere (/12) |
-0.748 |
| Track Top-40 score |
+0.427 |
| Airplay Top-20 score |
-1.336 |
| chart entry recency 6w |
+1.203 |
| chart entry recency 26w |
-0.646 |
| log Track Top-40 hits |
-0.201 |
| log Airplay Top-20 hits |
-0.070 |
| Danish |
+0.470 |
| group |
-0.253 |
| female |
-0.097 |
| career years (/50) |
+0.222 |
| debut unknown |
+0.489 |
| Christmas in window |
+0.000 |
| discography known |
-0.392 |
| release recency 4w |
+0.435 |
| release recency 12w |
+0.680 |
| log releases last 52w |
-0.186 |
| releases in the window |
-0.216 |
| releases in window x not in rotation now |
+0.836 |
| DR: weeks since last add 13-25 |
+0.095 |
| DR: last add 26+ weeks ago |
-0.642 |
| DR: no add in the panel yet |
-0.845 |
| Airplay Top-20: last new song 13-25 weeks ago |
+0.278 |
| Airplay Top-20: 0.5-1.25x own usual gap |
+0.229 |
| Airplay Top-20: last new song 2+ years ago |
-0.527 |
| log archive plays per 100 sampled hours (2012-21) |
+0.289 |
| log DR lists with the artist before t (2023-) |
+0.209 |
| DR list recency 26w |
-0.924 |
| DR lists known before t at all |
+0.039 |
| NMF Denmark in the window |
+0.940 |
| NMF Denmark top 10 in the window |
+0.408 |
| log NMF Denmark placements last 52w |
-0.068 |
| share of releases on NMF Denmark (2 years) |
-0.417 |
| season index of the window (log, IFPI 2002-24) |
+0.412 |
| Per-channel feature |
Coefficient |
| intercept |
-7.117 |
| log plays t |
+1.106 |
| log plays t-1 |
+0.141 |
| log mean plays 4w |
+0.113 |
| log mean plays 12w |
-0.871 |
| log max plays 12w |
+1.022 |
| log other channels t |
+0.328 |
| log other channels 4w |
-0.095 |
| log P3 t |
+0.087 |
| weeks since played (/12) |
-0.509 |
| Track Top-40 score |
-0.132 |
| Airplay Top-20 score |
-1.376 |
| chart entry recency 6w |
+0.616 |
| chart entry recency 26w |
-0.506 |
| log Track Top-40 hits |
+0.258 |
| log Airplay Top-20 hits |
-0.398 |
| Danish |
+0.171 |
| group |
-0.636 |
| female |
+0.136 |
| career years (/50) |
-1.498 |
| debut unknown |
-0.333 |
| Christmas target week |
+0.000 |
| Danish x P3 |
+0.064 |
| Danish x P4 |
+0.058 |
| Danish x P6 |
+0.111 |
| ch p4kbh |
+1.094 |
| ch p4sjaelland |
+0.966 |
| ch p4fyn |
+0.946 |
| ch p4syd |
+0.975 |
| ch p4trekanten |
+0.985 |
| ch p4vest |
+1.083 |
| ch p4aarhus |
+0.944 |
| ch p4nord |
+1.006 |
| ch p4bornholm |
+0.995 |
| ch p4esbjerg |
+0.972 |
| ch p5 |
+2.081 |
| ch p6beat |
+1.296 |
| ch p8jazz |
+2.312 |
| in rotation on channel now |
+0.350 |
| in rotation anywhere now |
-0.334 |
| station groups in rotation now (/5) |
+0.041 |
| run on channel = 1 week |
+0.660 |
| run = 2 weeks |
+0.439 |
| run = 3-5 weeks |
-0.056 |
| run >= 6 weeks |
-0.693 |
| near threshold (>= half of tau, not in rotation) |
+1.381 |
| plays / tau (capped 3, /3) |
-3.363 |
| weeks since in rotation on channel (/12) |
-0.302 |
| discography known |
-0.052 |
| release recency 4w |
+1.017 |
| release recency 12w |
+0.014 |
| log releases last 52w |
+0.009 |
| releases in the window |
-0.136 |
| releases in window x not in rotation now |
+0.755 |
| DR: weeks since last add 13-25 |
+0.106 |
| DR: last add 26+ weeks ago |
-0.081 |
| DR: no add in the panel yet |
-0.483 |
| Airplay Top-20: last new song 13-25 weeks ago |
+0.194 |
| Airplay Top-20: 0.5-1.25x own usual gap |
-0.013 |
| Airplay Top-20: last new song 2+ years ago |
-0.418 |
| log archive plays per 100 sampled hours (2012-21) |
+0.347 |
| log DR lists with the artist before t (2023-) |
+0.023 |
| DR list recency 26w |
-0.442 |
| DR lists known before t at all |
+0.066 |
| NMF Denmark in the window |
+0.779 |
| NMF Denmark top 10 in the window |
+0.580 |
| log NMF Denmark placements last 52w |
-0.089 |
| share of releases on NMF Denmark (2 years) |
+0.212 |
| season index of the window (log, IFPI 2002-24) |
+1.211 |
| share of the artist's DR plays on this station (before t) |
+0.986 |
| log DR plays on this station (before t) |
+0.050 |
| never played on this station (before t) |
-2.825 |
| genre fit for this station |
+0.091 |