Backtest: if the artist releases a new song, will DR add it?
Events: an artist knowable at the time releases something (Deezer) in week t0+1, with no other release in the 6 weeks before (so an add can't be the previous song's); target: a DR add (entering rotation after 4+ weeks out, or plays jumping well above the artist's own 8-week baseline and staying up) within the 8 weeks from the release week. The model knows the artist's state the week before; pre-release rows blend the new song's NMF Denmark placement from the artist's history. "Release timing only" = the add rate by weeks since the artist's previous release. 2140 release events scored, 120 (5.6%) got an add on some DR channel.
| Model |
Log-loss, per channel |
PR-AUC, per channel |
Log-loss, any channel |
PR-AUC, any channel |
| rate by in-rotation-now |
0.0666 |
0.061 |
0.2070 |
0.113 |
| rate by last-12-weeks level |
0.0478 |
0.216 |
0.1805 |
0.221 |
| release timing only |
0.0719 |
0.014 |
0.2166 |
0.068 |
| model, calibrated on the last 6 origins (pre-release) |
0.0431 |
0.341 |
0.1881 |
0.243 |
| model, raw (pre-release) |
0.0437 |
0.326 |
0.1860 |
0.252 |
| model + pooled Platt (pre-release) |
0.0425 |
0.337 |
0.1916 |
0.221 |
| model + pooled isotonic (pre-release) |
0.0439 |
0.308 |
0.1871 |
0.243 |
| model, NMF known (post-release) |
0.0413 |
0.346 |
0.1853 |
0.266 |
Skill (1 − log-loss ÷ baseline log-loss; 90% interval from a bootstrap over blocks of 8 forecast weeks):
| Model |
Per channel vs last-12-weeks |
Per channel vs release timing |
Any channel vs last-12-weeks |
Any channel vs release timing |
| model, calibrated on the last 6 origins (pre-release) |
+0.098 (+0.054 to +0.162) |
+0.400 (+0.313 to +0.440) |
-0.042 (-0.068 to +0.041) |
+0.132 (+0.128 to +0.223) |
| model + pooled Platt (pre-release) |
+0.111 (+0.054 to +0.181) |
+0.408 (+0.330 to +0.440) |
-0.061 (-0.091 to -0.022) |
+0.116 (+0.097 to +0.160) |
| model + pooled isotonic (pre-release) |
+0.083 (+0.054 to +0.187) |
+0.390 (+0.333 to +0.438) |
-0.037 (-0.072 to +0.031) |
+0.136 (+0.121 to +0.206) |
Calibration, any channel:
| Predicted |
model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed |
model + pooled Platt (pre-release): rows / mean predicted / observed |
model + pooled isotonic (pre-release): rows / mean predicted / observed |
| 0.0–0.1 |
1936 / 0.018 / 0.034 |
1897 / 0.034 / 0.033 |
1844 / 0.027 / 0.027 |
| 0.1–0.2 |
124 / 0.137 / 0.234 |
154 / 0.141 / 0.201 |
124 / 0.143 / 0.202 |
| 0.2–0.3 |
40 / 0.245 / 0.325 |
56 / 0.247 / 0.286 |
90 / 0.251 / 0.200 |
| 0.3–0.4 |
18 / 0.349 / 0.278 |
20 / 0.355 / 0.300 |
40 / 0.350 / 0.250 |
| 0.4–0.5 |
13 / 0.442 / 0.385 |
7 / 0.463 / 0.429 |
23 / 0.449 / 0.435 |
| 0.5–0.6 |
5 / 0.534 / 0.400 |
3 / 0.536 / 0.333 |
8 / 0.571 / 0.500 |
| 0.6–0.7 |
1 / 0.606 / 0.000 |
2 / 0.625 / 0.000 |
3 / 0.637 / 0.333 |
| 0.7–0.8 |
3 / 0.732 / 0.333 |
— |
2 / 0.730 / 0.000 |
| 0.8–0.9 |
— |
1 / 0.823 / 1.000 |
5 / 0.824 / 0.600 |
| 0.9–1.0 |
— |
— |
1 / 0.911 / 0.000 |
Calibration, per channel:
| Predicted |
model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed |
model + pooled Platt (pre-release): rows / mean predicted / observed |
model + pooled isotonic (pre-release): rows / mean predicted / observed |
| 0.0–0.1 |
29258 / 0.004 / 0.007 |
29241 / 0.004 / 0.007 |
29307 / 0.004 / 0.007 |
| 0.1–0.2 |
342 / 0.140 / 0.208 |
347 / 0.143 / 0.182 |
328 / 0.143 / 0.235 |
| 0.2–0.3 |
136 / 0.246 / 0.206 |
136 / 0.252 / 0.287 |
122 / 0.248 / 0.189 |
| 0.3–0.4 |
87 / 0.347 / 0.207 |
102 / 0.335 / 0.118 |
81 / 0.344 / 0.185 |
| 0.4–0.5 |
66 / 0.447 / 0.500 |
44 / 0.450 / 0.500 |
35 / 0.449 / 0.429 |
| 0.5–0.6 |
10 / 0.527 / 0.000 |
28 / 0.538 / 0.714 |
25 / 0.551 / 0.760 |
| 0.6–0.7 |
8 / 0.675 / 0.125 |
5 / 0.655 / 0.400 |
4 / 0.652 / 0.250 |
| 0.7–0.8 |
42 / 0.745 / 0.762 |
42 / 0.757 / 0.714 |
35 / 0.751 / 0.886 |
| 0.8–0.9 |
2 / 0.845 / 1.000 |
6 / 0.817 / 0.500 |
2 / 0.845 / 1.000 |
| 0.9–1.0 |
9 / 0.912 / 1.000 |
9 / 0.912 / 1.000 |
21 / 0.954 / 0.476 |
| Any-channel feature (all events) |
Coefficient |
| intercept |
-3.768 |
| log DR plays t |
+0.560 |
| log DR plays 4w |
+0.510 |
| log DR plays 12w |
-0.409 |
| log max DR plays 12w |
+0.318 |
| in rotation anywhere now |
-0.474 |
| station groups in rotation now (/5) |
-0.249 |
| current run anywhere (/12) |
-1.173 |
| weeks since in rotation anywhere (/12) |
-0.489 |
| Track Top-40 score |
-0.265 |
| Airplay Top-20 score |
-0.317 |
| chart entry recency 6w |
+0.685 |
| chart entry recency 26w |
+0.501 |
| log Track Top-40 hits |
-0.291 |
| log Airplay Top-20 hits |
-0.120 |
| Danish |
+0.548 |
| group |
+0.013 |
| female |
-0.075 |
| career years (/50) |
+0.422 |
| debut unknown |
+0.435 |
| Christmas in window |
+0.000 |
| discography known |
-0.000 |
| release recency 4w |
+0.216 |
| release recency 12w |
-0.783 |
| log releases last 52w |
-0.189 |
| releases in the window |
-0.000 |
| releases in window x not in rotation now |
+0.474 |
| DR: weeks since last add 13-25 |
-0.050 |
| DR: last add 26+ weeks ago |
+0.502 |
| DR: no add in the panel yet |
-0.677 |
| Airplay Top-20: last new song 13-25 weeks ago |
+0.465 |
| Airplay Top-20: 0.5-1.25x own usual gap |
-0.144 |
| Airplay Top-20: last new song 2+ years ago |
-0.113 |
| log archive plays per 100 sampled hours (2012-21) |
+0.515 |
| log DR lists with the artist before t (2023-) |
+0.105 |
| DR list recency 26w |
-0.984 |
| DR lists known before t at all |
+0.593 |
| NMF Denmark in the window |
+0.836 |
| NMF Denmark top 10 in the window |
+0.682 |
| log NMF Denmark placements last 52w |
-0.067 |
| share of releases on NMF Denmark (2 years) |
-0.273 |
| season index of the window (log, IFPI 2002-24) |
+0.334 |