Backtest: if the artist releases a new song, will DR add it?
Events: an artist knowable at the time releases something (Deezer) in week t0+1, with no other release in the 6 weeks before (so an add can't be the previous song's); target: a DR add (entering rotation after 4+ weeks out, or plays jumping well above the artist's own 8-week baseline and staying up) within 2 week(s) from the release week. The model knows the artist's state the week before; pre-release rows blend the new song's NMF Denmark placement from the artist's history. "Release timing only" = the add rate by weeks since the artist's previous release. 3269 release events scored, 63 (1.9%) got an add on some DR channel.
| Model |
Log-loss, per channel |
PR-AUC, per channel |
Log-loss, any channel |
PR-AUC, any channel |
| rate by in-rotation-now |
0.0240 |
0.014 |
0.0923 |
0.062 |
| rate by last-12-weeks level |
0.0170 |
0.082 |
0.0721 |
0.140 |
| release timing only |
0.0262 |
0.003 |
0.0964 |
0.020 |
| model, calibrated on the last 6 origins (pre-release) |
0.0179 |
0.085 |
0.0723 |
0.179 |
| model, raw (pre-release) |
0.0178 |
0.097 |
0.0676 |
0.245 |
| model + pooled Platt (pre-release) |
0.0172 |
0.098 |
0.0683 |
0.232 |
| model + pooled isotonic (pre-release) |
0.0175 |
0.088 |
0.0689 |
0.252 |
| model, NMF known (post-release) |
0.0180 |
0.090 |
0.0723 |
0.170 |
Skill (1 − log-loss ÷ baseline log-loss; 90% interval from a bootstrap over blocks of 8 forecast weeks):
| Model |
Per channel vs last-12-weeks |
Per channel vs release timing |
Any channel vs last-12-weeks |
Any channel vs release timing |
| model, calibrated on the last 6 origins (pre-release) |
-0.054 (-0.109 to -0.010) |
+0.315 (+0.264 to +0.373) |
-0.002 (-0.006 to +0.060) |
+0.250 (+0.237 to +0.299) |
| model + pooled Platt (pre-release) |
-0.010 (-0.041 to +0.026) |
+0.343 (+0.296 to +0.404) |
+0.053 (+0.000 to +0.115) |
+0.291 (+0.243 to +0.343) |
| model + pooled isotonic (pre-release) |
-0.028 (-0.082 to +0.022) |
+0.331 (+0.248 to +0.418) |
+0.045 (-0.000 to +0.098) |
+0.286 (+0.228 to +0.338) |
Calibration, any channel:
| Predicted |
model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed |
model + pooled Platt (pre-release): rows / mean predicted / observed |
model + pooled isotonic (pre-release): rows / mean predicted / observed |
| 0.0–0.1 |
3159 / 0.014 / 0.012 |
3108 / 0.011 / 0.009 |
3100 / 0.009 / 0.009 |
| 0.1–0.2 |
80 / 0.140 / 0.212 |
103 / 0.140 / 0.165 |
105 / 0.137 / 0.152 |
| 0.2–0.3 |
15 / 0.259 / 0.200 |
23 / 0.227 / 0.217 |
33 / 0.227 / 0.273 |
| 0.3–0.4 |
10 / 0.337 / 0.400 |
17 / 0.345 / 0.353 |
9 / 0.353 / 0.222 |
| 0.4–0.5 |
2 / 0.440 / 0.000 |
8 / 0.438 / 0.250 |
7 / 0.445 / 0.429 |
| 0.5–0.6 |
1 / 0.517 / 0.000 |
5 / 0.534 / 0.200 |
4 / 0.561 / 0.000 |
| 0.6–0.7 |
2 / 0.658 / 0.000 |
4 / 0.631 / 0.750 |
5 / 0.644 / 0.200 |
| 0.7–0.8 |
— |
— |
1 / 0.709 / 1.000 |
| 0.8–0.9 |
— |
1 / 0.813 / 0.000 |
5 / 0.839 / 0.600 |
Calibration, per channel:
| Predicted |
model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed |
model + pooled Platt (pre-release): rows / mean predicted / observed |
model + pooled isotonic (pre-release): rows / mean predicted / observed |
| 0.0–0.1 |
45525 / 0.003 / 0.003 |
45392 / 0.002 / 0.003 |
45368 / 0.003 / 0.003 |
| 0.1–0.2 |
126 / 0.135 / 0.135 |
182 / 0.133 / 0.082 |
301 / 0.124 / 0.083 |
| 0.2–0.3 |
39 / 0.247 / 0.256 |
95 / 0.246 / 0.168 |
36 / 0.242 / 0.056 |
| 0.3–0.4 |
54 / 0.340 / 0.037 |
54 / 0.333 / 0.093 |
32 / 0.341 / 0.094 |
| 0.4–0.5 |
8 / 0.467 / 0.125 |
25 / 0.461 / 0.160 |
10 / 0.447 / 0.400 |
| 0.5–0.6 |
9 / 0.533 / 0.000 |
12 / 0.521 / 0.250 |
8 / 0.554 / 0.375 |
| 0.6–0.7 |
1 / 0.657 / 0.000 |
4 / 0.660 / 0.000 |
7 / 0.650 / 0.000 |
| 0.7–0.8 |
4 / 0.754 / 0.250 |
2 / 0.730 / 0.500 |
4 / 0.734 / 0.250 |
Quiet artists (out of rotation everywhere for 26 weeks) releasing, any channel:
| Group |
Events |
Adds |
Rate |
Mean predicted (raw) |
Mean predicted (isotonic) |
Log-loss (isotonic) |
PR-AUC (raw) |
| former favourites (4+ weeks on DR 2019-24) |
326 |
7 |
0.021 |
0.025 |
0.030 |
0.1028 |
0.095 |
| other quiet artists |
2616 |
15 |
0.006 |
0.007 |
0.008 |
0.0313 |
0.126 |
| Any-channel feature (all events) |
Coefficient |
| intercept |
-5.988 |
| log DR plays t |
+0.677 |
| log DR plays 4w |
+0.897 |
| log DR plays 12w |
-0.425 |
| log max DR plays 12w |
+0.233 |
| in rotation anywhere now |
-0.575 |
| station groups in rotation now (/5) |
-0.137 |
| current run anywhere (/12) |
-1.280 |
| weeks since in rotation anywhere (/12) |
-0.016 |
| Track Top-40 score |
+0.603 |
| Airplay Top-20 score |
-0.700 |
| chart entry recency 6w |
+0.172 |
| chart entry recency 26w |
-0.684 |
| log Track Top-40 hits |
-0.283 |
| log Airplay Top-20 hits |
-0.147 |
| Danish |
+0.830 |
| group |
-0.078 |
| female |
-0.173 |
| career years (/50) |
+0.646 |
| debut unknown |
+0.019 |
| Christmas in window |
+0.000 |
| discography known |
-0.000 |
| release recency 4w |
-0.157 |
| release recency 12w |
-1.118 |
| log releases last 52w |
-0.285 |
| releases in the window |
-0.000 |
| releases in window x not in rotation now |
+0.575 |
| DR: weeks since last add 13-25 |
-0.493 |
| DR: last add 26+ weeks ago |
-0.099 |
| DR: no add in the panel yet |
-0.708 |
| Airplay Top-20: last new song 13-25 weeks ago |
+0.826 |
| Airplay Top-20: 0.5-1.25x own usual gap |
-0.176 |
| Airplay Top-20: last new song 2+ years ago |
+0.334 |
| log archive plays per 100 sampled hours (2012-21) |
+0.618 |
| log DR lists with the artist before t (2023-) |
+0.156 |
| DR list recency 26w |
-0.607 |
| DR lists known before t at all |
-0.210 |
| quiet former favourite (4+ weeks on DR 2019-24, out of rotation 26w) |
-0.665 |
| NMF Denmark in the window |
+0.943 |
| NMF Denmark top 10 in the window |
+0.202 |
| log NMF Denmark placements last 52w |
+0.251 |
| share of releases on NMF Denmark (2 years) |
+0.020 |
| season index of the window (log, IFPI 2002-24) |
-0.049 |