Backtest: if the artist releases a new song, will DR add it?
Events: an artist knowable at the time releases something (Deezer) in week t0+1, with no other release in the 6 weeks before (so an add can't be the previous song's); target: a DR add (entering rotation after 4+ weeks out, or plays jumping well above the artist's own 8-week baseline and staying up) within 2 week(s) from the release week. The model knows the artist's state the week before; pre-release rows blend the new song's NMF Denmark placement from the artist's history. "Release timing only" = the add rate by weeks since the artist's previous release. 3588 release events scored, 64 (1.8%) got an add on some DR channel.
| Model |
Log-loss, per channel |
PR-AUC, per channel |
Log-loss, any channel |
PR-AUC, any channel |
| rate by in-rotation-now |
0.0223 |
0.013 |
0.0873 |
0.048 |
| rate by last-12-weeks level |
0.0158 |
0.076 |
0.0697 |
0.126 |
| release timing only |
0.0242 |
0.003 |
0.0904 |
0.019 |
| model, calibrated on the last 6 origins (pre-release) |
0.0163 |
0.089 |
0.0682 |
0.174 |
| model, raw (pre-release) |
0.0163 |
0.095 |
0.0649 |
0.235 |
| model + pooled Platt (pre-release) |
0.0158 |
0.090 |
0.0655 |
0.224 |
| model + pooled isotonic (pre-release) |
0.0160 |
0.083 |
0.0654 |
0.228 |
| model, NMF known (post-release) |
0.0164 |
0.087 |
0.0685 |
0.160 |
Skill (1 − log-loss ÷ baseline log-loss; 90% interval from a bootstrap over blocks of 8 forecast weeks):
| Model |
Per channel vs last-12-weeks |
Per channel vs release timing |
Any channel vs last-12-weeks |
Any channel vs release timing |
| model, calibrated on the last 6 origins (pre-release) |
-0.031 (-0.101 to +0.021) |
+0.326 (+0.274 to +0.383) |
+0.022 (+0.001 to +0.080) |
+0.246 (+0.232 to +0.295) |
| model + pooled Platt (pre-release) |
+0.002 (-0.041 to +0.034) |
+0.348 (+0.291 to +0.406) |
+0.060 (+0.031 to +0.108) |
+0.275 (+0.232 to +0.327) |
| model + pooled isotonic (pre-release) |
-0.010 (-0.040 to +0.024) |
+0.340 (+0.282 to +0.408) |
+0.062 (+0.042 to +0.098) |
+0.277 (+0.231 to +0.327) |
Calibration, any channel:
| Predicted |
model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed |
model + pooled Platt (pre-release): rows / mean predicted / observed |
model + pooled isotonic (pre-release): rows / mean predicted / observed |
| 0.0–0.1 |
3485 / 0.014 / 0.011 |
3432 / 0.010 / 0.010 |
3416 / 0.009 / 0.009 |
| 0.1–0.2 |
72 / 0.138 / 0.222 |
104 / 0.142 / 0.135 |
129 / 0.142 / 0.155 |
| 0.2–0.3 |
19 / 0.245 / 0.316 |
24 / 0.240 / 0.250 |
19 / 0.238 / 0.211 |
| 0.3–0.4 |
7 / 0.334 / 0.143 |
12 / 0.329 / 0.167 |
7 / 0.334 / 0.143 |
| 0.4–0.5 |
4 / 0.460 / 0.250 |
10 / 0.447 / 0.500 |
6 / 0.425 / 0.500 |
| 0.5–0.6 |
1 / 0.579 / 0.000 |
1 / 0.575 / 1.000 |
1 / 0.535 / 1.000 |
| 0.6–0.7 |
— |
4 / 0.625 / 0.500 |
4 / 0.630 / 0.750 |
| 0.7–0.8 |
— |
1 / 0.799 / 0.000 |
3 / 0.749 / 0.333 |
| 0.8–0.9 |
— |
— |
3 / 0.831 / 0.333 |
Calibration, per channel:
| Predicted |
model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed |
model + pooled Platt (pre-release): rows / mean predicted / observed |
model + pooled isotonic (pre-release): rows / mean predicted / observed |
| 0.0–0.1 |
49992 / 0.003 / 0.003 |
49847 / 0.002 / 0.002 |
49630 / 0.002 / 0.002 |
| 0.1–0.2 |
143 / 0.132 / 0.161 |
199 / 0.136 / 0.136 |
482 / 0.142 / 0.110 |
| 0.2–0.3 |
55 / 0.263 / 0.036 |
108 / 0.250 / 0.157 |
70 / 0.240 / 0.114 |
| 0.3–0.4 |
23 / 0.335 / 0.000 |
43 / 0.339 / 0.163 |
34 / 0.356 / 0.176 |
| 0.4–0.5 |
8 / 0.453 / 0.250 |
25 / 0.457 / 0.040 |
15 / 0.441 / 0.067 |
| 0.5–0.6 |
8 / 0.531 / 0.000 |
8 / 0.522 / 0.125 |
— |
| 0.6–0.7 |
2 / 0.677 / 0.000 |
2 / 0.626 / 0.000 |
1 / 0.647 / 0.000 |
| 0.7–0.8 |
1 / 0.723 / 1.000 |
— |
— |
Quiet artists (out of rotation everywhere for 26 weeks) releasing, any channel:
| Group |
Events |
Adds |
Rate |
Mean predicted (raw) |
Mean predicted (isotonic) |
Log-loss (isotonic) |
PR-AUC (raw) |
| former favourites (4+ weeks on DR 2019-24) |
348 |
7 |
0.020 |
0.022 |
0.027 |
0.0930 |
0.104 |
| other quiet artists |
2892 |
16 |
0.006 |
0.006 |
0.008 |
0.0299 |
0.126 |
| Any-channel feature (all events) |
Coefficient |
| intercept |
-5.798 |
| log DR plays t |
+0.782 |
| log DR plays 4w |
+0.714 |
| log DR plays 12w |
-0.537 |
| log max DR plays 12w |
+0.402 |
| in rotation anywhere now |
-0.644 |
| station groups in rotation now (/5) |
-0.157 |
| current run anywhere (/12) |
-1.205 |
| weeks since in rotation anywhere (/12) |
-0.271 |
| Track Top-40 score |
+0.756 |
| Airplay Top-20 score |
-0.521 |
| chart entry recency 6w |
+0.323 |
| chart entry recency 26w |
-0.592 |
| log Track Top-40 hits |
-0.212 |
| log Airplay Top-20 hits |
-0.186 |
| Danish |
+0.819 |
| group |
-0.151 |
| female |
-0.024 |
| career years (/50) |
+0.476 |
| debut unknown |
-0.153 |
| Christmas in window |
+0.000 |
| discography known |
-0.000 |
| release recency 4w |
-0.270 |
| release recency 12w |
-1.329 |
| log releases last 52w |
-0.233 |
| releases in the window |
-0.000 |
| releases in window x not in rotation now |
+0.644 |
| DR: weeks since last add 13-25 |
-0.315 |
| DR: last add 26+ weeks ago |
-0.155 |
| DR: no add in the panel yet |
-0.611 |
| Airplay Top-20: last new song 13-25 weeks ago |
+0.771 |
| Airplay Top-20: 0.5-1.25x own usual gap |
-0.238 |
| Airplay Top-20: last new song 2+ years ago |
+0.257 |
| log archive plays per 100 sampled hours (2012-21) |
+0.550 |
| log DR lists with the artist before t (2023-) |
+0.194 |
| DR list recency 26w |
-0.668 |
| DR lists known before t at all |
-0.266 |
| quiet former favourite (4+ weeks on DR 2019-24, out of rotation 26w) |
-0.645 |
| NMF Denmark in the window |
+0.820 |
| NMF Denmark top 10 in the window |
+0.107 |
| log NMF Denmark placements last 52w |
+0.131 |
| share of releases on NMF Denmark (2 years) |
+0.204 |
| season index of the window (log, IFPI 2002-24) |
-0.066 |