Backtest: if the artist releases a new song, will DR add it?
Events: an artist knowable at the time releases something (Deezer) in week t0+1, with no other release in the 6 weeks before (so an add can't be the previous song's); target: a DR add (entering rotation after 4+ weeks out, or plays jumping well above the artist's own 8-week baseline and staying up) within 2 week(s) from the release week. The model knows the artist's state the week before; pre-release rows blend the new song's NMF Denmark placement from the artist's history. "Release timing only" = the add rate by weeks since the artist's previous release. 3193 release events scored, 63 (2.0%) got an add on some DR channel.
| Model |
Log-loss, per channel |
PR-AUC, per channel |
Log-loss, any channel |
PR-AUC, any channel |
| rate by in-rotation-now |
0.0245 |
0.015 |
0.0941 |
0.062 |
| rate by last-12-weeks level |
0.0174 |
0.082 |
0.0738 |
0.140 |
| release timing only |
0.0267 |
0.003 |
0.0982 |
0.021 |
| model, calibrated on the last 6 origins (pre-release) |
0.0183 |
0.080 |
0.0748 |
0.170 |
| model, raw (pre-release) |
0.0182 |
0.091 |
0.0696 |
0.233 |
| model + pooled Platt (pre-release) |
0.0177 |
0.094 |
0.0704 |
0.224 |
| model + pooled isotonic (pre-release) |
0.0176 |
0.091 |
0.0706 |
0.235 |
| model, NMF known (post-release) |
0.0184 |
0.085 |
0.0750 |
0.163 |
Skill (1 − log-loss ÷ baseline log-loss; 90% interval from a bootstrap over blocks of 8 forecast weeks):
| Model |
Per channel vs last-12-weeks |
Per channel vs release timing |
Any channel vs last-12-weeks |
Any channel vs release timing |
| model, calibrated on the last 6 origins (pre-release) |
-0.053 (-0.131 to +0.002) |
+0.313 (+0.257 to +0.378) |
-0.013 (-0.017 to +0.048) |
+0.239 (+0.226 to +0.288) |
| model + pooled Platt (pre-release) |
-0.015 (-0.053 to +0.020) |
+0.338 (+0.286 to +0.399) |
+0.046 (-0.008 to +0.107) |
+0.283 (+0.234 to +0.334) |
| model + pooled isotonic (pre-release) |
-0.010 (-0.095 to +0.039) |
+0.342 (+0.234 to +0.430) |
+0.044 (-0.011 to +0.100) |
+0.281 (+0.223 to +0.335) |
Calibration, any channel:
| Predicted |
model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed |
model + pooled Platt (pre-release): rows / mean predicted / observed |
model + pooled isotonic (pre-release): rows / mean predicted / observed |
| 0.0–0.1 |
3081 / 0.015 / 0.013 |
3033 / 0.011 / 0.010 |
2979 / 0.009 / 0.008 |
| 0.1–0.2 |
82 / 0.136 / 0.207 |
107 / 0.140 / 0.168 |
169 / 0.135 / 0.160 |
| 0.2–0.3 |
16 / 0.245 / 0.312 |
20 / 0.239 / 0.300 |
16 / 0.238 / 0.188 |
| 0.3–0.4 |
6 / 0.316 / 0.333 |
12 / 0.340 / 0.250 |
8 / 0.344 / 0.375 |
| 0.4–0.5 |
4 / 0.451 / 0.000 |
13 / 0.437 / 0.231 |
8 / 0.436 / 0.250 |
| 0.5–0.6 |
2 / 0.533 / 0.000 |
6 / 0.566 / 0.667 |
4 / 0.563 / 0.250 |
| 0.6–0.7 |
2 / 0.636 / 0.000 |
1 / 0.605 / 0.000 |
2 / 0.620 / 0.000 |
| 0.7–0.8 |
— |
— |
5 / 0.775 / 0.600 |
| 0.8–0.9 |
— |
1 / 0.818 / 0.000 |
2 / 0.827 / 0.500 |
Calibration, per channel:
| Predicted |
model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed |
model + pooled Platt (pre-release): rows / mean predicted / observed |
model + pooled isotonic (pre-release): rows / mean predicted / observed |
| 0.0–0.1 |
44463 / 0.004 / 0.003 |
44305 / 0.002 / 0.003 |
44334 / 0.003 / 0.003 |
| 0.1–0.2 |
125 / 0.136 / 0.128 |
229 / 0.137 / 0.105 |
278 / 0.123 / 0.090 |
| 0.2–0.3 |
69 / 0.259 / 0.159 |
86 / 0.242 / 0.093 |
30 / 0.245 / 0.067 |
| 0.3–0.4 |
21 / 0.337 / 0.048 |
42 / 0.346 / 0.071 |
35 / 0.358 / 0.057 |
| 0.4–0.5 |
8 / 0.456 / 0.000 |
28 / 0.454 / 0.143 |
13 / 0.448 / 0.308 |
| 0.5–0.6 |
11 / 0.527 / 0.000 |
6 / 0.540 / 0.500 |
6 / 0.546 / 0.500 |
| 0.6–0.7 |
3 / 0.658 / 0.333 |
4 / 0.634 / 0.250 |
1 / 0.697 / 0.000 |
| 0.7–0.8 |
2 / 0.745 / 0.000 |
2 / 0.770 / 0.000 |
5 / 0.754 / 0.200 |
| Any-channel feature (all events) |
Coefficient |
| intercept |
-5.682 |
| log DR plays t |
+0.672 |
| log DR plays 4w |
+0.859 |
| log DR plays 12w |
-0.429 |
| log max DR plays 12w |
+0.201 |
| in rotation anywhere now |
-0.575 |
| station groups in rotation now (/5) |
-0.131 |
| current run anywhere (/12) |
-1.254 |
| weeks since in rotation anywhere (/12) |
-0.225 |
| Track Top-40 score |
+0.602 |
| Airplay Top-20 score |
-0.736 |
| chart entry recency 6w |
+0.145 |
| chart entry recency 26w |
-0.668 |
| log Track Top-40 hits |
-0.292 |
| log Airplay Top-20 hits |
-0.136 |
| Danish |
+0.821 |
| group |
-0.081 |
| female |
-0.185 |
| career years (/50) |
+0.645 |
| debut unknown |
+0.056 |
| Christmas in window |
+0.000 |
| discography known |
-0.000 |
| release recency 4w |
-0.154 |
| release recency 12w |
-1.148 |
| log releases last 52w |
-0.286 |
| releases in the window |
-0.000 |
| releases in window x not in rotation now |
+0.575 |
| DR: weeks since last add 13-25 |
-0.412 |
| DR: last add 26+ weeks ago |
-0.142 |
| DR: no add in the panel yet |
-0.811 |
| Airplay Top-20: last new song 13-25 weeks ago |
+0.833 |
| Airplay Top-20: 0.5-1.25x own usual gap |
-0.222 |
| Airplay Top-20: last new song 2+ years ago |
+0.215 |
| log archive plays per 100 sampled hours (2012-21) |
+0.591 |
| log DR lists with the artist before t (2023-) |
+0.160 |
| DR list recency 26w |
-0.506 |
| DR lists known before t at all |
-0.279 |
| NMF Denmark in the window |
+0.923 |
| NMF Denmark top 10 in the window |
+0.249 |
| log NMF Denmark placements last 52w |
+0.276 |
| share of releases on NMF Denmark (2 years) |
+0.002 |
| season index of the window (log, IFPI 2002-24) |
-0.071 |