Files
2026-11-19 15:19:00 +00:00

6.0 KiB
Raw Permalink Blame History

Backtest: if the artist releases a new song, will DR add it?

Events: an artist knowable at the time releases something (Deezer) in week t0+1, with no other release in the 6 weeks before (so an add can't be the previous song's); target: a DR add (entering rotation after 4+ weeks out, or plays jumping well above the artist's own 8-week baseline and staying up) within 2 week(s) from the release week. The model knows the artist's state the week before; pre-release rows blend the new song's NMF Denmark placement from the artist's history. "Release timing only" = the add rate by weeks since the artist's previous release. 3588 release events scored, 64 (1.8%) got an add on some DR channel.

Model Log-loss, per channel PR-AUC, per channel Log-loss, any channel PR-AUC, any channel
rate by in-rotation-now 0.0223 0.013 0.0873 0.048
rate by last-12-weeks level 0.0158 0.076 0.0697 0.126
release timing only 0.0242 0.003 0.0904 0.019
model, calibrated on the last 6 origins (pre-release) 0.0163 0.089 0.0682 0.174
model, raw (pre-release) 0.0163 0.095 0.0649 0.235
model + pooled Platt (pre-release) 0.0158 0.090 0.0655 0.224
model + pooled isotonic (pre-release) 0.0160 0.083 0.0654 0.228
model, NMF known (post-release) 0.0164 0.087 0.0685 0.160

Skill (1 − log-loss ÷ baseline log-loss; 90% interval from a bootstrap over blocks of 8 forecast weeks):

Model Per channel vs last-12-weeks Per channel vs release timing Any channel vs last-12-weeks Any channel vs release timing
model, calibrated on the last 6 origins (pre-release) -0.031 (-0.101 to +0.021) +0.326 (+0.274 to +0.383) +0.022 (+0.001 to +0.080) +0.246 (+0.232 to +0.295)
model + pooled Platt (pre-release) +0.002 (-0.041 to +0.034) +0.348 (+0.291 to +0.406) +0.060 (+0.031 to +0.108) +0.275 (+0.232 to +0.327)
model + pooled isotonic (pre-release) -0.010 (-0.040 to +0.024) +0.340 (+0.282 to +0.408) +0.062 (+0.042 to +0.098) +0.277 (+0.231 to +0.327)

Calibration, any channel:

Predicted model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed model + pooled Platt (pre-release): rows / mean predicted / observed model + pooled isotonic (pre-release): rows / mean predicted / observed
0.0–0.1 3485 / 0.014 / 0.011 3432 / 0.010 / 0.010 3416 / 0.009 / 0.009
0.1–0.2 72 / 0.138 / 0.222 104 / 0.142 / 0.135 129 / 0.142 / 0.155
0.2–0.3 19 / 0.245 / 0.316 24 / 0.240 / 0.250 19 / 0.238 / 0.211
0.3–0.4 7 / 0.334 / 0.143 12 / 0.329 / 0.167 7 / 0.334 / 0.143
0.4–0.5 4 / 0.460 / 0.250 10 / 0.447 / 0.500 6 / 0.425 / 0.500
0.5–0.6 1 / 0.579 / 0.000 1 / 0.575 / 1.000 1 / 0.535 / 1.000
0.6–0.7 — 4 / 0.625 / 0.500 4 / 0.630 / 0.750
0.7–0.8 — 1 / 0.799 / 0.000 3 / 0.749 / 0.333
0.8–0.9 — — 3 / 0.831 / 0.333

Calibration, per channel:

Predicted model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed model + pooled Platt (pre-release): rows / mean predicted / observed model + pooled isotonic (pre-release): rows / mean predicted / observed
0.0–0.1 49992 / 0.003 / 0.003 49847 / 0.002 / 0.002 49630 / 0.002 / 0.002
0.1–0.2 143 / 0.132 / 0.161 199 / 0.136 / 0.136 482 / 0.142 / 0.110
0.2–0.3 55 / 0.263 / 0.036 108 / 0.250 / 0.157 70 / 0.240 / 0.114
0.3–0.4 23 / 0.335 / 0.000 43 / 0.339 / 0.163 34 / 0.356 / 0.176
0.4–0.5 8 / 0.453 / 0.250 25 / 0.457 / 0.040 15 / 0.441 / 0.067
0.5–0.6 8 / 0.531 / 0.000 8 / 0.522 / 0.125 —
0.6–0.7 2 / 0.677 / 0.000 2 / 0.626 / 0.000 1 / 0.647 / 0.000
0.7–0.8 1 / 0.723 / 1.000 — —

Quiet artists (out of rotation everywhere for 26 weeks) releasing, any channel:

Group Events Adds Rate Mean predicted (raw) Mean predicted (isotonic) Log-loss (isotonic) PR-AUC (raw)
former favourites (4+ weeks on DR 2019-24) 348 7 0.020 0.022 0.027 0.0930 0.104
other quiet artists 2892 16 0.006 0.006 0.008 0.0299 0.126
Any-channel feature (all events) Coefficient
intercept -5.798
log DR plays t +0.782
log DR plays 4w +0.714
log DR plays 12w -0.537
log max DR plays 12w +0.402
in rotation anywhere now -0.644
station groups in rotation now (/5) -0.157
current run anywhere (/12) -1.205
weeks since in rotation anywhere (/12) -0.271
Track Top-40 score +0.756
Airplay Top-20 score -0.521
chart entry recency 6w +0.323
chart entry recency 26w -0.592
log Track Top-40 hits -0.212
log Airplay Top-20 hits -0.186
Danish +0.819
group -0.151
female -0.024
career years (/50) +0.476
debut unknown -0.153
Christmas in window +0.000
discography known -0.000
release recency 4w -0.270
release recency 12w -1.329
log releases last 52w -0.233
releases in the window -0.000
releases in window x not in rotation now +0.644
DR: weeks since last add 13-25 -0.315
DR: last add 26+ weeks ago -0.155
DR: no add in the panel yet -0.611
Airplay Top-20: last new song 13-25 weeks ago +0.771
Airplay Top-20: 0.5-1.25x own usual gap -0.238
Airplay Top-20: last new song 2+ years ago +0.257
log archive plays per 100 sampled hours (2012-21) +0.550
log DR lists with the artist before t (2023-) +0.194
DR list recency 26w -0.668
DR lists known before t at all -0.266
quiet former favourite (4+ weeks on DR 2019-24, out of rotation 26w) -0.645
NMF Denmark in the window +0.820
NMF Denmark top 10 in the window +0.107
log NMF Denmark placements last 52w +0.131
share of releases on NMF Denmark (2 years) +0.204
season index of the window (log, IFPI 2002-24) -0.066