# Backtest: if the artist releases a new song, will DR add it? Events: an artist knowable at the time releases something (Deezer) in week t0+1, with no other release in the 6 weeks before (so an add can't be the previous song's); target: a DR add (entering rotation after 4+ weeks out, or plays jumping well above the artist's own 8-week baseline and staying up) within 2 week(s) from the release week. The model knows the artist's state the week before; pre-release rows blend the new song's NMF Denmark placement from the artist's history. "Release timing only" = the add rate by weeks since the artist's previous release. 3588 release events scored, 64 (1.8%) got an add on some DR channel. | Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel | |---|---|---|---|---| | rate by in-rotation-now | 0.0223 | 0.013 | 0.0873 | 0.048 | | rate by last-12-weeks level | 0.0158 | 0.076 | 0.0697 | 0.126 | | release timing only | 0.0242 | 0.003 | 0.0904 | 0.019 | | model, calibrated on the last 6 origins (pre-release) | 0.0163 | 0.089 | 0.0682 | 0.174 | | model, raw (pre-release) | 0.0163 | 0.095 | 0.0649 | 0.235 | | model + pooled Platt (pre-release) | 0.0158 | 0.090 | 0.0655 | 0.224 | | model + pooled isotonic (pre-release) | 0.0160 | 0.083 | 0.0654 | 0.228 | | model, NMF known (post-release) | 0.0164 | 0.087 | 0.0685 | 0.160 | Skill (1 − log-loss ÷ baseline log-loss; 90% interval from a bootstrap over blocks of 8 forecast weeks): | Model | Per channel vs last-12-weeks | Per channel vs release timing | Any channel vs last-12-weeks | Any channel vs release timing | |---|---|---|---|---| | model, calibrated on the last 6 origins (pre-release) | -0.031 (-0.101 to +0.021) | +0.326 (+0.274 to +0.383) | +0.022 (+0.001 to +0.080) | +0.246 (+0.232 to +0.295) | | model + pooled Platt (pre-release) | +0.002 (-0.041 to +0.034) | +0.348 (+0.291 to +0.406) | +0.060 (+0.031 to +0.108) | +0.275 (+0.232 to +0.327) | | model + pooled isotonic (pre-release) | -0.010 (-0.040 to +0.024) | +0.340 (+0.282 to +0.408) | +0.062 (+0.042 to +0.098) | +0.277 (+0.231 to +0.327) | Calibration, any channel: | Predicted | model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed | model + pooled Platt (pre-release): rows / mean predicted / observed | model + pooled isotonic (pre-release): rows / mean predicted / observed | |---|---|---|---| | 0.0–0.1 | 3485 / 0.014 / 0.011 | 3432 / 0.010 / 0.010 | 3416 / 0.009 / 0.009 | | 0.1–0.2 | 72 / 0.138 / 0.222 | 104 / 0.142 / 0.135 | 129 / 0.142 / 0.155 | | 0.2–0.3 | 19 / 0.245 / 0.316 | 24 / 0.240 / 0.250 | 19 / 0.238 / 0.211 | | 0.3–0.4 | 7 / 0.334 / 0.143 | 12 / 0.329 / 0.167 | 7 / 0.334 / 0.143 | | 0.4–0.5 | 4 / 0.460 / 0.250 | 10 / 0.447 / 0.500 | 6 / 0.425 / 0.500 | | 0.5–0.6 | 1 / 0.579 / 0.000 | 1 / 0.575 / 1.000 | 1 / 0.535 / 1.000 | | 0.6–0.7 | — | 4 / 0.625 / 0.500 | 4 / 0.630 / 0.750 | | 0.7–0.8 | — | 1 / 0.799 / 0.000 | 3 / 0.749 / 0.333 | | 0.8–0.9 | — | — | 3 / 0.831 / 0.333 | Calibration, per channel: | Predicted | model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed | model + pooled Platt (pre-release): rows / mean predicted / observed | model + pooled isotonic (pre-release): rows / mean predicted / observed | |---|---|---|---| | 0.0–0.1 | 49992 / 0.003 / 0.003 | 49847 / 0.002 / 0.002 | 49630 / 0.002 / 0.002 | | 0.1–0.2 | 143 / 0.132 / 0.161 | 199 / 0.136 / 0.136 | 482 / 0.142 / 0.110 | | 0.2–0.3 | 55 / 0.263 / 0.036 | 108 / 0.250 / 0.157 | 70 / 0.240 / 0.114 | | 0.3–0.4 | 23 / 0.335 / 0.000 | 43 / 0.339 / 0.163 | 34 / 0.356 / 0.176 | | 0.4–0.5 | 8 / 0.453 / 0.250 | 25 / 0.457 / 0.040 | 15 / 0.441 / 0.067 | | 0.5–0.6 | 8 / 0.531 / 0.000 | 8 / 0.522 / 0.125 | — | | 0.6–0.7 | 2 / 0.677 / 0.000 | 2 / 0.626 / 0.000 | 1 / 0.647 / 0.000 | | 0.7–0.8 | 1 / 0.723 / 1.000 | — | — | Quiet artists (out of rotation everywhere for 26 weeks) releasing, any channel: | Group | Events | Adds | Rate | Mean predicted (raw) | Mean predicted (isotonic) | Log-loss (isotonic) | PR-AUC (raw) | |---|---|---|---|---|---|---|---| | former favourites (4+ weeks on DR 2019-24) | 348 | 7 | 0.020 | 0.022 | 0.027 | 0.0930 | 0.104 | | other quiet artists | 2892 | 16 | 0.006 | 0.006 | 0.008 | 0.0299 | 0.126 | | Any-channel feature (all events) | Coefficient | |---|---| | intercept | -5.798 | | log DR plays t | +0.782 | | log DR plays 4w | +0.714 | | log DR plays 12w | -0.537 | | log max DR plays 12w | +0.402 | | in rotation anywhere now | -0.644 | | station groups in rotation now (/5) | -0.157 | | current run anywhere (/12) | -1.205 | | weeks since in rotation anywhere (/12) | -0.271 | | Track Top-40 score | +0.756 | | Airplay Top-20 score | -0.521 | | chart entry recency 6w | +0.323 | | chart entry recency 26w | -0.592 | | log Track Top-40 hits | -0.212 | | log Airplay Top-20 hits | -0.186 | | Danish | +0.819 | | group | -0.151 | | female | -0.024 | | career years (/50) | +0.476 | | debut unknown | -0.153 | | Christmas in window | +0.000 | | discography known | -0.000 | | release recency 4w | -0.270 | | release recency 12w | -1.329 | | log releases last 52w | -0.233 | | releases in the window | -0.000 | | releases in window x not in rotation now | +0.644 | | DR: weeks since last add 13-25 | -0.315 | | DR: last add 26+ weeks ago | -0.155 | | DR: no add in the panel yet | -0.611 | | Airplay Top-20: last new song 13-25 weeks ago | +0.771 | | Airplay Top-20: 0.5-1.25x own usual gap | -0.238 | | Airplay Top-20: last new song 2+ years ago | +0.257 | | log archive plays per 100 sampled hours (2012-21) | +0.550 | | log DR lists with the artist before t (2023-) | +0.194 | | DR list recency 26w | -0.668 | | DR lists known before t at all | -0.266 | | quiet former favourite (4+ weeks on DR 2019-24, out of rotation 26w) | -0.645 | | NMF Denmark in the window | +0.820 | | NMF Denmark top 10 in the window | +0.107 | | log NMF Denmark placements last 52w | +0.131 | | share of releases on NMF Denmark (2 years) | +0.204 | | season index of the window (log, IFPI 2002-24) | -0.066 |