# Backtest: if the artist releases a new song, will DR add it? Events: an artist knowable at the time releases something (Deezer) in week t0+1, with no other release in the 6 weeks before (so an add can't be the previous song's); target: a DR add (entering rotation after 4+ weeks out, or plays jumping well above the artist's own 8-week baseline and staying up) within 2 week(s) from the release week. The model knows the artist's state the week before; pre-release rows blend the new song's NMF Denmark placement from the artist's history. "Release timing only" = the add rate by weeks since the artist's previous release. 3269 release events scored, 63 (1.9%) got an add on some DR channel. | Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel | |---|---|---|---|---| | rate by in-rotation-now | 0.0240 | 0.014 | 0.0923 | 0.062 | | rate by last-12-weeks level | 0.0170 | 0.082 | 0.0721 | 0.140 | | release timing only | 0.0262 | 0.003 | 0.0964 | 0.020 | | model, calibrated on the last 6 origins (pre-release) | 0.0179 | 0.085 | 0.0723 | 0.179 | | model, raw (pre-release) | 0.0178 | 0.097 | 0.0676 | 0.245 | | model + pooled Platt (pre-release) | 0.0172 | 0.098 | 0.0683 | 0.232 | | model + pooled isotonic (pre-release) | 0.0175 | 0.088 | 0.0689 | 0.252 | | model, NMF known (post-release) | 0.0180 | 0.090 | 0.0723 | 0.170 | Skill (1 − log-loss ÷ baseline log-loss; 90% interval from a bootstrap over blocks of 8 forecast weeks): | Model | Per channel vs last-12-weeks | Per channel vs release timing | Any channel vs last-12-weeks | Any channel vs release timing | |---|---|---|---|---| | model, calibrated on the last 6 origins (pre-release) | -0.054 (-0.109 to -0.010) | +0.315 (+0.264 to +0.373) | -0.002 (-0.006 to +0.060) | +0.250 (+0.237 to +0.299) | | model + pooled Platt (pre-release) | -0.010 (-0.041 to +0.026) | +0.343 (+0.296 to +0.404) | +0.053 (+0.000 to +0.115) | +0.291 (+0.243 to +0.343) | | model + pooled isotonic (pre-release) | -0.028 (-0.082 to +0.022) | +0.331 (+0.248 to +0.418) | +0.045 (-0.000 to +0.098) | +0.286 (+0.228 to +0.338) | Calibration, any channel: | Predicted | model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed | model + pooled Platt (pre-release): rows / mean predicted / observed | model + pooled isotonic (pre-release): rows / mean predicted / observed | |---|---|---|---| | 0.0–0.1 | 3159 / 0.014 / 0.012 | 3108 / 0.011 / 0.009 | 3100 / 0.009 / 0.009 | | 0.1–0.2 | 80 / 0.140 / 0.212 | 103 / 0.140 / 0.165 | 105 / 0.137 / 0.152 | | 0.2–0.3 | 15 / 0.259 / 0.200 | 23 / 0.227 / 0.217 | 33 / 0.227 / 0.273 | | 0.3–0.4 | 10 / 0.337 / 0.400 | 17 / 0.345 / 0.353 | 9 / 0.353 / 0.222 | | 0.4–0.5 | 2 / 0.440 / 0.000 | 8 / 0.438 / 0.250 | 7 / 0.445 / 0.429 | | 0.5–0.6 | 1 / 0.517 / 0.000 | 5 / 0.534 / 0.200 | 4 / 0.561 / 0.000 | | 0.6–0.7 | 2 / 0.658 / 0.000 | 4 / 0.631 / 0.750 | 5 / 0.644 / 0.200 | | 0.7–0.8 | — | — | 1 / 0.709 / 1.000 | | 0.8–0.9 | — | 1 / 0.813 / 0.000 | 5 / 0.839 / 0.600 | Calibration, per channel: | Predicted | model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed | model + pooled Platt (pre-release): rows / mean predicted / observed | model + pooled isotonic (pre-release): rows / mean predicted / observed | |---|---|---|---| | 0.0–0.1 | 45525 / 0.003 / 0.003 | 45392 / 0.002 / 0.003 | 45368 / 0.003 / 0.003 | | 0.1–0.2 | 126 / 0.135 / 0.135 | 182 / 0.133 / 0.082 | 301 / 0.124 / 0.083 | | 0.2–0.3 | 39 / 0.247 / 0.256 | 95 / 0.246 / 0.168 | 36 / 0.242 / 0.056 | | 0.3–0.4 | 54 / 0.340 / 0.037 | 54 / 0.333 / 0.093 | 32 / 0.341 / 0.094 | | 0.4–0.5 | 8 / 0.467 / 0.125 | 25 / 0.461 / 0.160 | 10 / 0.447 / 0.400 | | 0.5–0.6 | 9 / 0.533 / 0.000 | 12 / 0.521 / 0.250 | 8 / 0.554 / 0.375 | | 0.6–0.7 | 1 / 0.657 / 0.000 | 4 / 0.660 / 0.000 | 7 / 0.650 / 0.000 | | 0.7–0.8 | 4 / 0.754 / 0.250 | 2 / 0.730 / 0.500 | 4 / 0.734 / 0.250 | Quiet artists (out of rotation everywhere for 26 weeks) releasing, any channel: | Group | Events | Adds | Rate | Mean predicted (raw) | Mean predicted (isotonic) | Log-loss (isotonic) | PR-AUC (raw) | |---|---|---|---|---|---|---|---| | former favourites (4+ weeks on DR 2019-24) | 326 | 7 | 0.021 | 0.025 | 0.030 | 0.1028 | 0.095 | | other quiet artists | 2616 | 15 | 0.006 | 0.007 | 0.008 | 0.0313 | 0.126 | | Any-channel feature (all events) | Coefficient | |---|---| | intercept | -5.988 | | log DR plays t | +0.677 | | log DR plays 4w | +0.897 | | log DR plays 12w | -0.425 | | log max DR plays 12w | +0.233 | | in rotation anywhere now | -0.575 | | station groups in rotation now (/5) | -0.137 | | current run anywhere (/12) | -1.280 | | weeks since in rotation anywhere (/12) | -0.016 | | Track Top-40 score | +0.603 | | Airplay Top-20 score | -0.700 | | chart entry recency 6w | +0.172 | | chart entry recency 26w | -0.684 | | log Track Top-40 hits | -0.283 | | log Airplay Top-20 hits | -0.147 | | Danish | +0.830 | | group | -0.078 | | female | -0.173 | | career years (/50) | +0.646 | | debut unknown | +0.019 | | Christmas in window | +0.000 | | discography known | -0.000 | | release recency 4w | -0.157 | | release recency 12w | -1.118 | | log releases last 52w | -0.285 | | releases in the window | -0.000 | | releases in window x not in rotation now | +0.575 | | DR: weeks since last add 13-25 | -0.493 | | DR: last add 26+ weeks ago | -0.099 | | DR: no add in the panel yet | -0.708 | | Airplay Top-20: last new song 13-25 weeks ago | +0.826 | | Airplay Top-20: 0.5-1.25x own usual gap | -0.176 | | Airplay Top-20: last new song 2+ years ago | +0.334 | | log archive plays per 100 sampled hours (2012-21) | +0.618 | | log DR lists with the artist before t (2023-) | +0.156 | | DR list recency 26w | -0.607 | | DR lists known before t at all | -0.210 | | quiet former favourite (4+ weeks on DR 2019-24, out of rotation 26w) | -0.665 | | NMF Denmark in the window | +0.943 | | NMF Denmark top 10 in the window | +0.202 | | log NMF Denmark placements last 52w | +0.251 | | share of releases on NMF Denmark (2 years) | +0.020 | | season index of the window (log, IFPI 2002-24) | -0.049 |