# Backtest: if the artist releases a new song, will DR add it? Events: an artist knowable at the time releases something (Deezer) in week t0+1, with no other release in the 6 weeks before (so an add can't be the previous song's); target: a DR add (entering rotation after 4+ weeks out, or plays jumping well above the artist's own 8-week baseline and staying up) within 2 week(s) from the release week. The model knows the artist's state the week before; pre-release rows blend the new song's NMF Denmark placement from the artist's history. "Release timing only" = the add rate by weeks since the artist's previous release. 3193 release events scored, 63 (2.0%) got an add on some DR channel. | Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel | |---|---|---|---|---| | rate by in-rotation-now | 0.0245 | 0.015 | 0.0941 | 0.062 | | rate by last-12-weeks level | 0.0174 | 0.082 | 0.0738 | 0.140 | | release timing only | 0.0267 | 0.003 | 0.0982 | 0.021 | | model, calibrated on the last 6 origins (pre-release) | 0.0183 | 0.080 | 0.0748 | 0.170 | | model, raw (pre-release) | 0.0182 | 0.091 | 0.0696 | 0.233 | | model + pooled Platt (pre-release) | 0.0177 | 0.094 | 0.0704 | 0.224 | | model + pooled isotonic (pre-release) | 0.0176 | 0.091 | 0.0706 | 0.235 | | model, NMF known (post-release) | 0.0184 | 0.085 | 0.0750 | 0.163 | Skill (1 − log-loss ÷ baseline log-loss; 90% interval from a bootstrap over blocks of 8 forecast weeks): | Model | Per channel vs last-12-weeks | Per channel vs release timing | Any channel vs last-12-weeks | Any channel vs release timing | |---|---|---|---|---| | model, calibrated on the last 6 origins (pre-release) | -0.053 (-0.131 to +0.002) | +0.313 (+0.257 to +0.378) | -0.013 (-0.017 to +0.048) | +0.239 (+0.226 to +0.288) | | model + pooled Platt (pre-release) | -0.015 (-0.053 to +0.020) | +0.338 (+0.286 to +0.399) | +0.046 (-0.008 to +0.107) | +0.283 (+0.234 to +0.334) | | model + pooled isotonic (pre-release) | -0.010 (-0.095 to +0.039) | +0.342 (+0.234 to +0.430) | +0.044 (-0.011 to +0.100) | +0.281 (+0.223 to +0.335) | Calibration, any channel: | Predicted | model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed | model + pooled Platt (pre-release): rows / mean predicted / observed | model + pooled isotonic (pre-release): rows / mean predicted / observed | |---|---|---|---| | 0.0–0.1 | 3081 / 0.015 / 0.013 | 3033 / 0.011 / 0.010 | 2979 / 0.009 / 0.008 | | 0.1–0.2 | 82 / 0.136 / 0.207 | 107 / 0.140 / 0.168 | 169 / 0.135 / 0.160 | | 0.2–0.3 | 16 / 0.245 / 0.312 | 20 / 0.239 / 0.300 | 16 / 0.238 / 0.188 | | 0.3–0.4 | 6 / 0.316 / 0.333 | 12 / 0.340 / 0.250 | 8 / 0.344 / 0.375 | | 0.4–0.5 | 4 / 0.451 / 0.000 | 13 / 0.437 / 0.231 | 8 / 0.436 / 0.250 | | 0.5–0.6 | 2 / 0.533 / 0.000 | 6 / 0.566 / 0.667 | 4 / 0.563 / 0.250 | | 0.6–0.7 | 2 / 0.636 / 0.000 | 1 / 0.605 / 0.000 | 2 / 0.620 / 0.000 | | 0.7–0.8 | — | — | 5 / 0.775 / 0.600 | | 0.8–0.9 | — | 1 / 0.818 / 0.000 | 2 / 0.827 / 0.500 | Calibration, per channel: | Predicted | model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed | model + pooled Platt (pre-release): rows / mean predicted / observed | model + pooled isotonic (pre-release): rows / mean predicted / observed | |---|---|---|---| | 0.0–0.1 | 44463 / 0.004 / 0.003 | 44305 / 0.002 / 0.003 | 44334 / 0.003 / 0.003 | | 0.1–0.2 | 125 / 0.136 / 0.128 | 229 / 0.137 / 0.105 | 278 / 0.123 / 0.090 | | 0.2–0.3 | 69 / 0.259 / 0.159 | 86 / 0.242 / 0.093 | 30 / 0.245 / 0.067 | | 0.3–0.4 | 21 / 0.337 / 0.048 | 42 / 0.346 / 0.071 | 35 / 0.358 / 0.057 | | 0.4–0.5 | 8 / 0.456 / 0.000 | 28 / 0.454 / 0.143 | 13 / 0.448 / 0.308 | | 0.5–0.6 | 11 / 0.527 / 0.000 | 6 / 0.540 / 0.500 | 6 / 0.546 / 0.500 | | 0.6–0.7 | 3 / 0.658 / 0.333 | 4 / 0.634 / 0.250 | 1 / 0.697 / 0.000 | | 0.7–0.8 | 2 / 0.745 / 0.000 | 2 / 0.770 / 0.000 | 5 / 0.754 / 0.200 | | Any-channel feature (all events) | Coefficient | |---|---| | intercept | -5.682 | | log DR plays t | +0.672 | | log DR plays 4w | +0.859 | | log DR plays 12w | -0.429 | | log max DR plays 12w | +0.201 | | in rotation anywhere now | -0.575 | | station groups in rotation now (/5) | -0.131 | | current run anywhere (/12) | -1.254 | | weeks since in rotation anywhere (/12) | -0.225 | | Track Top-40 score | +0.602 | | Airplay Top-20 score | -0.736 | | chart entry recency 6w | +0.145 | | chart entry recency 26w | -0.668 | | log Track Top-40 hits | -0.292 | | log Airplay Top-20 hits | -0.136 | | Danish | +0.821 | | group | -0.081 | | female | -0.185 | | career years (/50) | +0.645 | | debut unknown | +0.056 | | Christmas in window | +0.000 | | discography known | -0.000 | | release recency 4w | -0.154 | | release recency 12w | -1.148 | | log releases last 52w | -0.286 | | releases in the window | -0.000 | | releases in window x not in rotation now | +0.575 | | DR: weeks since last add 13-25 | -0.412 | | DR: last add 26+ weeks ago | -0.142 | | DR: no add in the panel yet | -0.811 | | Airplay Top-20: last new song 13-25 weeks ago | +0.833 | | Airplay Top-20: 0.5-1.25x own usual gap | -0.222 | | Airplay Top-20: last new song 2+ years ago | +0.215 | | log archive plays per 100 sampled hours (2012-21) | +0.591 | | log DR lists with the artist before t (2023-) | +0.160 | | DR list recency 26w | -0.506 | | DR lists known before t at all | -0.279 | | NMF Denmark in the window | +0.923 | | NMF Denmark top 10 in the window | +0.249 | | log NMF Denmark placements last 52w | +0.276 | | share of releases on NMF Denmark (2 years) | +0.002 | | season index of the window (log, IFPI 2002-24) | -0.071 |