Comebacks and event backtest with iTunes release dates
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+71
-71
@@ -1,103 +1,103 @@
|
||||
# Backtest: if the artist releases a new song, will DR add it?
|
||||
|
||||
Events: an artist knowable at the time releases something (Deezer) in week t0+1, with no other release in the 6 weeks before (so an add can't be the previous song's); target: a DR add (entering rotation after 4+ weeks out, or plays jumping well above the artist's own 8-week baseline and staying up) within 2 week(s) from the release week. The model knows the artist's state the week before; pre-release rows blend the new song's NMF Denmark placement from the artist's history. "Release timing only" = the add rate by weeks since the artist's previous release. 3269 release events scored, 63 (1.9%) got an add on some DR channel.
|
||||
Events: an artist knowable at the time releases something (Deezer) in week t0+1, with no other release in the 6 weeks before (so an add can't be the previous song's); target: a DR add (entering rotation after 4+ weeks out, or plays jumping well above the artist's own 8-week baseline and staying up) within 2 week(s) from the release week. The model knows the artist's state the week before; pre-release rows blend the new song's NMF Denmark placement from the artist's history. "Release timing only" = the add rate by weeks since the artist's previous release. 3588 release events scored, 64 (1.8%) got an add on some DR channel.
|
||||
|
||||
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|
||||
|---|---|---|---|---|
|
||||
| rate by in-rotation-now | 0.0240 | 0.014 | 0.0923 | 0.062 |
|
||||
| rate by last-12-weeks level | 0.0170 | 0.082 | 0.0721 | 0.140 |
|
||||
| release timing only | 0.0262 | 0.003 | 0.0964 | 0.020 |
|
||||
| model, calibrated on the last 6 origins (pre-release) | 0.0179 | 0.085 | 0.0723 | 0.179 |
|
||||
| model, raw (pre-release) | 0.0178 | 0.097 | 0.0676 | 0.245 |
|
||||
| model + pooled Platt (pre-release) | 0.0172 | 0.098 | 0.0683 | 0.232 |
|
||||
| model + pooled isotonic (pre-release) | 0.0175 | 0.088 | 0.0689 | 0.252 |
|
||||
| model, NMF known (post-release) | 0.0180 | 0.090 | 0.0723 | 0.170 |
|
||||
| rate by in-rotation-now | 0.0223 | 0.013 | 0.0873 | 0.048 |
|
||||
| rate by last-12-weeks level | 0.0158 | 0.076 | 0.0697 | 0.126 |
|
||||
| release timing only | 0.0242 | 0.003 | 0.0904 | 0.019 |
|
||||
| model, calibrated on the last 6 origins (pre-release) | 0.0163 | 0.089 | 0.0682 | 0.174 |
|
||||
| model, raw (pre-release) | 0.0163 | 0.095 | 0.0649 | 0.235 |
|
||||
| model + pooled Platt (pre-release) | 0.0158 | 0.090 | 0.0655 | 0.224 |
|
||||
| model + pooled isotonic (pre-release) | 0.0160 | 0.083 | 0.0654 | 0.228 |
|
||||
| model, NMF known (post-release) | 0.0164 | 0.087 | 0.0685 | 0.160 |
|
||||
|
||||
Skill (1 − log-loss ÷ baseline log-loss; 90% interval from a bootstrap over blocks of 8 forecast weeks):
|
||||
|
||||
| Model | Per channel vs last-12-weeks | Per channel vs release timing | Any channel vs last-12-weeks | Any channel vs release timing |
|
||||
|---|---|---|---|---|
|
||||
| model, calibrated on the last 6 origins (pre-release) | -0.054 (-0.109 to -0.010) | +0.315 (+0.264 to +0.373) | -0.002 (-0.006 to +0.060) | +0.250 (+0.237 to +0.299) |
|
||||
| model + pooled Platt (pre-release) | -0.010 (-0.041 to +0.026) | +0.343 (+0.296 to +0.404) | +0.053 (+0.000 to +0.115) | +0.291 (+0.243 to +0.343) |
|
||||
| model + pooled isotonic (pre-release) | -0.028 (-0.082 to +0.022) | +0.331 (+0.248 to +0.418) | +0.045 (-0.000 to +0.098) | +0.286 (+0.228 to +0.338) |
|
||||
| model, calibrated on the last 6 origins (pre-release) | -0.031 (-0.101 to +0.021) | +0.326 (+0.274 to +0.383) | +0.022 (+0.001 to +0.080) | +0.246 (+0.232 to +0.295) |
|
||||
| model + pooled Platt (pre-release) | +0.002 (-0.041 to +0.034) | +0.348 (+0.291 to +0.406) | +0.060 (+0.031 to +0.108) | +0.275 (+0.232 to +0.327) |
|
||||
| model + pooled isotonic (pre-release) | -0.010 (-0.040 to +0.024) | +0.340 (+0.282 to +0.408) | +0.062 (+0.042 to +0.098) | +0.277 (+0.231 to +0.327) |
|
||||
|
||||
Calibration, any channel:
|
||||
|
||||
| Predicted | model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed | model + pooled Platt (pre-release): rows / mean predicted / observed | model + pooled isotonic (pre-release): rows / mean predicted / observed |
|
||||
|---|---|---|---|
|
||||
| 0.0–0.1 | 3159 / 0.014 / 0.012 | 3108 / 0.011 / 0.009 | 3100 / 0.009 / 0.009 |
|
||||
| 0.1–0.2 | 80 / 0.140 / 0.212 | 103 / 0.140 / 0.165 | 105 / 0.137 / 0.152 |
|
||||
| 0.2–0.3 | 15 / 0.259 / 0.200 | 23 / 0.227 / 0.217 | 33 / 0.227 / 0.273 |
|
||||
| 0.3–0.4 | 10 / 0.337 / 0.400 | 17 / 0.345 / 0.353 | 9 / 0.353 / 0.222 |
|
||||
| 0.4–0.5 | 2 / 0.440 / 0.000 | 8 / 0.438 / 0.250 | 7 / 0.445 / 0.429 |
|
||||
| 0.5–0.6 | 1 / 0.517 / 0.000 | 5 / 0.534 / 0.200 | 4 / 0.561 / 0.000 |
|
||||
| 0.6–0.7 | 2 / 0.658 / 0.000 | 4 / 0.631 / 0.750 | 5 / 0.644 / 0.200 |
|
||||
| 0.7–0.8 | — | — | 1 / 0.709 / 1.000 |
|
||||
| 0.8–0.9 | — | 1 / 0.813 / 0.000 | 5 / 0.839 / 0.600 |
|
||||
| 0.0–0.1 | 3485 / 0.014 / 0.011 | 3432 / 0.010 / 0.010 | 3416 / 0.009 / 0.009 |
|
||||
| 0.1–0.2 | 72 / 0.138 / 0.222 | 104 / 0.142 / 0.135 | 129 / 0.142 / 0.155 |
|
||||
| 0.2–0.3 | 19 / 0.245 / 0.316 | 24 / 0.240 / 0.250 | 19 / 0.238 / 0.211 |
|
||||
| 0.3–0.4 | 7 / 0.334 / 0.143 | 12 / 0.329 / 0.167 | 7 / 0.334 / 0.143 |
|
||||
| 0.4–0.5 | 4 / 0.460 / 0.250 | 10 / 0.447 / 0.500 | 6 / 0.425 / 0.500 |
|
||||
| 0.5–0.6 | 1 / 0.579 / 0.000 | 1 / 0.575 / 1.000 | 1 / 0.535 / 1.000 |
|
||||
| 0.6–0.7 | — | 4 / 0.625 / 0.500 | 4 / 0.630 / 0.750 |
|
||||
| 0.7–0.8 | — | 1 / 0.799 / 0.000 | 3 / 0.749 / 0.333 |
|
||||
| 0.8–0.9 | — | — | 3 / 0.831 / 0.333 |
|
||||
|
||||
Calibration, per channel:
|
||||
|
||||
| Predicted | model, calibrated on the last 6 origins (pre-release): rows / mean predicted / observed | model + pooled Platt (pre-release): rows / mean predicted / observed | model + pooled isotonic (pre-release): rows / mean predicted / observed |
|
||||
|---|---|---|---|
|
||||
| 0.0–0.1 | 45525 / 0.003 / 0.003 | 45392 / 0.002 / 0.003 | 45368 / 0.003 / 0.003 |
|
||||
| 0.1–0.2 | 126 / 0.135 / 0.135 | 182 / 0.133 / 0.082 | 301 / 0.124 / 0.083 |
|
||||
| 0.2–0.3 | 39 / 0.247 / 0.256 | 95 / 0.246 / 0.168 | 36 / 0.242 / 0.056 |
|
||||
| 0.3–0.4 | 54 / 0.340 / 0.037 | 54 / 0.333 / 0.093 | 32 / 0.341 / 0.094 |
|
||||
| 0.4–0.5 | 8 / 0.467 / 0.125 | 25 / 0.461 / 0.160 | 10 / 0.447 / 0.400 |
|
||||
| 0.5–0.6 | 9 / 0.533 / 0.000 | 12 / 0.521 / 0.250 | 8 / 0.554 / 0.375 |
|
||||
| 0.6–0.7 | 1 / 0.657 / 0.000 | 4 / 0.660 / 0.000 | 7 / 0.650 / 0.000 |
|
||||
| 0.7–0.8 | 4 / 0.754 / 0.250 | 2 / 0.730 / 0.500 | 4 / 0.734 / 0.250 |
|
||||
| 0.0–0.1 | 49992 / 0.003 / 0.003 | 49847 / 0.002 / 0.002 | 49630 / 0.002 / 0.002 |
|
||||
| 0.1–0.2 | 143 / 0.132 / 0.161 | 199 / 0.136 / 0.136 | 482 / 0.142 / 0.110 |
|
||||
| 0.2–0.3 | 55 / 0.263 / 0.036 | 108 / 0.250 / 0.157 | 70 / 0.240 / 0.114 |
|
||||
| 0.3–0.4 | 23 / 0.335 / 0.000 | 43 / 0.339 / 0.163 | 34 / 0.356 / 0.176 |
|
||||
| 0.4–0.5 | 8 / 0.453 / 0.250 | 25 / 0.457 / 0.040 | 15 / 0.441 / 0.067 |
|
||||
| 0.5–0.6 | 8 / 0.531 / 0.000 | 8 / 0.522 / 0.125 | — |
|
||||
| 0.6–0.7 | 2 / 0.677 / 0.000 | 2 / 0.626 / 0.000 | 1 / 0.647 / 0.000 |
|
||||
| 0.7–0.8 | 1 / 0.723 / 1.000 | — | — |
|
||||
|
||||
|
||||
Quiet artists (out of rotation everywhere for 26 weeks) releasing, any channel:
|
||||
|
||||
| Group | Events | Adds | Rate | Mean predicted (raw) | Mean predicted (isotonic) | Log-loss (isotonic) | PR-AUC (raw) |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| former favourites (4+ weeks on DR 2019-24) | 326 | 7 | 0.021 | 0.025 | 0.030 | 0.1028 | 0.095 |
|
||||
| other quiet artists | 2616 | 15 | 0.006 | 0.007 | 0.008 | 0.0313 | 0.126 |
|
||||
| former favourites (4+ weeks on DR 2019-24) | 348 | 7 | 0.020 | 0.022 | 0.027 | 0.0930 | 0.104 |
|
||||
| other quiet artists | 2892 | 16 | 0.006 | 0.006 | 0.008 | 0.0299 | 0.126 |
|
||||
|
||||
| Any-channel feature (all events) | Coefficient |
|
||||
|---|---|
|
||||
| intercept | -5.988 |
|
||||
| log DR plays t | +0.677 |
|
||||
| log DR plays 4w | +0.897 |
|
||||
| log DR plays 12w | -0.425 |
|
||||
| log max DR plays 12w | +0.233 |
|
||||
| in rotation anywhere now | -0.575 |
|
||||
| station groups in rotation now (/5) | -0.137 |
|
||||
| current run anywhere (/12) | -1.280 |
|
||||
| weeks since in rotation anywhere (/12) | -0.016 |
|
||||
| Track Top-40 score | +0.603 |
|
||||
| Airplay Top-20 score | -0.700 |
|
||||
| chart entry recency 6w | +0.172 |
|
||||
| chart entry recency 26w | -0.684 |
|
||||
| log Track Top-40 hits | -0.283 |
|
||||
| log Airplay Top-20 hits | -0.147 |
|
||||
| Danish | +0.830 |
|
||||
| group | -0.078 |
|
||||
| female | -0.173 |
|
||||
| career years (/50) | +0.646 |
|
||||
| debut unknown | +0.019 |
|
||||
| intercept | -5.798 |
|
||||
| log DR plays t | +0.782 |
|
||||
| log DR plays 4w | +0.714 |
|
||||
| log DR plays 12w | -0.537 |
|
||||
| log max DR plays 12w | +0.402 |
|
||||
| in rotation anywhere now | -0.644 |
|
||||
| station groups in rotation now (/5) | -0.157 |
|
||||
| current run anywhere (/12) | -1.205 |
|
||||
| weeks since in rotation anywhere (/12) | -0.271 |
|
||||
| Track Top-40 score | +0.756 |
|
||||
| Airplay Top-20 score | -0.521 |
|
||||
| chart entry recency 6w | +0.323 |
|
||||
| chart entry recency 26w | -0.592 |
|
||||
| log Track Top-40 hits | -0.212 |
|
||||
| log Airplay Top-20 hits | -0.186 |
|
||||
| Danish | +0.819 |
|
||||
| group | -0.151 |
|
||||
| female | -0.024 |
|
||||
| career years (/50) | +0.476 |
|
||||
| debut unknown | -0.153 |
|
||||
| Christmas in window | +0.000 |
|
||||
| discography known | -0.000 |
|
||||
| release recency 4w | -0.157 |
|
||||
| release recency 12w | -1.118 |
|
||||
| log releases last 52w | -0.285 |
|
||||
| release recency 4w | -0.270 |
|
||||
| release recency 12w | -1.329 |
|
||||
| log releases last 52w | -0.233 |
|
||||
| releases in the window | -0.000 |
|
||||
| releases in window x not in rotation now | +0.575 |
|
||||
| DR: weeks since last add 13-25 | -0.493 |
|
||||
| DR: last add 26+ weeks ago | -0.099 |
|
||||
| DR: no add in the panel yet | -0.708 |
|
||||
| Airplay Top-20: last new song 13-25 weeks ago | +0.826 |
|
||||
| Airplay Top-20: 0.5-1.25x own usual gap | -0.176 |
|
||||
| Airplay Top-20: last new song 2+ years ago | +0.334 |
|
||||
| log archive plays per 100 sampled hours (2012-21) | +0.618 |
|
||||
| log DR lists with the artist before t (2023-) | +0.156 |
|
||||
| DR list recency 26w | -0.607 |
|
||||
| DR lists known before t at all | -0.210 |
|
||||
| quiet former favourite (4+ weeks on DR 2019-24, out of rotation 26w) | -0.665 |
|
||||
| NMF Denmark in the window | +0.943 |
|
||||
| NMF Denmark top 10 in the window | +0.202 |
|
||||
| log NMF Denmark placements last 52w | +0.251 |
|
||||
| share of releases on NMF Denmark (2 years) | +0.020 |
|
||||
| season index of the window (log, IFPI 2002-24) | -0.049 |
|
||||
| releases in window x not in rotation now | +0.644 |
|
||||
| DR: weeks since last add 13-25 | -0.315 |
|
||||
| DR: last add 26+ weeks ago | -0.155 |
|
||||
| DR: no add in the panel yet | -0.611 |
|
||||
| Airplay Top-20: last new song 13-25 weeks ago | +0.771 |
|
||||
| Airplay Top-20: 0.5-1.25x own usual gap | -0.238 |
|
||||
| Airplay Top-20: last new song 2+ years ago | +0.257 |
|
||||
| log archive plays per 100 sampled hours (2012-21) | +0.550 |
|
||||
| log DR lists with the artist before t (2023-) | +0.194 |
|
||||
| DR list recency 26w | -0.668 |
|
||||
| DR lists known before t at all | -0.266 |
|
||||
| quiet former favourite (4+ weeks on DR 2019-24, out of rotation 26w) | -0.645 |
|
||||
| NMF Denmark in the window | +0.820 |
|
||||
| NMF Denmark top 10 in the window | +0.107 |
|
||||
| log NMF Denmark placements last 52w | +0.131 |
|
||||
| share of releases on NMF Denmark (2 years) | +0.204 |
|
||||
| season index of the window (log, IFPI 2002-24) | -0.066 |
|
||||
|
||||
+21
-15
@@ -4,11 +4,11 @@
|
||||
|
||||
| before the add | adds | share | discography known | release 8w before–1w after | NMF DK 8w before–1w after | release or NMF |
|
||||
|---|---|---|---|---|---|---|
|
||||
| active in the last 26 weeks (back after 8-25 weeks off) | 202 | 27% | 92% | 50% | 20% | 53% |
|
||||
| quiet 26w, former favourite (4+ weeks on DR before) | 108 | 14% | 94% | 59% | 36% | 63% |
|
||||
| quiet 26w, played before but never on | 335 | 45% | 75% | 49% | 15% | 51% |
|
||||
| active in the last 26 weeks (back after 8-25 weeks off) | 202 | 27% | 97% | 56% | 20% | 59% |
|
||||
| quiet 26w, former favourite (4+ weeks on DR before) | 108 | 14% | 97% | 62% | 36% | 64% |
|
||||
| quiet 26w, played before but never on | 335 | 45% | 87% | 58% | 15% | 60% |
|
||||
| quiet 26w, no older data | 24 | 3% | 96% | 46% | 8% | 46% |
|
||||
| too little data to tell | 76 | 10% | 83% | 57% | 13% | 59% |
|
||||
| too little data to tell | 76 | 10% | 92% | 64% | 13% | 67% |
|
||||
|
||||
Examples, former favourites: Søren Huss (2024-04-15), Carpark North (2023-10-30), Maroon 5 (2020-09-14), Michael Bublé (2023-11-27), Nephew (2020-03-30), Calvin Harris (2020-09-07), S.O.A.P. (2023-11-27), KH Marie (2023-05-01), Adele (2020-09-21), Lady Gaga (2021-05-17), Caroline Polachek (2022-12-19), Kellermensch (2022-05-09)
|
||||
|
||||
@@ -17,10 +17,10 @@ Examples, former favourites: Søren Huss (2024-04-15), Carpark North (2023-10-30
|
||||
| before the add | adds | share | discography known | release 8w before–1w after | NMF DK 8w before–1w after | release or NMF |
|
||||
|---|---|---|---|---|---|---|
|
||||
| active in the last 26 weeks (back after 8-25 weeks off) | 28 | 18% | 100% | 50% | 54% | 54% |
|
||||
| quiet 26w, former favourite (4+ weeks on DR before) | 23 | 15% | 100% | 74% | 61% | 83% |
|
||||
| quiet 26w, played before but never on | 42 | 27% | 90% | 67% | 48% | 76% |
|
||||
| quiet 26w, former favourite (4+ weeks on DR before) | 23 | 15% | 100% | 78% | 61% | 83% |
|
||||
| quiet 26w, played before but never on | 42 | 27% | 95% | 71% | 48% | 79% |
|
||||
| quiet 26w, no older data | 5 | 3% | 100% | 60% | 60% | 80% |
|
||||
| too little data to tell | 57 | 37% | 96% | 60% | 51% | 74% |
|
||||
| too little data to tell | 57 | 37% | 96% | 63% | 51% | 75% |
|
||||
|
||||
Examples, former favourites: The Strokes (2026-04-06), Agnes (2026-04-27), Astrid Engberg (2026-08-31), Stromae (2026-06-15), Jungle (2026-04-06), Lowly (2026-08-10), Stormzy (2026-04-27), Maximillian (2026-05-25), Phoebe Bridgers (2026-08-17), KOPS (2026-04-13), Peggy Gou (2026-07-13), beabadoobee (2026-05-04)
|
||||
|
||||
@@ -28,19 +28,25 @@ Examples, former favourites: The Strokes (2026-04-06), Agnes (2026-04-27), Astri
|
||||
|
||||
| period | artist | releases | adds | rate (90% CI) |
|
||||
|---|---|---|---|---|
|
||||
| archive 2019-24 | quiet 26w, former favourite (4+ weeks on DR before) | 387 | 30 | 7.8% (5.8–10.3%) |
|
||||
| archive 2019-24 | quiet 26w, played before but never on | 4815 | 80 | 1.7% (1.4–2.0%) |
|
||||
| archive 2019-24 | quiet 26w, no older data | 709 | 8 | 1.1% (0.6–2.0%) |
|
||||
| current panel 2025-26 | quiet 26w, former favourite (4+ weeks on DR before) | 188 | 8 | 4.3% (2.4–7.4%) |
|
||||
| current panel 2025-26 | quiet 26w, played before but never on | 2689 | 9 | 0.3% (0.2–0.6%) |
|
||||
| current panel 2025-26 | quiet 26w, no older data | 432 | 2 | 0.5% (0.2–1.4%) |
|
||||
| archive 2019-24 | quiet 26w, former favourite (4+ weeks on DR before) | 436 | 33 | 7.6% (5.7–9.9%) |
|
||||
| archive 2019-24 | quiet 26w, played before but never on | 7325 | 97 | 1.3% (1.1–1.6%) |
|
||||
| archive 2019-24 | quiet 26w, no older data | 1193 | 8 | 0.7% (0.4–1.2%) |
|
||||
| current panel 2025-26 | quiet 26w, former favourite (4+ weeks on DR before) | 204 | 8 | 3.9% (2.2–6.8%) |
|
||||
| current panel 2025-26 | quiet 26w, played before but never on | 2987 | 10 | 0.3% (0.2–0.6%) |
|
||||
| current panel 2025-26 | quiet 26w, no older data | 486 | 2 | 0.4% (0.1–1.2%) |
|
||||
|
||||
Discographies (Deezer) known for 7980 of 20924 artists (7980 of the 8786 in today's pool).
|
||||
Discographies (Deezer) known for 13160 of 21222 artists (8581 of the 9195 in today's pool).
|
||||
|
||||
## Adds with a discography but no release nearby (current panel, 58 cases, first 25 read by hand)
|
||||
## Adds with a discography but no release nearby (Deezer only, current panel, 58 cases, first 25 read by hand)
|
||||
|
||||
- **Throwbacks and old songs** (Colby O'Donis, The Calling, Sean Kingston, MAGIC!, Echosmith): back on air with an old hit, no release at all.
|
||||
- **No usable Deezer data** (Den Danske Strygekvartet, Folkeklubben, Christian Ki, L.I.G.A., Nikolaj Vonsild, Tobias Laust): Danish acts with nothing near the add date.
|
||||
- **Added before the release** (Stormzy +4 weeks, Mitski +4 weeks, The Calling +7 weeks): radio plays a single before Deezer lists it. A listing of released music can't see this; it needs announced or radio dates.
|
||||
- **A later track from an album** (Agnes, Iceage, Dave, Rob Thomas): picked up 2–4 months after the release.
|
||||
- One junk artist ("Ukendt Kunstner").
|
||||
|
||||
## With iTunes release dates (2026-11-19)
|
||||
|
||||
Release dates now merge Deezer and iTunes (and a little Spotify); dates within 3 days count as one release. Discographies are known for 8,581 of the 9,195 artists in today's pool (was 7,980 of 8,786).
|
||||
- Adds with a release or NMF placement nearby went up 3-9 points (archive "played before but never on": 51% -> 60%; current former favourites: 83%). Most adds without one are genuine: throwbacks, radio adding a single before release, album tracks picked up later.
|
||||
- The former-favourite effect holds: after a release, quiet former favourites are added 7.6% (archive) and 3.9% (current) of the time, against 1.3% and 0.3% for other quiet artists.
|
||||
|
||||
Reference in New Issue
Block a user