Old DR artist pages from the Wayback Machine (53 weeks of plays per capture, 2019-2024) into dr_artist_history; leak fixes (weeks since last add counts only adds up to t-1; training windows close a week before the origin) and a leak test (model leakcheck) that rebuilds every feature from data cut at the origin; backtests split pre-release (NMF from history) and post-release (NMF known); site shows the new-song backtest

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-16 00:22:55 +00:00
co-authored by Claude Opus 5.5
parent 7a16eb45ef
commit 94abfa0255
9 changed files with 512 additions and 223 deletions
+146 -137
View File
@@ -4,181 +4,190 @@ An add = the artist enters rotation (plays ≥ τ) after at least 4 weeks out of
## Window starting h = 0 weeks ahead (17 forecast origins)
Base rates: per channel 0.32% of artist-channels get an add in the window; on any channel 1.8% of artists.
Base rates: per channel 0.43% of artist-channels get an add in the window; on any channel 2.4% of artists.
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|---|---|---|---|---|
| rate by in-rotation-now | 0.0214 | 0.006 | 0.0913 | 0.021 |
| rate by last-12-weeks level | 0.0147 | 0.093 | 0.0781 | 0.072 |
| logit | 0.0119 | 0.229 | 0.0679 | 0.204 |
| rate by in-rotation-now | 0.0255 | 0.032 | 0.1058 | 0.067 |
| rate by last-12-weeks level | 0.0177 | 0.147 | 0.0909 | 0.133 |
| logit (pre-release: NMF from history) | 0.0147 | 0.258 | 0.0800 | 0.251 |
| logit (post-release: NMF known) | 0.0145 | 0.266 | 0.0799 | 0.259 |
Skill of the logit (90% block-bootstrap interval over origins):
| | vs in-rotation-now | vs last-12-weeks level |
|---|---|---|
| per channel | 0.441 (0.427 to 0.451) | 0.187 (0.174 to 0.191) |
| any channel | 0.256 (0.238 to 0.275) | 0.130 (0.120 to 0.141) |
| per channel | 0.424 (0.413 to 0.437) | 0.171 (0.168 to 0.178) |
| any channel | 0.244 (0.234 to 0.257) | 0.120 (0.115 to 0.127) |
Calibration, per channel:
| Predicted | Rows | Mean predicted | Observed |
|---|---|---|---|
| 0.0–0.1 | 1876920 | 0.001 | 0.002 |
| 0.1–0.2 | 6259 | 0.142 | 0.211 |
| 0.2–0.3 | 2324 | 0.243 | 0.272 |
| 0.3–0.4 | 933 | 0.342 | 0.279 |
| 0.4–0.5 | 710 | 0.450 | 0.304 |
| 0.5–0.6 | 613 | 0.550 | 0.303 |
| 0.6–0.7 | 445 | 0.647 | 0.353 |
| 0.7–0.8 | 283 | 0.748 | 0.502 |
| 0.8–0.9 | 237 | 0.844 | 0.422 |
| 0.9–1.0 | 44 | 0.921 | 0.545 |
| 0.0–0.1 | 1868823 | 0.002 | 0.002 |
| 0.1–0.2 | 9096 | 0.142 | 0.160 |
| 0.2–0.3 | 3991 | 0.247 | 0.230 |
| 0.3–0.4 | 2858 | 0.344 | 0.215 |
| 0.4–0.5 | 1615 | 0.448 | 0.310 |
| 0.5–0.6 | 1081 | 0.544 | 0.457 |
| 0.6–0.7 | 728 | 0.646 | 0.475 |
| 0.7–0.8 | 429 | 0.749 | 0.441 |
| 0.8–0.9 | 104 | 0.834 | 0.346 |
| 0.9–1.0 | 43 | 0.917 | 0.930 |
## Window starting h = 4 weeks ahead (13 forecast origins)
Base rates: per channel 0.33% of artist-channels get an add in the window; on any channel 1.8% of artists.
Base rates: per channel 0.44% of artist-channels get an add in the window; on any channel 2.4% of artists.
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|---|---|---|---|---|
| rate by in-rotation-now | 0.0213 | 0.014 | 0.0896 | 0.031 |
| rate by last-12-weeks level | 0.0158 | 0.084 | 0.0801 | 0.066 |
| logit | 0.0147 | 0.134 | 0.0744 | 0.143 |
| rate by in-rotation-now | 0.0263 | 0.032 | 0.1051 | 0.083 |
| rate by last-12-weeks level | 0.0193 | 0.114 | 0.0937 | 0.136 |
| logit (pre-release: NMF from history) | 0.0176 | 0.166 | 0.0857 | 0.208 |
| logit (post-release: NMF known) | 0.0172 | 0.175 | 0.0856 | 0.218 |
Skill of the logit (90% block-bootstrap interval over origins):
| | vs in-rotation-now | vs last-12-weeks level |
|---|---|---|
| per channel | 0.310 (0.302 to 0.316) | 0.070 (0.042 to 0.087) |
| any channel | 0.170 (0.152 to 0.186) | 0.071 (0.064 to 0.080) |
| per channel | 0.330 (0.315 to 0.337) | 0.091 (0.065 to 0.106) |
| any channel | 0.185 (0.174 to 0.194) | 0.086 (0.081 to 0.090) |
Calibration, per channel:
| Predicted | Rows | Mean predicted | Observed |
|---|---|---|---|
| 0.0–0.1 | 1434699 | 0.002 | 0.002 |
| 0.1–0.2 | 5169 | 0.142 | 0.147 |
| 0.2–0.3 | 2108 | 0.243 | 0.164 |
| 0.3–0.4 | 980 | 0.344 | 0.198 |
| 0.4–0.5 | 621 | 0.446 | 0.203 |
| 0.5–0.6 | 404 | 0.544 | 0.223 |
| 0.6–0.7 | 255 | 0.644 | 0.475 |
| 0.7–0.8 | 74 | 0.751 | 0.203 |
| 0.8–0.9 | 42 | 0.837 | 0.310 |
| 0.0–0.1 | 1428760 | 0.002 | 0.003 |
| 0.1–0.2 | 7389 | 0.141 | 0.129 |
| 0.2–0.3 | 3960 | 0.247 | 0.186 |
| 0.3–0.4 | 2024 | 0.344 | 0.199 |
| 0.4–0.5 | 1134 | 0.444 | 0.283 |
| 0.5–0.6 | 604 | 0.546 | 0.224 |
| 0.6–0.7 | 394 | 0.643 | 0.459 |
| 0.7–0.8 | 83 | 0.732 | 0.277 |
| 0.8–0.9 | 3 | 0.833 | 1.000 |
| 0.9–1.0 | 1 | 0.907 | 1.000 |
## Coefficients, window starting now (h = 0), all closed windows
| Any-channel feature | Coefficient |
|---|---|
| intercept | -4.804 |
| log DR plays t | +0.988 |
| log DR plays 4w | +0.335 |
| log DR plays 12w | -0.407 |
| log max DR plays 12w | +0.696 |
| in rotation anywhere now | -1.287 |
| station groups in rotation now (/5) | -3.848 |
| current run anywhere (/12) | -1.865 |
| weeks since in rotation anywhere (/12) | -0.272 |
| Track Top-40 score | +0.074 |
| Airplay Top-20 score | -0.687 |
| chart entry recency 6w | +1.656 |
| chart entry recency 26w | -1.565 |
| log Track Top-40 hits | -0.209 |
| log Airplay Top-20 hits | -0.197 |
| Danish | +0.499 |
| group | -0.341 |
| female | -0.140 |
| career years (/50) | +0.297 |
| debut unknown | +0.622 |
| intercept | -4.213 |
| log DR plays t | +0.883 |
| log DR plays 4w | +0.253 |
| log DR plays 12w | -0.263 |
| log max DR plays 12w | +0.594 |
| in rotation anywhere now | -1.148 |
| station groups in rotation now (/5) | +0.297 |
| current run anywhere (/12) | -1.212 |
| weeks since in rotation anywhere (/12) | -0.749 |
| Track Top-40 score | +0.427 |
| Airplay Top-20 score | -1.332 |
| chart entry recency 6w | +1.210 |
| chart entry recency 26w | -0.653 |
| log Track Top-40 hits | -0.200 |
| log Airplay Top-20 hits | -0.069 |
| Danish | +0.462 |
| group | -0.240 |
| female | -0.102 |
| career years (/50) | +0.210 |
| debut unknown | +0.497 |
| Christmas in window | +0.000 |
| discography known | -0.354 |
| release recency 4w | +0.199 |
| release recency 12w | +0.894 |
| log releases last 52w | -0.200 |
| releases in the window | -1.019 |
| releases in window x not in rotation now | +1.573 |
| DR: weeks since last add 13-25 | +0.099 |
| DR: last add 26+ weeks ago | -0.438 |
| DR: no add in the panel yet | -0.948 |
| Airplay Top-20: last new song 13-25 weeks ago | +0.299 |
| Airplay Top-20: 0.5-1.25x own usual gap | -0.018 |
| discography known | -0.325 |
| release recency 4w | +0.440 |
| release recency 12w | +0.670 |
| log releases last 52w | -0.188 |
| releases in the window | -0.218 |
| releases in window x not in rotation now | +0.854 |
| DR: weeks since last add 13-25 | +0.095 |
| DR: last add 26+ weeks ago | -0.642 |
| DR: no add in the panel yet | -0.847 |
| Airplay Top-20: last new song 13-25 weeks ago | +0.279 |
| Airplay Top-20: 0.5-1.25x own usual gap | +0.231 |
| Airplay Top-20: last new song 2+ years ago | -0.529 |
| log archive plays per 100 sampled hours (2012-21) | +0.366 |
| log DR lists with the artist before t (2023-) | +0.332 |
| DR list recency 26w | -0.621 |
| DR lists known before t at all | -0.166 |
| NMF Denmark in the window | +0.985 |
| NMF Denmark top 10 in the window | +0.444 |
| log NMF Denmark placements last 52w | -0.079 |
| share of releases on NMF Denmark (2 years) | -0.623 |
| log archive plays per 100 sampled hours (2012-21) | +0.288 |
| log DR lists with the artist before t (2023-) | +0.209 |
| DR list recency 26w | -0.929 |
| DR lists known before t at all | +0.040 |
| NMF Denmark in the window | +0.941 |
| NMF Denmark top 10 in the window | +0.416 |
| log NMF Denmark placements last 52w | -0.071 |
| share of releases on NMF Denmark (2 years) | -0.415 |
| season index of the window (log, IFPI 2002-24) | +0.403 |
| Per-channel feature | Coefficient |
|---|---|
| intercept | -7.632 |
| log plays t | +0.821 |
| log plays t-1 | +0.119 |
| log mean plays 4w | -0.008 |
| log mean plays 12w | -0.531 |
| log max plays 12w | +1.318 |
| log other channels t | +0.235 |
| log other channels 4w | -0.087 |
| log P3 t | +0.037 |
| weeks since played (/12) | -1.762 |
| Track Top-40 score | -0.273 |
| Airplay Top-20 score | -0.757 |
| chart entry recency 6w | +0.747 |
| chart entry recency 26w | -0.459 |
| log Track Top-40 hits | +0.226 |
| log Airplay Top-20 hits | -0.351 |
| Danish | +0.235 |
| group | -0.650 |
| female | +0.012 |
| career years (/50) | -1.128 |
| debut unknown | -0.056 |
| intercept | -7.101 |
| log plays t | +1.122 |
| log plays t-1 | +0.142 |
| log mean plays 4w | +0.113 |
| log mean plays 12w | -0.862 |
| log max plays 12w | +1.014 |
| log other channels t | +0.332 |
| log other channels 4w | -0.100 |
| log P3 t | +0.085 |
| weeks since played (/12) | -0.437 |
| Track Top-40 score | -0.127 |
| Airplay Top-20 score | -1.373 |
| chart entry recency 6w | +0.625 |
| chart entry recency 26w | -0.512 |
| log Track Top-40 hits | +0.257 |
| log Airplay Top-20 hits | -0.395 |
| Danish | +0.162 |
| group | -0.629 |
| female | +0.127 |
| career years (/50) | -1.500 |
| debut unknown | -0.289 |
| Christmas target week | +0.000 |
| Danish x P3 | -0.051 |
| Danish x P4 | +0.302 |
| Danish x P6 | +0.004 |
| ch p4kbh | +0.695 |
| ch p4sjaelland | +0.514 |
| ch p4fyn | +0.499 |
| ch p4syd | +0.544 |
| ch p4trekanten | +0.581 |
| ch p4vest | +0.611 |
| ch p4aarhus | +0.526 |
| ch p4nord | +0.587 |
| ch p4bornholm | +0.536 |
| ch p4esbjerg | +0.555 |
| ch p5 | +1.820 |
| ch p6beat | +1.118 |
| ch p8jazz | +2.402 |
| in rotation on channel now | -0.952 |
| in rotation anywhere now | +0.118 |
| station groups in rotation now (/5) | -1.127 |
| run on channel = 1 week | +0.893 |
| run = 2 weeks | +0.413 |
| run = 3-5 weeks | -1.287 |
| run >= 6 weeks | -0.971 |
| near threshold (>= half of tau, not in rotation) | +1.421 |
| plays / tau (capped 3, /3) | -0.967 |
| weeks since in rotation on channel (/12) | +0.425 |
| discography known | -0.042 |
| release recency 4w | +0.840 |
| release recency 12w | +0.180 |
| log releases last 52w | -0.059 |
| releases in the window | -0.478 |
| releases in window x not in rotation now | +1.007 |
| DR: weeks since last add 13-25 | +0.007 |
| DR: last add 26+ weeks ago | +0.310 |
| DR: no add in the panel yet | -0.072 |
| Airplay Top-20: last new song 13-25 weeks ago | +0.084 |
| Airplay Top-20: 0.5-1.25x own usual gap | +0.103 |
| Airplay Top-20: last new song 2+ years ago | -0.240 |
| log archive plays per 100 sampled hours (2012-21) | +0.179 |
| log DR lists with the artist before t (2023-) | +0.175 |
| DR list recency 26w | -0.254 |
| DR lists known before t at all | -0.234 |
| NMF Denmark in the window | +0.853 |
| NMF Denmark top 10 in the window | +0.479 |
| log NMF Denmark placements last 52w | -0.122 |
| share of releases on NMF Denmark (2 years) | -0.189 |
| Danish x P3 | +0.072 |
| Danish x P4 | +0.085 |
| Danish x P6 | +0.154 |
| ch p4kbh | +1.100 |
| ch p4sjaelland | +0.968 |
| ch p4fyn | +0.948 |
| ch p4syd | +0.976 |
| ch p4trekanten | +0.992 |
| ch p4vest | +1.078 |
| ch p4aarhus | +0.934 |
| ch p4nord | +1.002 |
| ch p4bornholm | +0.986 |
| ch p4esbjerg | +0.968 |
| ch p5 | +2.081 |
| ch p6beat | +1.295 |
| ch p8jazz | +2.330 |
| in rotation on channel now | +0.358 |
| in rotation anywhere now | -0.338 |
| station groups in rotation now (/5) | +0.043 |
| run on channel = 1 week | +0.663 |
| run = 2 weeks | +0.441 |
| run = 3-5 weeks | -0.055 |
| run >= 6 weeks | -0.691 |
| near threshold (>= half of tau, not in rotation) | +1.376 |
| plays / tau (capped 3, /3) | -3.407 |
| weeks since in rotation on channel (/12) | -0.300 |
| discography known | -0.069 |
| release recency 4w | +1.033 |
| release recency 12w | -0.008 |
| log releases last 52w | -0.003 |
| releases in the window | -0.143 |
| releases in window x not in rotation now | +0.782 |
| DR: weeks since last add 13-25 | +0.107 |
| DR: last add 26+ weeks ago | -0.085 |
| DR: no add in the panel yet | -0.486 |
| Airplay Top-20: last new song 13-25 weeks ago | +0.194 |
| Airplay Top-20: 0.5-1.25x own usual gap | -0.009 |
| Airplay Top-20: last new song 2+ years ago | -0.417 |
| log archive plays per 100 sampled hours (2012-21) | +0.348 |
| log DR lists with the artist before t (2023-) | +0.028 |
| DR list recency 26w | -0.444 |
| DR lists known before t at all | +0.055 |
| NMF Denmark in the window | +0.805 |
| NMF Denmark top 10 in the window | +0.579 |
| log NMF Denmark placements last 52w | -0.077 |
| share of releases on NMF Denmark (2 years) | +0.166 |
| season index of the window (log, IFPI 2002-24) | +1.240 |
| share of the artist's DR plays on this station (before t) | +0.963 |
| log DR plays on this station (before t) | +0.049 |
| never played on this station (before t) | -2.888 |
| genre fit for this station | +0.108 |
+46 -45
View File
@@ -4,53 +4,54 @@ Events: an artist in the panel releases something (Deezer) in week t0+1, with no
| Model | Log-loss, any channel | PR-AUC, any channel | Log-loss, per channel | PR-AUC, per channel |
|---|---|---|---|---|
| rate by in-rotation-now | 0.2073 | 0.118 | 0.0666 | 0.052 |
| rate by last-12-weeks level | 0.1831 | 0.201 | 0.0492 | 0.190 |
| event model | 0.1803 | 0.258 | 0.0436 | 0.320 |
| rate by in-rotation-now | 0.2073 | 0.115 | 0.0666 | 0.059 |
| rate by last-12-weeks level | 0.1838 | 0.215 | 0.0491 | 0.202 |
| event model (pre-release: NMF from history) | 0.1854 | 0.248 | 0.0445 | 0.314 |
| event model (post-release: NMF known) | 0.1814 | 0.280 | 0.0422 | 0.325 |
Skill of the event model, any channel: 0.130 (0.085 to 0.187) vs in-rotation-now, 0.015 (-0.036 to 0.061) vs last-12-weeks level.
Skill of the event model, any channel: 0.106 (0.081 to 0.178) vs in-rotation-now, -0.008 (-0.051 to 0.068) vs last-12-weeks level.
| Any-channel feature (all events) | Coefficient |
|---|---|
| intercept | -3.676 |
| log DR plays t | +0.461 |
| log DR plays 4w | +0.667 |
| log DR plays 12w | -0.467 |
| log max DR plays 12w | +0.260 |
| in rotation anywhere now | -0.464 |
| station groups in rotation now (/5) | -0.219 |
| current run anywhere (/12) | -1.217 |
| weeks since in rotation anywhere (/12) | -0.542 |
| Track Top-40 score | -0.339 |
| Airplay Top-20 score | -0.125 |
| chart entry recency 6w | +0.509 |
| chart entry recency 26w | +0.377 |
| log Track Top-40 hits | -0.310 |
| log Airplay Top-20 hits | -0.180 |
| Danish | +0.640 |
| group | +0.038 |
| female | -0.081 |
| career years (/50) | +0.505 |
| debut unknown | +0.552 |
| intercept | -3.665 |
| log DR plays t | +0.557 |
| log DR plays 4w | +0.507 |
| log DR plays 12w | -0.363 |
| log max DR plays 12w | +0.249 |
| in rotation anywhere now | -0.476 |
| station groups in rotation now (/5) | -0.252 |
| current run anywhere (/12) | -1.197 |
| weeks since in rotation anywhere (/12) | -0.514 |
| Track Top-40 score | -0.282 |
| Airplay Top-20 score | -0.313 |
| chart entry recency 6w | +0.713 |
| chart entry recency 26w | +0.477 |
| log Track Top-40 hits | -0.303 |
| log Airplay Top-20 hits | -0.118 |
| Danish | +0.580 |
| group | +0.043 |
| female | -0.064 |
| career years (/50) | +0.433 |
| debut unknown | +0.530 |
| Christmas in window | +0.000 |
| discography known | -0.000 |
| release recency 4w | +0.167 |
| release recency 12w | -0.863 |
| log releases last 52w | -0.208 |
| releases in the window | -0.000 |
| releases in window x not in rotation now | +0.464 |
| DR: weeks since last add 13-25 | -0.027 |
| DR: last add 26+ weeks ago | +0.411 |
| DR: no add in the panel yet | -0.679 |
| Airplay Top-20: last new song 13-25 weeks ago | +0.478 |
| Airplay Top-20: 0.5-1.25x own usual gap | -0.177 |
| Airplay Top-20: last new song 2+ years ago | -0.035 |
| log archive plays per 100 sampled hours (2012-21) | +0.583 |
| log DR lists with the artist before t (2023-) | +0.192 |
| DR list recency 26w | -0.900 |
| DR lists known before t at all | +0.491 |
| NMF Denmark in the window | +0.886 |
| NMF Denmark top 10 in the window | +0.810 |
| log NMF Denmark placements last 52w | -0.024 |
| share of releases on NMF Denmark (2 years) | -0.468 |
| season index of the window (log, IFPI 2002-24) | +0.414 |
| discography known | +0.000 |
| release recency 4w | +0.243 |
| release recency 12w | -0.814 |
| log releases last 52w | -0.224 |
| releases in the window | +0.000 |
| releases in window x not in rotation now | +0.476 |
| DR: weeks since last add 13-25 | -0.054 |
| DR: last add 26+ weeks ago | +0.497 |
| DR: no add in the panel yet | -0.684 |
| Airplay Top-20: last new song 13-25 weeks ago | +0.463 |
| Airplay Top-20: 0.5-1.25x own usual gap | -0.144 |
| Airplay Top-20: last new song 2+ years ago | -0.107 |
| log archive plays per 100 sampled hours (2012-21) | +0.532 |
| log DR lists with the artist before t (2023-) | +0.108 |
| DR list recency 26w | -0.955 |
| DR lists known before t at all | +0.572 |
| NMF Denmark in the window | +0.866 |
| NMF Denmark top 10 in the window | +0.788 |
| log NMF Denmark placements last 52w | -0.047 |
| share of releases on NMF Denmark (2 years) | -0.354 |
| season index of the window (log, IFPI 2002-24) | +0.372 |