Rotation thresholds fitted to DR's own lists (P3 23, P4 ~14, P6 16 plays/week: ~3x the old guess); IFPI 2010-2026 walk-forward test bed and route/lead-lag analysis; site and docs updated
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,142 @@
|
||||
# Backtest: new adds
|
||||
|
||||
An add = the artist enters rotation (plays ≥ τ) after at least 4 weeks out of rotation: per channel, and on any DR channel after 4 weeks out of rotation everywhere. Target: an add within the 8 weeks that start h weeks after the forecast origin. Origins from week 27; each fit uses only windows closed by its origin.
|
||||
|
||||
## Window starting h = 0 weeks ahead (17 forecast origins)
|
||||
|
||||
Base rates: per channel 0.75% of artist-channels get an add in the window; on any channel 4.2% of artists.
|
||||
|
||||
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|
||||
|---|---|---|---|---|
|
||||
| rate by in-rotation-now | 0.0440 | 0.012 | 0.1758 | 0.042 |
|
||||
| rate by last-12-weeks level | 0.0328 | 0.099 | 0.1656 | 0.083 |
|
||||
| logit | 0.0273 | 0.230 | 0.1563 | 0.156 |
|
||||
|
||||
Skill of the logit (90% block-bootstrap interval over origins):
|
||||
|
||||
| | vs in-rotation-now | vs last-12-weeks level |
|
||||
|---|---|---|
|
||||
| per channel | 0.381 (0.361 to 0.396) | 0.168 (0.148 to 0.181) |
|
||||
| any channel | 0.111 (0.099 to 0.120) | 0.056 (0.040 to 0.068) |
|
||||
|
||||
Calibration, per channel:
|
||||
|
||||
| Predicted | Rows | Mean predicted | Observed |
|
||||
|---|---|---|---|
|
||||
| 0.0–0.1 | 760136 | 0.003 | 0.004 |
|
||||
| 0.1–0.2 | 4598 | 0.148 | 0.180 |
|
||||
| 0.2–0.3 | 3797 | 0.246 | 0.219 |
|
||||
| 0.3–0.4 | 1815 | 0.345 | 0.297 |
|
||||
| 0.4–0.5 | 1080 | 0.444 | 0.359 |
|
||||
| 0.5–0.6 | 715 | 0.548 | 0.365 |
|
||||
| 0.6–0.7 | 252 | 0.643 | 0.476 |
|
||||
| 0.7–0.8 | 149 | 0.742 | 0.470 |
|
||||
| 0.8–0.9 | 6 | 0.809 | 0.000 |
|
||||
|
||||
## Window starting h = 4 weeks ahead (13 forecast origins)
|
||||
|
||||
Base rates: per channel 0.77% of artist-channels get an add in the window; on any channel 4.3% of artists.
|
||||
|
||||
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|
||||
|---|---|---|---|---|
|
||||
| rate by in-rotation-now | 0.0438 | 0.019 | 0.1752 | 0.053 |
|
||||
| rate by last-12-weeks level | 0.0347 | 0.091 | 0.1689 | 0.078 |
|
||||
| logit | 0.0326 | 0.139 | 0.1629 | 0.127 |
|
||||
|
||||
Skill of the logit (90% block-bootstrap interval over origins):
|
||||
|
||||
| | vs in-rotation-now | vs last-12-weeks level |
|
||||
|---|---|---|
|
||||
| per channel | 0.257 (0.242 to 0.276) | 0.062 (0.043 to 0.077) |
|
||||
| any channel | 0.070 (0.063 to 0.075) | 0.035 (0.020 to 0.046) |
|
||||
|
||||
Calibration, per channel:
|
||||
|
||||
| Predicted | Rows | Mean predicted | Observed |
|
||||
|---|---|---|---|
|
||||
| 0.0–0.1 | 579503 | 0.004 | 0.005 |
|
||||
| 0.1–0.2 | 6709 | 0.143 | 0.118 |
|
||||
| 0.2–0.3 | 2781 | 0.243 | 0.193 |
|
||||
| 0.3–0.4 | 1213 | 0.341 | 0.232 |
|
||||
| 0.4–0.5 | 375 | 0.442 | 0.291 |
|
||||
| 0.5–0.6 | 131 | 0.542 | 0.198 |
|
||||
| 0.6–0.7 | 42 | 0.637 | 0.571 |
|
||||
| 0.7–0.8 | 18 | 0.721 | 0.667 |
|
||||
|
||||
## Coefficients, window starting now (h = 0), all closed windows
|
||||
|
||||
| Any-channel feature | Coefficient |
|
||||
|---|---|
|
||||
| intercept | -1.503 |
|
||||
| log DR plays t | +0.669 |
|
||||
| log DR plays 4w | +0.119 |
|
||||
| log DR plays 12w | -0.428 |
|
||||
| log max DR plays 12w | +0.140 |
|
||||
| in rotation anywhere now | -2.146 |
|
||||
| channels in rotation now (/14) | -2.288 |
|
||||
| current run anywhere (/12) | -1.783 |
|
||||
| weeks since in rotation anywhere (/12) | -2.397 |
|
||||
| Track Top-40 score | +0.248 |
|
||||
| Airplay Top-20 score | +0.210 |
|
||||
| chart entry recency 6w | +0.482 |
|
||||
| chart entry recency 26w | -0.328 |
|
||||
| log Track Top-40 hits | +0.002 |
|
||||
| log Airplay Top-20 hits | -0.280 |
|
||||
| Danish | +0.500 |
|
||||
| group | -0.468 |
|
||||
| female | -0.142 |
|
||||
| career years (/50) | -0.250 |
|
||||
| debut unknown | +0.129 |
|
||||
| Christmas in window | +0.000 |
|
||||
|
||||
| Per-channel feature | Coefficient |
|
||||
|---|---|
|
||||
| intercept | -6.792 |
|
||||
| log plays t | +0.665 |
|
||||
| log plays t-1 | +0.129 |
|
||||
| log mean plays 4w | +0.169 |
|
||||
| log mean plays 12w | -0.662 |
|
||||
| log max plays 12w | +1.391 |
|
||||
| log other channels t | +0.257 |
|
||||
| log other channels 4w | -0.214 |
|
||||
| log P3 t | +0.174 |
|
||||
| weeks since played (/12) | -0.733 |
|
||||
| Track Top-40 score | -0.563 |
|
||||
| Airplay Top-20 score | -0.416 |
|
||||
| chart entry recency 6w | +0.160 |
|
||||
| chart entry recency 26w | +0.288 |
|
||||
| log Track Top-40 hits | +0.260 |
|
||||
| log Airplay Top-20 hits | -0.324 |
|
||||
| Danish | +0.295 |
|
||||
| group | -0.840 |
|
||||
| female | +0.007 |
|
||||
| career years (/50) | -1.415 |
|
||||
| debut unknown | -0.130 |
|
||||
| Christmas target week | +0.000 |
|
||||
| Danish x P3 | -0.176 |
|
||||
| Danish x P4 | +0.154 |
|
||||
| Danish x P6 | -0.213 |
|
||||
| ch p4kbh | +0.236 |
|
||||
| ch p4sjaelland | +0.256 |
|
||||
| ch p4fyn | +0.235 |
|
||||
| ch p4syd | +0.297 |
|
||||
| ch p4trekanten | +0.272 |
|
||||
| ch p4vest | +0.361 |
|
||||
| ch p4aarhus | +0.297 |
|
||||
| ch p4nord | +0.186 |
|
||||
| ch p4bornholm | +0.278 |
|
||||
| ch p4esbjerg | +0.290 |
|
||||
| ch p5 | +1.468 |
|
||||
| ch p6beat | +0.871 |
|
||||
| ch p8jazz | +1.960 |
|
||||
| in rotation on channel now | -1.352 |
|
||||
| in rotation anywhere now | +0.104 |
|
||||
| channels in rotation now (/14) | -0.674 |
|
||||
| run on channel = 1 week | +0.944 |
|
||||
| run = 2 weeks | +0.352 |
|
||||
| run = 3-5 weeks | -1.570 |
|
||||
| run >= 6 weeks | -1.078 |
|
||||
| near threshold (>= half of tau, not in rotation) | +1.389 |
|
||||
| plays / tau (capped 3, /3) | -0.664 |
|
||||
| weeks since in rotation on channel (/12) | +0.421 |
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
# Rotation thresholds from DR's lists
|
||||
|
||||
Lists inside the DR panel: weeks starting 2025-11-24, 2026-03-16, 2026-04-13, 2026-05-18, 2026-09-14, 2026-11-02. List lines in those weeks: 1094; artist not among the DR artists read: 167.
|
||||
|
||||
| Channel | τ (best F1) | F1 | Listed artist-weeks | Median plays A | B | C | Median plays, not listed but played | Old τ |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| p3 | 23 | 0.72 | 226 | 42 | 29 | 16 | 3 | 7 |
|
||||
| p4kbh | 15 | 0.74 | 270 | 26 | 23 | 15 | 2 | 4 |
|
||||
| p4sjaelland | 14 | 0.74 | 270 | 26 | 23 | 15 | 2 | 4 |
|
||||
| p4fyn | 14 | 0.74 | 270 | 25 | 23 | 15 | 2 | 4 |
|
||||
| p4syd | 14 | 0.74 | 270 | 26 | 24 | 15 | 2 | 4 |
|
||||
| p4trekanten | 14 | 0.74 | 270 | 26 | 24 | 14 | 2 | 4 |
|
||||
| p4vest | 13 | 0.75 | 270 | 26 | 24 | 15 | 2 | 4 |
|
||||
| p4aarhus | 14 | 0.75 | 270 | 26 | 23 | 15 | 2 | 4 |
|
||||
| p4nord | 15 | 0.74 | 270 | 25 | 24 | 15 | 2 | 4 |
|
||||
| p4bornholm | 14 | 0.74 | 270 | 26 | 23 | 14 | 2 | 4 |
|
||||
| p4esbjerg | 14 | 0.74 | 270 | 26 | 24 | 15 | 2 | 4 |
|
||||
| p5 | 10 | 0.48 | 108 | 24 | 11 | 11 | 2 | 4 |
|
||||
| p6beat | 16 | 0.78 | 201 | 39 | 23 | 14 | 2 | 5 |
|
||||
| p8jazz | 7 | 0.53 | 250 | 14 | 9 | 6 | 2 | 3 |
|
||||
|
||||
The models now read these thresholds (table `rotation_thresholds`). Listed artist-weeks count an artist once per week even with several songs listed; plays are the artist's total that week, including off-list plays.
|
||||
|
||||
@@ -0,0 +1,55 @@
|
||||
# IFPI test bed
|
||||
|
||||
Track Top-40 from 2007-01-01, Airplay Top-20 since 2001; latest chart week 2026-10-26. Songs matched across the charts by primary artist + title without version tags.
|
||||
|
||||
## Routes: which chart a song reached first
|
||||
|
||||
Songs on both charts whose first appearance on either was after the Track Top-40 began. Lag = first Airplay Top-20 week − first Track Top-40 week.
|
||||
|
||||
| Period (of the first appearance) | Songs on both | Charts first (lag ≥ 2) | Same time (±1 week) | Radio first (lag ≤ −2) | Median lag, charts first | Median lag, radio first |
|
||||
|---|---|---|---|---|---|---|
|
||||
| 2007 (sales only) | 59 | 15% | 46% | 39% | 4 wk | 8 wk |
|
||||
| Nov 2007–Oct 2014 (sales + downloads) | 532 | 43% | 43% | 13% | 6 wk | 4 wk |
|
||||
| Nov 2014– (streaming counted) | 614 | 55% | 29% | 17% | 5 wk | 6 wk |
|
||||
|
||||
Airplay Top-20 songs never on the Track Top-40 (radio only): 2007 (sales only): 40; Nov 2007–Oct 2014 (sales + downloads): 441; Nov 2014– (streaming counted): 837.
|
||||
|
||||
## Walk-forward: does a Track Top-40 entry reach the Airplay Top-20 within 8 weeks?
|
||||
|
||||
Candidates: songs entering the Track Top-40 that hadn't been on the Airplay Top-20. Each year is scored with models fitted on the years before it.
|
||||
|
||||
| Year | Entries | Reached airplay | Log-loss base rate | Log-loss entry position | Log-loss full | Skill full vs base | Skill full vs position |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| 2010 | 202 | 29% | 0.615 | 0.599 | 0.540 | +0.121 | +0.097 |
|
||||
| 2011 | 245 | 25% | 0.566 | 0.566 | 0.573 | -0.012 | -0.011 |
|
||||
| 2012 | 239 | 14% | 0.440 | 0.441 | 0.358 | +0.186 | +0.188 |
|
||||
| 2013 | 226 | 21% | 0.513 | 0.497 | 0.459 | +0.105 | +0.077 |
|
||||
| 2014 | 250 | 30% | 0.620 | 0.587 | 0.598 | +0.035 | -0.020 |
|
||||
| 2015 | 136 | 32% | 0.650 | 0.617 | 0.696 | -0.070 | -0.128 |
|
||||
| 2016 | 147 | 31% | 0.634 | 0.624 | 0.578 | +0.088 | +0.073 |
|
||||
| 2017 | 209 | 22% | 0.534 | 0.514 | 0.469 | +0.122 | +0.088 |
|
||||
| 2018 | 257 | 14% | 0.439 | 0.454 | 0.441 | -0.004 | +0.029 |
|
||||
| 2019 | 267 | 16% | 0.453 | 0.452 | 0.374 | +0.175 | +0.173 |
|
||||
| 2020 | 215 | 11% | 0.394 | 0.393 | 0.260 | +0.340 | +0.338 |
|
||||
| 2021 | 220 | 13% | 0.414 | 0.402 | 0.298 | +0.280 | +0.258 |
|
||||
| 2022 | 204 | 19% | 0.489 | 0.473 | 0.427 | +0.128 | +0.097 |
|
||||
| 2023 | 195 | 13% | 0.413 | 0.424 | 0.324 | +0.214 | +0.236 |
|
||||
| 2024 | 211 | 12% | 0.397 | 0.404 | 0.331 | +0.166 | +0.180 |
|
||||
| 2025 | 204 | 9% | 0.347 | 0.357 | 0.239 | +0.309 | +0.329 |
|
||||
| 2026 | 157 | 15% | 0.434 | 0.431 | 0.345 | +0.206 | +0.201 |
|
||||
|
||||
All years: skill of the full model +0.127 vs base rate, +0.115 vs entry position only. PR-AUC: base 0.200, position 0.271, full 0.411.
|
||||
|
||||
| Feature (all years) | Coefficient |
|
||||
|---|---|
|
||||
| intercept | -2.463 |
|
||||
| entry position score | +0.215 |
|
||||
| entry score² | +1.482 |
|
||||
| log earlier Airplay Top-20 songs | +1.097 |
|
||||
| log earlier Track Top-40 songs | -0.793 |
|
||||
| artist's earlier conversion rate | +1.391 |
|
||||
| log Airplay Top-20 songs in the last year | +0.409 |
|
||||
| entered in weeks 46–52 | +0.035 |
|
||||
| streaming era | +0.160 |
|
||||
| streaming × entry score | -0.422 |
|
||||
|
||||
@@ -82,3 +82,10 @@ airplay model rank [--h 4] [--max-h 12] [--top 50] the ranking for h = 1..12 -
|
||||
- **Not conditioned on a release yet.** The ranking is expected airplay given current form. The release-effect event study needs release dates (Deezer backfill).
|
||||
- **"Release timing only" is not scored yet.** That baseline needs release dates too.
|
||||
- **Christmas is barely seen.** It appears only as lag history (weeks 48–52 of 2025), never as a target week.
|
||||
|
||||
## Update 15 Nov 2026: new adds, rotation lists, IFPI test bed
|
||||
|
||||
- **New adds** (`airplay model adds`, docs/BACKTEST-ADDS.md). The ranking used to show P(artist at or above the play threshold), which is close to 100% for anyone with a current hit. It now shows the chance of a **new add**: entering rotation after at least 4 weeks out, per channel, and on any DR channel after 4 weeks out everywhere, within an 8-week window starting h = 0–12 weeks ahead. Artist-level counts can't tell a new song from an old one returning. But in a week of song-level plays only 3–12% of artists in rotation had two songs in rotation at once, so "drop out, then get added" is the usual pattern. Features include run length on the list and how close the artist is to the threshold.
|
||||
- **Rotation lists** (`airplay rotations [--wayback] [--reparse]`, src/rotations.rs). Parses DR's PDF from glyph positions, with category headers, columns and the P3 summary page; decoration (rotated text) is dropped. Each page's channel comes from where its artists are played. Old lists came from the Wayback Machine's copies of DR's rotation page, giving **29 weekly lists from May 2023 to Nov 2026** in `rotation_lists`.
|
||||
- **Thresholds from the lists** (`airplay model calibrate`, docs/CALIBRATION.md). The six lists inside the DR window put "in rotation" at P3 23, P4 13–15, P5 10, P6 16 and P8 7 plays a week, about three times the old play-distribution guess. Every model now uses these.
|
||||
- **IFPI test bed** (`airplay model ifpi`, docs/IFPI.md): routes and lead–lag per chart period, and a 2010–2026 walk-forward test of Track Top-40 entry → Airplay Top-20 within 8 weeks.
|
||||
|
||||
Reference in New Issue
Block a user