5.9 KiB
5.9 KiB
Backtest: new adds
An add = the artist enters rotation (plays ≥ τ) after at least 4 weeks out of rotation: per channel, and on any DR channel after 4 weeks out of rotation everywhere. Target: an add within the 8 weeks that start h weeks after the forecast origin. Origins from week 27; each fit uses only windows closed by its origin.
Window starting h = 0 weeks ahead (17 forecast origins)
Base rates: per channel 0.36% of artist-channels get an add in the window; on any channel 2.0% of artists.
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|---|---|---|---|---|
| rate by in-rotation-now | 0.0234 | 0.006 | 0.0994 | 0.023 |
| rate by last-12-weeks level | 0.0162 | 0.097 | 0.0863 | 0.073 |
| logit | 0.0130 | 0.245 | 0.0747 | 0.199 |
Skill of the logit (90% block-bootstrap interval over origins):
| vs in-rotation-now | vs last-12-weeks level | |
|---|---|---|
| per channel | 0.444 (0.427 to 0.458) | 0.196 (0.186 to 0.199) |
| any channel | 0.249 (0.230 to 0.265) | 0.134 (0.124 to 0.142) |
Calibration, per channel:
| Predicted | Rows | Mean predicted | Observed |
|---|---|---|---|
| 0.0–0.1 | 1669178 | 0.002 | 0.002 |
| 0.1–0.2 | 5763 | 0.143 | 0.193 |
| 0.2–0.3 | 2302 | 0.244 | 0.232 |
| 0.3–0.4 | 1408 | 0.345 | 0.294 |
| 0.4–0.5 | 872 | 0.441 | 0.336 |
| 0.5–0.6 | 583 | 0.547 | 0.453 |
| 0.6–0.7 | 306 | 0.653 | 0.461 |
| 0.7–0.8 | 189 | 0.739 | 0.339 |
| 0.8–0.9 | 152 | 0.839 | 0.658 |
| 0.9–1.0 | 3 | 0.902 | 1.000 |
Window starting h = 4 weeks ahead (13 forecast origins)
Base rates: per channel 0.37% of artist-channels get an add in the window; on any channel 2.0% of artists.
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|---|---|---|---|---|
| rate by in-rotation-now | 0.0232 | 0.014 | 0.0979 | 0.031 |
| rate by last-12-weeks level | 0.0174 | 0.087 | 0.0887 | 0.066 |
| logit | 0.0160 | 0.134 | 0.0805 | 0.142 |
Skill of the logit (90% block-bootstrap interval over origins):
| vs in-rotation-now | vs last-12-weeks level | |
|---|---|---|
| per channel | 0.313 (0.305 to 0.323) | 0.081 (0.057 to 0.096) |
| any channel | 0.178 (0.162 to 0.190) | 0.092 (0.084 to 0.098) |
Calibration, per channel:
| Predicted | Rows | Mean predicted | Observed |
|---|---|---|---|
| 0.0–0.1 | 1275342 | 0.002 | 0.002 |
| 0.1–0.2 | 6075 | 0.141 | 0.120 |
| 0.2–0.3 | 2066 | 0.244 | 0.169 |
| 0.3–0.4 | 812 | 0.342 | 0.197 |
| 0.4–0.5 | 509 | 0.447 | 0.287 |
| 0.5–0.6 | 332 | 0.539 | 0.455 |
| 0.6–0.7 | 110 | 0.650 | 0.273 |
| 0.7–0.8 | 36 | 0.737 | 0.278 |
| 0.8–0.9 | 2 | 0.841 | 0.000 |
Coefficients, window starting now (h = 0), all closed windows
| Any-channel feature | Coefficient |
|---|---|
| intercept | -5.107 |
| log DR plays t | +0.979 |
| log DR plays 4w | +0.245 |
| log DR plays 12w | -0.284 |
| log max DR plays 12w | +0.420 |
| in rotation anywhere now | -1.359 |
| station groups in rotation now (/5) | -3.411 |
| current run anywhere (/12) | -1.907 |
| weeks since in rotation anywhere (/12) | -0.348 |
| Track Top-40 score | +0.147 |
| Airplay Top-20 score | -0.633 |
| chart entry recency 6w | +1.325 |
| chart entry recency 26w | -1.443 |
| log Track Top-40 hits | -0.083 |
| log Airplay Top-20 hits | -0.057 |
| Danish | +0.505 |
| group | -0.363 |
| female | -0.169 |
| career years (/50) | +0.108 |
| debut unknown | +0.514 |
| Christmas in window | +0.000 |
| discography known | +0.877 |
| release recency 4w | +0.345 |
| release recency 12w | +0.630 |
| log releases last 52w | -0.191 |
| releases in the window | -0.425 |
| releases in window x not in rotation now | +1.334 |
| DR: weeks since last add 13-25 | -0.015 |
| DR: last add 26+ weeks ago | -0.511 |
| DR: no add in the panel yet | -0.998 |
| Airplay Top-20: last new song 13-25 weeks ago | +0.255 |
| Airplay Top-20: 0.5-1.25x own usual gap | -0.106 |
| Airplay Top-20: last new song 2+ years ago | -0.548 |
| Per-channel feature | Coefficient |
|---|---|
| intercept | -8.129 |
| log plays t | +0.772 |
| log plays t-1 | +0.119 |
| log mean plays 4w | +0.121 |
| log mean plays 12w | -0.700 |
| log max plays 12w | +1.336 |
| log other channels t | +0.210 |
| log other channels 4w | -0.129 |
| log P3 t | +0.121 |
| weeks since played (/12) | -1.250 |
| Track Top-40 score | -0.315 |
| Airplay Top-20 score | -0.715 |
| chart entry recency 6w | +0.400 |
| chart entry recency 26w | -0.160 |
| log Track Top-40 hits | +0.273 |
| log Airplay Top-20 hits | -0.269 |
| Danish | +0.222 |
| group | -0.661 |
| female | +0.019 |
| career years (/50) | -1.152 |
| debut unknown | -0.069 |
| Christmas target week | +0.000 |
| Danish x P3 | +0.010 |
| Danish x P4 | +0.343 |
| Danish x P6 | -0.044 |
| ch p4kbh | +0.293 |
| ch p4sjaelland | +0.344 |
| ch p4fyn | +0.357 |
| ch p4syd | +0.428 |
| ch p4trekanten | +0.408 |
| ch p4vest | +0.465 |
| ch p4aarhus | +0.378 |
| ch p4nord | +0.230 |
| ch p4bornholm | +0.373 |
| ch p4esbjerg | +0.398 |
| ch p5 | +1.665 |
| ch p6beat | +1.017 |
| ch p8jazz | +2.213 |
| in rotation on channel now | -1.017 |
| in rotation anywhere now | +0.188 |
| station groups in rotation now (/5) | -1.490 |
| run on channel = 1 week | +0.936 |
| run = 2 weeks | +0.398 |
| run = 3-5 weeks | -1.398 |
| run >= 6 weeks | -0.954 |
| near threshold (>= half of tau, not in rotation) | +1.429 |
| plays / tau (capped 3, /3) | -0.704 |
| weeks since in rotation on channel (/12) | +0.497 |
| discography known | +0.741 |
| release recency 4w | +0.924 |
| release recency 12w | +0.021 |
| log releases last 52w | -0.185 |
| releases in the window | +0.226 |
| releases in window x not in rotation now | +0.822 |
| DR: weeks since last add 13-25 | +0.070 |
| DR: last add 26+ weeks ago | +0.247 |
| DR: no add in the panel yet | -0.105 |
| Airplay Top-20: last new song 13-25 weeks ago | +0.171 |
| Airplay Top-20: 0.5-1.25x own usual gap | +0.032 |
| Airplay Top-20: last new song 2+ years ago | -0.254 |