169 lines
5.9 KiB
Markdown
169 lines
5.9 KiB
Markdown
# Backtest: new adds
|
||
|
||
An add = the artist enters rotation (plays ≥ τ) after at least 4 weeks out of rotation: per channel, and on any DR channel after 4 weeks out of rotation everywhere. Target: an add within the 8 weeks that start h weeks after the forecast origin. Origins from week 27; each fit uses only windows closed by its origin.
|
||
|
||
## Window starting h = 0 weeks ahead (17 forecast origins)
|
||
|
||
Base rates: per channel 0.36% of artist-channels get an add in the window; on any channel 2.0% of artists.
|
||
|
||
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|
||
|---|---|---|---|---|
|
||
| rate by in-rotation-now | 0.0234 | 0.006 | 0.0994 | 0.023 |
|
||
| rate by last-12-weeks level | 0.0162 | 0.097 | 0.0863 | 0.073 |
|
||
| logit | 0.0130 | 0.245 | 0.0747 | 0.199 |
|
||
|
||
Skill of the logit (90% block-bootstrap interval over origins):
|
||
|
||
| | vs in-rotation-now | vs last-12-weeks level |
|
||
|---|---|---|
|
||
| per channel | 0.444 (0.427 to 0.458) | 0.196 (0.186 to 0.199) |
|
||
| any channel | 0.249 (0.230 to 0.265) | 0.134 (0.124 to 0.142) |
|
||
|
||
Calibration, per channel:
|
||
|
||
| Predicted | Rows | Mean predicted | Observed |
|
||
|---|---|---|---|
|
||
| 0.0–0.1 | 1669178 | 0.002 | 0.002 |
|
||
| 0.1–0.2 | 5763 | 0.143 | 0.193 |
|
||
| 0.2–0.3 | 2302 | 0.244 | 0.232 |
|
||
| 0.3–0.4 | 1408 | 0.345 | 0.294 |
|
||
| 0.4–0.5 | 872 | 0.441 | 0.336 |
|
||
| 0.5–0.6 | 583 | 0.547 | 0.453 |
|
||
| 0.6–0.7 | 306 | 0.653 | 0.461 |
|
||
| 0.7–0.8 | 189 | 0.739 | 0.339 |
|
||
| 0.8–0.9 | 152 | 0.839 | 0.658 |
|
||
| 0.9–1.0 | 3 | 0.902 | 1.000 |
|
||
|
||
## Window starting h = 4 weeks ahead (13 forecast origins)
|
||
|
||
Base rates: per channel 0.37% of artist-channels get an add in the window; on any channel 2.0% of artists.
|
||
|
||
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|
||
|---|---|---|---|---|
|
||
| rate by in-rotation-now | 0.0232 | 0.014 | 0.0979 | 0.031 |
|
||
| rate by last-12-weeks level | 0.0174 | 0.087 | 0.0887 | 0.066 |
|
||
| logit | 0.0160 | 0.134 | 0.0805 | 0.142 |
|
||
|
||
Skill of the logit (90% block-bootstrap interval over origins):
|
||
|
||
| | vs in-rotation-now | vs last-12-weeks level |
|
||
|---|---|---|
|
||
| per channel | 0.313 (0.305 to 0.323) | 0.081 (0.057 to 0.096) |
|
||
| any channel | 0.178 (0.162 to 0.190) | 0.092 (0.084 to 0.098) |
|
||
|
||
Calibration, per channel:
|
||
|
||
| Predicted | Rows | Mean predicted | Observed |
|
||
|---|---|---|---|
|
||
| 0.0–0.1 | 1275342 | 0.002 | 0.002 |
|
||
| 0.1–0.2 | 6075 | 0.141 | 0.120 |
|
||
| 0.2–0.3 | 2066 | 0.244 | 0.169 |
|
||
| 0.3–0.4 | 812 | 0.342 | 0.197 |
|
||
| 0.4–0.5 | 509 | 0.447 | 0.287 |
|
||
| 0.5–0.6 | 332 | 0.539 | 0.455 |
|
||
| 0.6–0.7 | 110 | 0.650 | 0.273 |
|
||
| 0.7–0.8 | 36 | 0.737 | 0.278 |
|
||
| 0.8–0.9 | 2 | 0.841 | 0.000 |
|
||
|
||
## Coefficients, window starting now (h = 0), all closed windows
|
||
|
||
| Any-channel feature | Coefficient |
|
||
|---|---|
|
||
| intercept | -5.107 |
|
||
| log DR plays t | +0.979 |
|
||
| log DR plays 4w | +0.245 |
|
||
| log DR plays 12w | -0.284 |
|
||
| log max DR plays 12w | +0.420 |
|
||
| in rotation anywhere now | -1.359 |
|
||
| station groups in rotation now (/5) | -3.411 |
|
||
| current run anywhere (/12) | -1.907 |
|
||
| weeks since in rotation anywhere (/12) | -0.348 |
|
||
| Track Top-40 score | +0.147 |
|
||
| Airplay Top-20 score | -0.633 |
|
||
| chart entry recency 6w | +1.325 |
|
||
| chart entry recency 26w | -1.443 |
|
||
| log Track Top-40 hits | -0.083 |
|
||
| log Airplay Top-20 hits | -0.057 |
|
||
| Danish | +0.505 |
|
||
| group | -0.363 |
|
||
| female | -0.169 |
|
||
| career years (/50) | +0.108 |
|
||
| debut unknown | +0.514 |
|
||
| Christmas in window | +0.000 |
|
||
| discography known | +0.877 |
|
||
| release recency 4w | +0.345 |
|
||
| release recency 12w | +0.630 |
|
||
| log releases last 52w | -0.191 |
|
||
| releases in the window | -0.425 |
|
||
| releases in window x not in rotation now | +1.334 |
|
||
| DR: weeks since last add 13-25 | -0.015 |
|
||
| DR: last add 26+ weeks ago | -0.511 |
|
||
| DR: no add in the panel yet | -0.998 |
|
||
| Airplay Top-20: last new song 13-25 weeks ago | +0.255 |
|
||
| Airplay Top-20: 0.5-1.25x own usual gap | -0.106 |
|
||
| Airplay Top-20: last new song 2+ years ago | -0.548 |
|
||
|
||
| Per-channel feature | Coefficient |
|
||
|---|---|
|
||
| intercept | -8.129 |
|
||
| log plays t | +0.772 |
|
||
| log plays t-1 | +0.119 |
|
||
| log mean plays 4w | +0.121 |
|
||
| log mean plays 12w | -0.700 |
|
||
| log max plays 12w | +1.336 |
|
||
| log other channels t | +0.210 |
|
||
| log other channels 4w | -0.129 |
|
||
| log P3 t | +0.121 |
|
||
| weeks since played (/12) | -1.250 |
|
||
| Track Top-40 score | -0.315 |
|
||
| Airplay Top-20 score | -0.715 |
|
||
| chart entry recency 6w | +0.400 |
|
||
| chart entry recency 26w | -0.160 |
|
||
| log Track Top-40 hits | +0.273 |
|
||
| log Airplay Top-20 hits | -0.269 |
|
||
| Danish | +0.222 |
|
||
| group | -0.661 |
|
||
| female | +0.019 |
|
||
| career years (/50) | -1.152 |
|
||
| debut unknown | -0.069 |
|
||
| Christmas target week | +0.000 |
|
||
| Danish x P3 | +0.010 |
|
||
| Danish x P4 | +0.343 |
|
||
| Danish x P6 | -0.044 |
|
||
| ch p4kbh | +0.293 |
|
||
| ch p4sjaelland | +0.344 |
|
||
| ch p4fyn | +0.357 |
|
||
| ch p4syd | +0.428 |
|
||
| ch p4trekanten | +0.408 |
|
||
| ch p4vest | +0.465 |
|
||
| ch p4aarhus | +0.378 |
|
||
| ch p4nord | +0.230 |
|
||
| ch p4bornholm | +0.373 |
|
||
| ch p4esbjerg | +0.398 |
|
||
| ch p5 | +1.665 |
|
||
| ch p6beat | +1.017 |
|
||
| ch p8jazz | +2.213 |
|
||
| in rotation on channel now | -1.017 |
|
||
| in rotation anywhere now | +0.188 |
|
||
| station groups in rotation now (/5) | -1.490 |
|
||
| run on channel = 1 week | +0.936 |
|
||
| run = 2 weeks | +0.398 |
|
||
| run = 3-5 weeks | -1.398 |
|
||
| run >= 6 weeks | -0.954 |
|
||
| near threshold (>= half of tau, not in rotation) | +1.429 |
|
||
| plays / tau (capped 3, /3) | -0.704 |
|
||
| weeks since in rotation on channel (/12) | +0.497 |
|
||
| discography known | +0.741 |
|
||
| release recency 4w | +0.924 |
|
||
| release recency 12w | +0.021 |
|
||
| log releases last 52w | -0.185 |
|
||
| releases in the window | +0.226 |
|
||
| releases in window x not in rotation now | +0.822 |
|
||
| DR: weeks since last add 13-25 | +0.070 |
|
||
| DR: last add 26+ weeks ago | +0.247 |
|
||
| DR: no add in the panel yet | -0.105 |
|
||
| Airplay Top-20: last new song 13-25 weeks ago | +0.171 |
|
||
| Airplay Top-20: 0.5-1.25x own usual gap | +0.032 |
|
||
| Airplay Top-20: last new song 2+ years ago | -0.254 |
|
||
|