143 lines
4.8 KiB
Markdown
143 lines
4.8 KiB
Markdown
# Backtest: new adds
|
||
|
||
An add = the artist enters rotation (plays ≥ τ) after at least 4 weeks out of rotation: per channel, and on any DR channel after 4 weeks out of rotation everywhere. Target: an add within the 8 weeks that start h weeks after the forecast origin. Origins from week 27; each fit uses only windows closed by its origin.
|
||
|
||
## Window starting h = 0 weeks ahead (17 forecast origins)
|
||
|
||
Base rates: per channel 0.75% of artist-channels get an add in the window; on any channel 4.2% of artists.
|
||
|
||
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|
||
|---|---|---|---|---|
|
||
| rate by in-rotation-now | 0.0440 | 0.012 | 0.1758 | 0.042 |
|
||
| rate by last-12-weeks level | 0.0328 | 0.099 | 0.1656 | 0.083 |
|
||
| logit | 0.0273 | 0.230 | 0.1563 | 0.156 |
|
||
|
||
Skill of the logit (90% block-bootstrap interval over origins):
|
||
|
||
| | vs in-rotation-now | vs last-12-weeks level |
|
||
|---|---|---|
|
||
| per channel | 0.381 (0.361 to 0.396) | 0.168 (0.148 to 0.181) |
|
||
| any channel | 0.111 (0.099 to 0.120) | 0.056 (0.040 to 0.068) |
|
||
|
||
Calibration, per channel:
|
||
|
||
| Predicted | Rows | Mean predicted | Observed |
|
||
|---|---|---|---|
|
||
| 0.0–0.1 | 760136 | 0.003 | 0.004 |
|
||
| 0.1–0.2 | 4598 | 0.148 | 0.180 |
|
||
| 0.2–0.3 | 3797 | 0.246 | 0.219 |
|
||
| 0.3–0.4 | 1815 | 0.345 | 0.297 |
|
||
| 0.4–0.5 | 1080 | 0.444 | 0.359 |
|
||
| 0.5–0.6 | 715 | 0.548 | 0.365 |
|
||
| 0.6–0.7 | 252 | 0.643 | 0.476 |
|
||
| 0.7–0.8 | 149 | 0.742 | 0.470 |
|
||
| 0.8–0.9 | 6 | 0.809 | 0.000 |
|
||
|
||
## Window starting h = 4 weeks ahead (13 forecast origins)
|
||
|
||
Base rates: per channel 0.77% of artist-channels get an add in the window; on any channel 4.3% of artists.
|
||
|
||
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|
||
|---|---|---|---|---|
|
||
| rate by in-rotation-now | 0.0438 | 0.019 | 0.1752 | 0.053 |
|
||
| rate by last-12-weeks level | 0.0347 | 0.091 | 0.1689 | 0.078 |
|
||
| logit | 0.0326 | 0.139 | 0.1629 | 0.127 |
|
||
|
||
Skill of the logit (90% block-bootstrap interval over origins):
|
||
|
||
| | vs in-rotation-now | vs last-12-weeks level |
|
||
|---|---|---|
|
||
| per channel | 0.257 (0.242 to 0.276) | 0.062 (0.043 to 0.077) |
|
||
| any channel | 0.070 (0.063 to 0.075) | 0.035 (0.020 to 0.046) |
|
||
|
||
Calibration, per channel:
|
||
|
||
| Predicted | Rows | Mean predicted | Observed |
|
||
|---|---|---|---|
|
||
| 0.0–0.1 | 579503 | 0.004 | 0.005 |
|
||
| 0.1–0.2 | 6709 | 0.143 | 0.118 |
|
||
| 0.2–0.3 | 2781 | 0.243 | 0.193 |
|
||
| 0.3–0.4 | 1213 | 0.341 | 0.232 |
|
||
| 0.4–0.5 | 375 | 0.442 | 0.291 |
|
||
| 0.5–0.6 | 131 | 0.542 | 0.198 |
|
||
| 0.6–0.7 | 42 | 0.637 | 0.571 |
|
||
| 0.7–0.8 | 18 | 0.721 | 0.667 |
|
||
|
||
## Coefficients, window starting now (h = 0), all closed windows
|
||
|
||
| Any-channel feature | Coefficient |
|
||
|---|---|
|
||
| intercept | -1.503 |
|
||
| log DR plays t | +0.669 |
|
||
| log DR plays 4w | +0.119 |
|
||
| log DR plays 12w | -0.428 |
|
||
| log max DR plays 12w | +0.140 |
|
||
| in rotation anywhere now | -2.146 |
|
||
| channels in rotation now (/14) | -2.288 |
|
||
| current run anywhere (/12) | -1.783 |
|
||
| weeks since in rotation anywhere (/12) | -2.397 |
|
||
| Track Top-40 score | +0.248 |
|
||
| Airplay Top-20 score | +0.210 |
|
||
| chart entry recency 6w | +0.482 |
|
||
| chart entry recency 26w | -0.328 |
|
||
| log Track Top-40 hits | +0.002 |
|
||
| log Airplay Top-20 hits | -0.280 |
|
||
| Danish | +0.500 |
|
||
| group | -0.468 |
|
||
| female | -0.142 |
|
||
| career years (/50) | -0.250 |
|
||
| debut unknown | +0.129 |
|
||
| Christmas in window | +0.000 |
|
||
|
||
| Per-channel feature | Coefficient |
|
||
|---|---|
|
||
| intercept | -6.792 |
|
||
| log plays t | +0.665 |
|
||
| log plays t-1 | +0.129 |
|
||
| log mean plays 4w | +0.169 |
|
||
| log mean plays 12w | -0.662 |
|
||
| log max plays 12w | +1.391 |
|
||
| log other channels t | +0.257 |
|
||
| log other channels 4w | -0.214 |
|
||
| log P3 t | +0.174 |
|
||
| weeks since played (/12) | -0.733 |
|
||
| Track Top-40 score | -0.563 |
|
||
| Airplay Top-20 score | -0.416 |
|
||
| chart entry recency 6w | +0.160 |
|
||
| chart entry recency 26w | +0.288 |
|
||
| log Track Top-40 hits | +0.260 |
|
||
| log Airplay Top-20 hits | -0.324 |
|
||
| Danish | +0.295 |
|
||
| group | -0.840 |
|
||
| female | +0.007 |
|
||
| career years (/50) | -1.415 |
|
||
| debut unknown | -0.130 |
|
||
| Christmas target week | +0.000 |
|
||
| Danish x P3 | -0.176 |
|
||
| Danish x P4 | +0.154 |
|
||
| Danish x P6 | -0.213 |
|
||
| ch p4kbh | +0.236 |
|
||
| ch p4sjaelland | +0.256 |
|
||
| ch p4fyn | +0.235 |
|
||
| ch p4syd | +0.297 |
|
||
| ch p4trekanten | +0.272 |
|
||
| ch p4vest | +0.361 |
|
||
| ch p4aarhus | +0.297 |
|
||
| ch p4nord | +0.186 |
|
||
| ch p4bornholm | +0.278 |
|
||
| ch p4esbjerg | +0.290 |
|
||
| ch p5 | +1.468 |
|
||
| ch p6beat | +0.871 |
|
||
| ch p8jazz | +1.960 |
|
||
| in rotation on channel now | -1.352 |
|
||
| in rotation anywhere now | +0.104 |
|
||
| channels in rotation now (/14) | -0.674 |
|
||
| run on channel = 1 week | +0.944 |
|
||
| run = 2 weeks | +0.352 |
|
||
| run = 3-5 weeks | -1.570 |
|
||
| run >= 6 weeks | -1.078 |
|
||
| near threshold (>= half of tau, not in rotation) | +1.389 |
|
||
| plays / tau (capped 3, /3) | -0.664 |
|
||
| weeks since in rotation on channel (/12) | +0.421 |
|
||
|