Files
Airplay-Predictor/docs/HISTORY.md
T

38 lines
2.4 KiB
Markdown

# Per-channel history 2023- (archived DR artist pages)
`AIRPLAY_HISTORY=1` puts the Wayback copies of DR's current artist pages (per channel, from 2023-09-11)
before the live panel (from 2025-11-17): 165 weeks instead of 51. An artist-week in the archive part
counts only inside an archived copy's 52 weeks (17% of archive artist-weeks); a training row needs the
artist observed for the 20 weeks before its origin and the 15 after. Christmas re-entries are taken out
of the add target (119 Christmas artists; `AIRPLAY_XMAS=0` keeps them). `AIRPLAY_ARCH_W` weights
archive rows in training. Test origins are the same calendar weeks in every run (from 2026-05-18).
Release events (3,588, 64 adds): log loss / PR-AUC / ROC-AUC, raw model
| Training | Per channel | Any channel |
|---|---|---|
| 1 year | 0.0163 / 0.095 / 0.964 | 0.0649 / 0.235 / 0.888 |
| 2023-26, full weight | 0.0150 / 0.122 / 0.968 | 0.0665 / 0.193 / 0.888 |
| 2023-26, archive weight 0.3 | 0.0146 / 0.140 / 0.971 | 0.0651 / 0.213 / 0.894 |
| 2023-26, archive weight 0.1 | 0.0146 / 0.142 / 0.969 | 0.0649 / 0.218 / 0.894 |
All artists, add in the next 8 weeks (h = 0 / h = 4): log loss and PR-AUC, raw model
| Training | Per channel, h=0 | Any, h=0 | Per channel, h=4 | Any, h=4 |
|---|---|---|---|---|
| 1 year | 0.0134 / 0.266 | 0.0741 / 0.249 | 0.0160 / 0.170 | 0.0789 / 0.199 |
| 2023-26, full weight | 0.0134 / 0.279 | 0.0776 / 0.241 | 0.0160 / 0.191 | 0.0850 / 0.202 |
| 2023-26, archive weight 0.3 | 0.0138 / 0.275 | 0.0783 / 0.238 | 0.0171 / 0.180 | 0.0889 / 0.194 |
Findings (2026-11-19):
- More history ranks *which channel* adds better (per-channel PR-AUC up 5-50%), most for release events.
- *Whether any channel adds* gets worse in log loss: the history-trained model overpredicts today
(former favourites 4.5% predicted vs 2.1% observed). An archive-era dummy came out at +0.02, so it is
not a different base rate in the archive; more likely features whose meaning depends on how far back
the panel goes ("weeks since last add", "no add in the panel yet") shift between archive and live rows.
- NMF position score with history: no gain (per-channel PR-AUC 0.116 vs 0.122).
- Christmas 2025 release events (origins 17 Nov - 29 Dec 2025): 105 events, 9 adds - too few to judge;
the model did worse than "recent plays" there.
- Not switched on for the site. Next: cap the history-dependent features at 52 weeks so archive and live
rows mean the same thing, then re-test.