# Per-channel history 2023- (archived DR artist pages) `AIRPLAY_HISTORY=1` puts the Wayback copies of DR's current artist pages (per channel, from 2023-09-11) before the live panel (from 2025-11-17): 165 weeks instead of 51. An artist-week in the archive part counts only inside an archived copy's 52 weeks (17% of archive artist-weeks); a training row needs the artist observed for the 20 weeks before its origin and the 15 after. Christmas re-entries are taken out of the add target (119 Christmas artists; `AIRPLAY_XMAS=0` keeps them). `AIRPLAY_ARCH_W` weights archive rows in training. Test origins are the same calendar weeks in every run (from 2026-05-18). Release events (3,588, 64 adds): log loss / PR-AUC / ROC-AUC, raw model | Training | Per channel | Any channel | |---|---|---| | 1 year | 0.0163 / 0.095 / 0.964 | 0.0649 / 0.235 / 0.888 | | 2023-26, full weight | 0.0150 / 0.122 / 0.968 | 0.0665 / 0.193 / 0.888 | | 2023-26, archive weight 0.3 | 0.0146 / 0.140 / 0.971 | 0.0651 / 0.213 / 0.894 | | 2023-26, archive weight 0.1 | 0.0146 / 0.142 / 0.969 | 0.0649 / 0.218 / 0.894 | All artists, add in the next 8 weeks (h = 0 / h = 4): log loss and PR-AUC, raw model | Training | Per channel, h=0 | Any, h=0 | Per channel, h=4 | Any, h=4 | |---|---|---|---|---| | 1 year | 0.0134 / 0.266 | 0.0741 / 0.249 | 0.0160 / 0.170 | 0.0789 / 0.199 | | 2023-26, full weight | 0.0134 / 0.279 | 0.0776 / 0.241 | 0.0160 / 0.191 | 0.0850 / 0.202 | | 2023-26, archive weight 0.3 | 0.0138 / 0.275 | 0.0783 / 0.238 | 0.0171 / 0.180 | 0.0889 / 0.194 | Findings (2026-11-19): - More history ranks *which channel* adds better (per-channel PR-AUC up 5-50%), most for release events. - *Whether any channel adds* gets worse in log loss: the history-trained model overpredicts today (former favourites 4.5% predicted vs 2.1% observed). An archive-era dummy came out at +0.02, so it is not a different base rate in the archive; more likely features whose meaning depends on how far back the panel goes ("weeks since last add", "no add in the panel yet") shift between archive and live rows. - NMF position score with history: no gain (per-channel PR-AUC 0.116 vs 0.122). - Christmas 2025 release events (origins 17 Nov - 29 Dec 2025): 105 events, 9 adds - too few to judge; the model did worse than "recent plays" there. - Not switched on for the site. Next: cap the history-dependent features at 52 weeks so archive and live rows mean the same thing, then re-test. ## Correction (2026-11-20): the earlier "history hurts" verdict was a bug At the boundary between archive and live weeks, artists without an archived copy looked silent in the archive weeks: an artist already in rotation when the live data begins showed a spurious add, and the "20 observed weeks before a row" rule dropped uncovered artists from the first 20 live weeks, so the general model trained mostly on popular (archived) artists. Fixed: an add needs GAP (surge: 8) observed weeks before it, and a row needs the 12 observed lag weeks the one-year panel needs; with archive weight 0 the history panel then reproduces the one-year model (up to the larger Christmas-artist list). After the fix, recalibrated (Platt), same test weeks (May-Nov 2026): | | 1 year | 2023-26, archive weight 0.3 | |---|---|---| | General model, any channel, log loss h=0 / h=4 | 0.0742 / 0.0789 | 0.0736 / 0.0782 | | General model, per channel, log loss h=0 / h=4 | 0.0135 / 0.0160 | 0.0130 / 0.0152 | | General model, per channel, PR-AUC h=0 / h=4 | 0.266 / 0.170 | 0.287 / 0.198 | | Release events, any channel, log loss / ROC-AUC / PR-AUC | 0.0655 / 0.882 / 0.224 | 0.0644 / 0.895 / 0.211 | | Release events, per channel, log loss | 0.0158 | 0.0151 | History now wins on the coming period nearly everywhere; by the project rule (largest dataset, then the best accuracy on the coming period) it should be on. Archive rows are subsampled (share = weight) to keep memory down. Switching it on for the site was reverted after the service hit its 1.5 GB memory limit. Also found: per-channel probabilities above ~0.2 are overconfident (40-50% predicted, ~8% observed), mostly P4 regions for artists already in rotation (Mumle: 44% on P4 vs 6% any channel). A floor making "any" >= the best station group made "any" worse (0.0664 -> 0.0690) until those are recalibrated.