Long any-DR model on 2019- archive pages: all years vs 1 year backtest
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,80 @@
|
||||
# The long any-DR model (2019–)
|
||||
|
||||
Total DR plays per artist and week (all channels, P4 regions included). An *add* is τ = 20 or more plays in a week of the next 2 week(s) after 8 weeks under τ. Archive era 2019-04-08 to 2025-11-10 (15538 artists observed, 1040482 qualifying rows, 4681 adds); current panel 2025-11-17 to 2026-11-02 (315968 rows, 1229 adds).
|
||||
|
||||
Models: **1-year** trains on the 52 weeks of origins before the test period (what the site can do from the current panel alone); **all years** on every earlier origin; **+ older history** adds four features from the artist's weeks before the last 52 (for today's predictions: the 2019–24 archive); **all years + era** adds an archive-era dummy, switched off when predicting (the archive over-represents popular artists, so its base rate is higher). Training rows end `win` weeks before the test origins, so no label is seen early. Δ < 0 means better than 1-year.
|
||||
|
||||
## Findings (2026-11-17)
|
||||
|
||||
- **Training on more years doesn't improve the model.** 2019–2024 vs. the last 52 weeks makes ±0.0003 difference in log loss in every test year and for the current panel, with the same average precision. τ = 50 gives the same picture. A model with ~30 coefficients is already pinned down by one year (300k rows, ~1,200 adds).
|
||||
- **An artist's older history doesn't help today's predictions either.** The four "older history" features (archive 2019–24 for today's origins) gain at most 0.0001.
|
||||
- **Pooling with the archive hurts calibration today** (mean p 0.0051 vs a rate of 0.0037), even with an era dummy. The archive over-represents artists whose pages got crawled.
|
||||
- **So the one-year limit isn't what holds the model back.** Signal is: there's little to learn about *when* an artist crosses into heavy rotation beyond their current trajectory, releases, NMF and charts. Better data on upcoming releases and promo (singles sent to radio) is more promising than more history.
|
||||
|
||||
## Out of sample
|
||||
|
||||
| test period | rows | adds | model | log loss | Brier skill | avg precision | mean p / rate | Δ log loss vs 1-year (90% CI) |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| 2021 | 342367 | 1605 | 1-year | 0.0219 | +0.062 | 0.110 | 0.0038 / 0.0047 | — |
|
||||
| 2021 | 342367 | 1605 | all years | 0.0219 | +0.061 | 0.108 | 0.0038 / 0.0047 | +0.0001 (+0.0000 to +0.0001) |
|
||||
| 2021 | 342367 | 1605 | all years + era | 0.0219 | +0.061 | 0.108 | 0.0038 / 0.0047 | +0.0001 (+0.0000 to +0.0001) |
|
||||
| 2021 | 342367 | 1605 | 1-year + older history | 0.0221 | +0.059 | 0.106 | 0.0034 / 0.0047 | +0.0002 (+0.0001 to +0.0004) |
|
||||
| 2021 | 342367 | 1605 | all years + era + older history | 0.0222 | +0.058 | 0.105 | 0.0034 / 0.0047 | +0.0003 (+0.0002 to +0.0004) |
|
||||
| 2022 | 146672 | 1098 | 1-year | 0.0329 | +0.042 | 0.109 | 0.0088 / 0.0075 | — |
|
||||
| 2022 | 146672 | 1098 | all years | 0.0330 | +0.049 | 0.108 | 0.0077 / 0.0075 | +0.0001 (-0.0002 to +0.0004) |
|
||||
| 2022 | 146672 | 1098 | all years + era | 0.0330 | +0.049 | 0.108 | 0.0077 / 0.0075 | +0.0001 (-0.0002 to +0.0004) |
|
||||
| 2022 | 146672 | 1098 | 1-year + older history | 0.0329 | +0.045 | 0.111 | 0.0087 / 0.0075 | -0.0001 (-0.0001 to -0.0000) |
|
||||
| 2022 | 146672 | 1098 | all years + era + older history | 0.0329 | +0.050 | 0.109 | 0.0080 / 0.0075 | -0.0000 (-0.0003 to +0.0003) |
|
||||
| 2023 | 134721 | 549 | 1-year | 0.0195 | +0.043 | 0.111 | 0.0062 / 0.0041 | — |
|
||||
| 2023 | 134721 | 549 | all years | 0.0193 | +0.040 | 0.111 | 0.0061 / 0.0041 | -0.0002 (-0.0003 to +0.0001) |
|
||||
| 2023 | 134721 | 549 | all years + era | 0.0193 | +0.040 | 0.111 | 0.0061 / 0.0041 | -0.0002 (-0.0003 to +0.0001) |
|
||||
| 2023 | 134721 | 549 | 1-year + older history | 0.0197 | +0.033 | 0.099 | 0.0064 / 0.0041 | +0.0002 (+0.0001 to +0.0003) |
|
||||
| 2023 | 134721 | 549 | all years + era + older history | 0.0194 | +0.036 | 0.101 | 0.0061 / 0.0041 | -0.0001 (-0.0002 to +0.0002) |
|
||||
| 2024 | 20378 | 107 | 1-year | 0.0220 | +0.097 | 0.179 | 0.0055 / 0.0053 | — |
|
||||
| 2024 | 20378 | 107 | all years | 0.0223 | +0.095 | 0.179 | 0.0067 / 0.0053 | +0.0003 (+0.0001 to +0.0009) |
|
||||
| 2024 | 20378 | 107 | all years + era | 0.0223 | +0.095 | 0.179 | 0.0067 / 0.0053 | +0.0003 (+0.0001 to +0.0009) |
|
||||
| 2024 | 20378 | 107 | 1-year + older history | 0.0218 | +0.101 | 0.186 | 0.0056 / 0.0053 | -0.0002 (-0.0004 to -0.0001) |
|
||||
| 2024 | 20378 | 107 | all years + era + older history | 0.0225 | +0.088 | 0.164 | 0.0066 / 0.0053 | +0.0005 (+0.0003 to +0.0010) |
|
||||
| current panel, origins from 2026-05-18 | 173697 | 641 | 1-year | 0.0163 | +0.069 | 0.128 | 0.0034 / 0.0037 | — |
|
||||
| current panel, origins from 2026-05-18 | 173697 | 641 | all years | 0.0164 | +0.072 | 0.125 | 0.0037 / 0.0037 | +0.0001 (-0.0000 to +0.0002) |
|
||||
| current panel, origins from 2026-05-18 | 173697 | 641 | all years + era | 0.0166 | +0.065 | 0.126 | 0.0051 / 0.0037 | +0.0003 (+0.0001 to +0.0005) |
|
||||
| current panel, origins from 2026-05-18 | 173697 | 641 | 1-year + older history | 0.0162 | +0.069 | 0.129 | 0.0033 / 0.0037 | -0.0000 (-0.0001 to +0.0001) |
|
||||
| current panel, origins from 2026-05-18 | 173697 | 641 | all years + era + older history | 0.0164 | +0.065 | 0.126 | 0.0048 / 0.0037 | +0.0002 (-0.0000 to +0.0003) |
|
||||
|
||||
## Coefficients (all years + era + older history, every row)
|
||||
|
||||
| feature | β |
|
||||
|---|---|
|
||||
| intercept | -7.944 |
|
||||
| log plays week t | +0.720 |
|
||||
| log mean plays last 4w | +0.329 |
|
||||
| log mean plays last 12w | +1.063 |
|
||||
| log max weekly plays last 52w | +0.583 |
|
||||
| on DR 9-25 weeks ago | +0.434 |
|
||||
| on DR 26-52 weeks ago | -0.122 |
|
||||
| weeks observed of last 52 (share) | +0.364 |
|
||||
| weeks with plays of last 12 (share) | -2.339 |
|
||||
| trend: last 4w vs the 8 before | +0.511 |
|
||||
| discography known | +0.256 |
|
||||
| release in last 4w | +0.405 |
|
||||
| release 5-12w ago | +0.187 |
|
||||
| release in the window | +0.381 |
|
||||
| log releases last 52w | -0.163 |
|
||||
| on Track Top-40 now | +0.587 |
|
||||
| on Airplay Top-20 now | +0.546 |
|
||||
| log Top-40 hits so far | +0.035 |
|
||||
| log Airplay hits so far | +0.094 |
|
||||
| new chart entry in last 4w | +0.078 |
|
||||
| NMF data covers t | -0.397 |
|
||||
| NMF Denmark in last 4w | +0.920 |
|
||||
| NMF Denmark in the window | +1.103 |
|
||||
| log NMF placements last 52w | -0.274 |
|
||||
| Danish | +0.065 |
|
||||
| group | +0.099 |
|
||||
| female | +0.284 |
|
||||
| season index of the window | +0.019 |
|
||||
| archive era | -0.204 |
|
||||
| older history: observed before the last 52w | +0.226 |
|
||||
| older history: log max weekly plays | -0.034 |
|
||||
| older history: log adds | +0.276 |
|
||||
| older history: share of weeks on DR | -1.115 |
|
||||
Reference in New Issue
Block a user