NMF Denmark in the add model (placed in the window, top 10, history), scenario type (release / NMF) for the ranking and the what-if; NMF Denmark export

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-11-15 11:43:52 +00:00
co-authored by Claude Opus 5.5
parent e6b080246b
commit e94601b997
6 changed files with 268 additions and 151 deletions
+137 -121
View File
@@ -4,165 +4,181 @@ An add = the artist enters rotation (plays ≥ τ) after at least 4 weeks out of
## Window starting h = 0 weeks ahead (17 forecast origins)
Base rates: per channel 0.36% of artist-channels get an add in the window; on any channel 2.0% of artists.
Base rates: per channel 0.32% of artist-channels get an add in the window; on any channel 1.8% of artists.
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|---|---|---|---|---|
| rate by in-rotation-now | 0.0234 | 0.006 | 0.0994 | 0.023 |
| rate by last-12-weeks level | 0.0162 | 0.097 | 0.0863 | 0.073 |
| logit | 0.0130 | 0.245 | 0.0747 | 0.199 |
| rate by in-rotation-now | 0.0214 | 0.006 | 0.0913 | 0.021 |
| rate by last-12-weeks level | 0.0147 | 0.093 | 0.0781 | 0.072 |
| logit | 0.0119 | 0.229 | 0.0679 | 0.204 |
Skill of the logit (90% block-bootstrap interval over origins):
| | vs in-rotation-now | vs last-12-weeks level |
|---|---|---|
| per channel | 0.444 (0.427 to 0.458) | 0.196 (0.186 to 0.199) |
| any channel | 0.249 (0.230 to 0.265) | 0.134 (0.124 to 0.142) |
| per channel | 0.441 (0.427 to 0.451) | 0.187 (0.174 to 0.191) |
| any channel | 0.256 (0.238 to 0.275) | 0.130 (0.120 to 0.141) |
Calibration, per channel:
| Predicted | Rows | Mean predicted | Observed |
|---|---|---|---|
| 0.0–0.1 | 1669178 | 0.002 | 0.002 |
| 0.1–0.2 | 5763 | 0.143 | 0.193 |
| 0.2–0.3 | 2302 | 0.244 | 0.232 |
| 0.3–0.4 | 1408 | 0.345 | 0.294 |
| 0.4–0.5 | 872 | 0.441 | 0.336 |
| 0.5–0.6 | 583 | 0.547 | 0.453 |
| 0.6–0.7 | 306 | 0.653 | 0.461 |
| 0.7–0.8 | 189 | 0.739 | 0.339 |
| 0.8–0.9 | 152 | 0.839 | 0.658 |
| 0.9–1.0 | 3 | 0.902 | 1.000 |
| 0.0–0.1 | 1876920 | 0.001 | 0.002 |
| 0.1–0.2 | 6259 | 0.142 | 0.211 |
| 0.2–0.3 | 2324 | 0.243 | 0.272 |
| 0.3–0.4 | 933 | 0.342 | 0.279 |
| 0.4–0.5 | 710 | 0.450 | 0.304 |
| 0.5–0.6 | 613 | 0.550 | 0.303 |
| 0.6–0.7 | 445 | 0.647 | 0.353 |
| 0.7–0.8 | 283 | 0.748 | 0.502 |
| 0.8–0.9 | 237 | 0.844 | 0.422 |
| 0.9–1.0 | 44 | 0.921 | 0.545 |
## Window starting h = 4 weeks ahead (13 forecast origins)
Base rates: per channel 0.37% of artist-channels get an add in the window; on any channel 2.0% of artists.
Base rates: per channel 0.33% of artist-channels get an add in the window; on any channel 1.8% of artists.
| Model | Log-loss, per channel | PR-AUC, per channel | Log-loss, any channel | PR-AUC, any channel |
|---|---|---|---|---|
| rate by in-rotation-now | 0.0232 | 0.014 | 0.0979 | 0.031 |
| rate by last-12-weeks level | 0.0174 | 0.087 | 0.0887 | 0.066 |
| logit | 0.0160 | 0.134 | 0.0805 | 0.142 |
| rate by in-rotation-now | 0.0213 | 0.014 | 0.0896 | 0.031 |
| rate by last-12-weeks level | 0.0158 | 0.084 | 0.0801 | 0.066 |
| logit | 0.0147 | 0.134 | 0.0744 | 0.143 |
Skill of the logit (90% block-bootstrap interval over origins):
| | vs in-rotation-now | vs last-12-weeks level |
|---|---|---|
| per channel | 0.313 (0.305 to 0.323) | 0.081 (0.057 to 0.096) |
| any channel | 0.178 (0.162 to 0.190) | 0.092 (0.084 to 0.098) |
| per channel | 0.310 (0.302 to 0.316) | 0.070 (0.042 to 0.087) |
| any channel | 0.170 (0.152 to 0.186) | 0.071 (0.064 to 0.080) |
Calibration, per channel:
| Predicted | Rows | Mean predicted | Observed |
|---|---|---|---|
| 0.0–0.1 | 1275342 | 0.002 | 0.002 |
| 0.1–0.2 | 6075 | 0.141 | 0.120 |
| 0.2–0.3 | 2066 | 0.244 | 0.169 |
| 0.3–0.4 | 812 | 0.342 | 0.197 |
| 0.4–0.5 | 509 | 0.447 | 0.287 |
| 0.5–0.6 | 332 | 0.539 | 0.455 |
| 0.6–0.7 | 110 | 0.650 | 0.273 |
| 0.7–0.8 | 36 | 0.737 | 0.278 |
| 0.8–0.9 | 2 | 0.841 | 0.000 |
| 0.0–0.1 | 1434699 | 0.002 | 0.002 |
| 0.1–0.2 | 5169 | 0.142 | 0.147 |
| 0.2–0.3 | 2108 | 0.243 | 0.164 |
| 0.3–0.4 | 980 | 0.344 | 0.198 |
| 0.4–0.5 | 621 | 0.446 | 0.203 |
| 0.5–0.6 | 404 | 0.544 | 0.223 |
| 0.6–0.7 | 255 | 0.644 | 0.475 |
| 0.7–0.8 | 74 | 0.751 | 0.203 |
| 0.8–0.9 | 42 | 0.837 | 0.310 |
## Coefficients, window starting now (h = 0), all closed windows
| Any-channel feature | Coefficient |
|---|---|
| intercept | -5.107 |
| log DR plays t | +0.979 |
| log DR plays 4w | +0.245 |
| log DR plays 12w | -0.284 |
| log max DR plays 12w | +0.420 |
| in rotation anywhere now | -1.359 |
| station groups in rotation now (/5) | -3.411 |
| current run anywhere (/12) | -1.907 |
| weeks since in rotation anywhere (/12) | -0.348 |
| Track Top-40 score | +0.147 |
| Airplay Top-20 score | -0.633 |
| chart entry recency 6w | +1.325 |
| chart entry recency 26w | -1.443 |
| log Track Top-40 hits | -0.083 |
| log Airplay Top-20 hits | -0.057 |
| Danish | +0.505 |
| group | -0.363 |
| female | -0.169 |
| career years (/50) | +0.108 |
| debut unknown | +0.514 |
| intercept | -4.804 |
| log DR plays t | +0.988 |
| log DR plays 4w | +0.335 |
| log DR plays 12w | -0.407 |
| log max DR plays 12w | +0.696 |
| in rotation anywhere now | -1.287 |
| station groups in rotation now (/5) | -3.848 |
| current run anywhere (/12) | -1.865 |
| weeks since in rotation anywhere (/12) | -0.272 |
| Track Top-40 score | +0.074 |
| Airplay Top-20 score | -0.687 |
| chart entry recency 6w | +1.656 |
| chart entry recency 26w | -1.565 |
| log Track Top-40 hits | -0.209 |
| log Airplay Top-20 hits | -0.197 |
| Danish | +0.499 |
| group | -0.341 |
| female | -0.140 |
| career years (/50) | +0.297 |
| debut unknown | +0.622 |
| Christmas in window | +0.000 |
| discography known | +0.877 |
| release recency 4w | +0.345 |
| release recency 12w | +0.630 |
| log releases last 52w | -0.191 |
| releases in the window | -0.425 |
| releases in window x not in rotation now | +1.334 |
| DR: weeks since last add 13-25 | -0.015 |
| DR: last add 26+ weeks ago | -0.511 |
| DR: no add in the panel yet | -0.998 |
| Airplay Top-20: last new song 13-25 weeks ago | +0.255 |
| Airplay Top-20: 0.5-1.25x own usual gap | -0.106 |
| Airplay Top-20: last new song 2+ years ago | -0.548 |
| discography known | -0.354 |
| release recency 4w | +0.199 |
| release recency 12w | +0.894 |
| log releases last 52w | -0.200 |
| releases in the window | -1.019 |
| releases in window x not in rotation now | +1.573 |
| DR: weeks since last add 13-25 | +0.099 |
| DR: last add 26+ weeks ago | -0.438 |
| DR: no add in the panel yet | -0.948 |
| Airplay Top-20: last new song 13-25 weeks ago | +0.299 |
| Airplay Top-20: 0.5-1.25x own usual gap | -0.018 |
| Airplay Top-20: last new song 2+ years ago | -0.529 |
| log archive plays per 100 sampled hours (2012-21) | +0.366 |
| log DR lists with the artist before t (2023-) | +0.332 |
| DR list recency 26w | -0.621 |
| DR lists known before t at all | -0.166 |
| NMF Denmark in the window | +0.985 |
| NMF Denmark top 10 in the window | +0.444 |
| log NMF Denmark placements last 52w | -0.079 |
| share of releases on NMF Denmark (2 years) | -0.623 |
| Per-channel feature | Coefficient |
|---|---|
| intercept | -8.129 |
| log plays t | +0.772 |
| intercept | -7.632 |
| log plays t | +0.821 |
| log plays t-1 | +0.119 |
| log mean plays 4w | +0.121 |
| log mean plays 12w | -0.700 |
| log max plays 12w | +1.336 |
| log other channels t | +0.210 |
| log other channels 4w | -0.129 |
| log P3 t | +0.121 |
| weeks since played (/12) | -1.250 |
| Track Top-40 score | -0.315 |
| Airplay Top-20 score | -0.715 |
| chart entry recency 6w | +0.400 |
| chart entry recency 26w | -0.160 |
| log Track Top-40 hits | +0.273 |
| log Airplay Top-20 hits | -0.269 |
| Danish | +0.222 |
| group | -0.661 |
| female | +0.019 |
| career years (/50) | -1.152 |
| debut unknown | -0.069 |
| log mean plays 4w | -0.008 |
| log mean plays 12w | -0.531 |
| log max plays 12w | +1.318 |
| log other channels t | +0.235 |
| log other channels 4w | -0.087 |
| log P3 t | +0.037 |
| weeks since played (/12) | -1.762 |
| Track Top-40 score | -0.273 |
| Airplay Top-20 score | -0.757 |
| chart entry recency 6w | +0.747 |
| chart entry recency 26w | -0.459 |
| log Track Top-40 hits | +0.226 |
| log Airplay Top-20 hits | -0.351 |
| Danish | +0.235 |
| group | -0.650 |
| female | +0.012 |
| career years (/50) | -1.128 |
| debut unknown | -0.056 |
| Christmas target week | +0.000 |
| Danish x P3 | +0.010 |
| Danish x P4 | +0.343 |
| Danish x P6 | -0.044 |
| ch p4kbh | +0.293 |
| ch p4sjaelland | +0.344 |
| ch p4fyn | +0.357 |
| ch p4syd | +0.428 |
| ch p4trekanten | +0.408 |
| ch p4vest | +0.465 |
| ch p4aarhus | +0.378 |
| ch p4nord | +0.230 |
| ch p4bornholm | +0.373 |
| ch p4esbjerg | +0.398 |
| ch p5 | +1.665 |
| ch p6beat | +1.017 |
| ch p8jazz | +2.213 |
| in rotation on channel now | -1.017 |
| in rotation anywhere now | +0.188 |
| station groups in rotation now (/5) | -1.490 |
| run on channel = 1 week | +0.936 |
| run = 2 weeks | +0.398 |
| run = 3-5 weeks | -1.398 |
| run >= 6 weeks | -0.954 |
| near threshold (>= half of tau, not in rotation) | +1.429 |
| plays / tau (capped 3, /3) | -0.704 |
| weeks since in rotation on channel (/12) | +0.497 |
| discography known | +0.741 |
| release recency 4w | +0.924 |
| release recency 12w | +0.021 |
| log releases last 52w | -0.185 |
| releases in the window | +0.226 |
| releases in window x not in rotation now | +0.822 |
| DR: weeks since last add 13-25 | +0.070 |
| DR: last add 26+ weeks ago | +0.247 |
| DR: no add in the panel yet | -0.105 |
| Airplay Top-20: last new song 13-25 weeks ago | +0.171 |
| Airplay Top-20: 0.5-1.25x own usual gap | +0.032 |
| Airplay Top-20: last new song 2+ years ago | -0.254 |
| Danish x P3 | -0.051 |
| Danish x P4 | +0.302 |
| Danish x P6 | +0.004 |
| ch p4kbh | +0.695 |
| ch p4sjaelland | +0.514 |
| ch p4fyn | +0.499 |
| ch p4syd | +0.544 |
| ch p4trekanten | +0.581 |
| ch p4vest | +0.611 |
| ch p4aarhus | +0.526 |
| ch p4nord | +0.587 |
| ch p4bornholm | +0.536 |
| ch p4esbjerg | +0.555 |
| ch p5 | +1.820 |
| ch p6beat | +1.118 |
| ch p8jazz | +2.402 |
| in rotation on channel now | -0.952 |
| in rotation anywhere now | +0.118 |
| station groups in rotation now (/5) | -1.127 |
| run on channel = 1 week | +0.893 |
| run = 2 weeks | +0.413 |
| run = 3-5 weeks | -1.287 |
| run >= 6 weeks | -0.971 |
| near threshold (>= half of tau, not in rotation) | +1.421 |
| plays / tau (capped 3, /3) | -0.967 |
| weeks since in rotation on channel (/12) | +0.425 |
| discography known | -0.042 |
| release recency 4w | +0.840 |
| release recency 12w | +0.180 |
| log releases last 52w | -0.059 |
| releases in the window | -0.478 |
| releases in window x not in rotation now | +1.007 |
| DR: weeks since last add 13-25 | +0.007 |
| DR: last add 26+ weeks ago | +0.310 |
| DR: no add in the panel yet | -0.072 |
| Airplay Top-20: last new song 13-25 weeks ago | +0.084 |
| Airplay Top-20: 0.5-1.25x own usual gap | +0.103 |
| Airplay Top-20: last new song 2+ years ago | -0.240 |
| log archive plays per 100 sampled hours (2012-21) | +0.179 |
| log DR lists with the artist before t (2023-) | +0.175 |
| DR list recency 26w | -0.254 |
| DR lists known before t at all | -0.234 |
| NMF Denmark in the window | +0.853 |
| NMF Denmark top 10 in the window | +0.479 |
| log NMF Denmark placements last 52w | -0.122 |
| share of releases on NMF Denmark (2 years) | -0.189 |
+68 -19
View File
@@ -52,6 +52,52 @@ pub fn wave_feats(p: &Panel, r: &Rot, a: usize, t: usize) -> [f64; WAVE] {
f(known && (13.0..=25.0).contains(&air)), f(ratio_ok), f(known && air >= 104.0)]
}
/// What the forecast assumes about the window: None = what actually happened (training and
/// backtests), Some = a scenario (the ranking, the what-if).
#[derive(Clone, Copy, Default)]
pub struct Scenario { pub release: Option<bool>, pub nmf: Option<bool> }
impl Scenario {
pub const ACTUAL: Scenario = Scenario { release: None, nmf: None };
pub fn new(release: bool, nmf: bool) -> Scenario { Scenario { release: Some(release), nmf: Some(nmf) } }
}
/// New Music Friday Denmark: placed in the window (and in its top 10), placements in the past
/// year, and the share of the artist's releases over two years that were placed.
pub const NMF: usize = 4;
pub const NMF_NAMES: &[&str] = &["NMF Denmark in the window", "NMF Denmark top 10 in the window", "log NMF Denmark placements last 52w",
"share of releases on NMF Denmark (2 years)"];
fn window(p: &Panel, t: usize, h: usize) -> (chrono::NaiveDate, chrono::NaiveDate) {
let w0 = p.weeks[t] + chrono::Duration::weeks(h as i64 + 1);
(w0, w0 + chrono::Duration::weeks(W as i64))
}
/// P(an artist's release is placed on NMF Denmark), from the past two years (shrunk).
pub fn p_nmf(p: &Panel, a: usize, t: usize) -> f64 {
let end_t = p.weeks[t] + chrono::Duration::days(6);
let placed = p.nmf_dk[a].iter().filter(|(d, _)| *d <= end_t && (end_t - *d).num_days() <= 730).count() as f64;
let rels = p.releases[a].as_ref().map(|v| v.iter().filter(|d| **d <= end_t && (end_t - **d).num_days() <= 730).count()).unwrap_or(0) as f64;
((placed + 0.1) / (rels + 1.0)).clamp(0.0, 0.95)
}
pub fn nmf_feats(p: &Panel, a: usize, t: usize, h: usize, sc: Scenario) -> [f64; NMF] {
if off("nmf") { return [0.0; NMF]; }
let end_t = p.weeks[t] + chrono::Duration::days(6);
let (w0, w1) = window(p, t, h);
let placed = p.nmf_dk[a].iter().filter(|(d, _)| *d >= w0 && *d < w1);
let (in_win, top10) = match sc.nmf {
None => { let v: Vec<_> = placed.collect(); (!v.is_empty(), v.iter().any(|(_, pos)| pos.is_some_and(|x| x <= 10))) }
Some(false) => (false, false),
// scenario "placed": top 10 as often as the artist's past placements were
Some(true) => (true, false),
};
let past: Vec<&(chrono::NaiveDate, Option<i64>)> = p.nmf_dk[a].iter().filter(|(d, _)| *d <= end_t).collect();
let top10_share = if past.is_empty() { 0.0 } else { past.iter().filter(|(_, pos)| pos.is_some_and(|x| x <= 10)).count() as f64 / past.len() as f64 };
let last_year = past.iter().filter(|(d, _)| (end_t - *d).num_days() <= 364).count() as f64;
let f = |b: bool| b as i32 as f64;
[f(in_win), if sc.nmf == Some(true) { top10_share } else { f(top10) }, last_year.ln_1p(), p_nmf(p, a, t)]
}
/// Long-history covariates: DR's 2012–2021 archive and DR's real rotation lists before week t.
pub const LONG: usize = 4;
pub const LONG_NAMES: &[&str] = &["log archive plays per 100 sampled hours (2012-21)", "log DR lists with the artist before t (2023-)",
@@ -74,7 +120,8 @@ pub const REL_NAMES: &[&str] = &["discography known", "release recency 4w", "rel
/// Weeks since the latest release, releases in the past year, and whether the artist releases in
/// the window t+h+1 ..= t+h+W (the actual one when `rel` is None: training and backtests; else the
/// scenario "releases / doesn't release in the window").
pub fn rel_feats(p: &Panel, a: usize, t: usize, h: usize, rel: Option<bool>, in_rot_now: bool) -> [f64; REL] {
pub fn rel_feats(p: &Panel, a: usize, t: usize, h: usize, sc: Scenario, in_rot_now: bool) -> [f64; REL] {
let rel = sc.release;
if off("rel") { return [0.0; REL]; }
let Some(dates) = p.releases[a].as_ref() else { return [0.0; REL] };
let end_t = p.weeks[t] + chrono::Duration::days(6);
@@ -102,7 +149,7 @@ pub const EXTRA: &[&str] = &["in rotation on channel now", "in rotation anywhere
"run on channel = 1 week", "run = 2 weeks", "run = 3-5 weeks", "run >= 6 weeks",
"near threshold (>= half of tau, not in rotation)", "plays / tau (capped 3, /3)",
"weeks since in rotation on channel (/12)"];
pub const PC: usize = P + 10 + REL + WAVE + LONG;
pub const PC: usize = P + 10 + REL + WAVE + LONG + NMF;
/// Artist-level ("any channel") rows.
pub const ANY_NAMES: &[&str] = &[
@@ -112,7 +159,7 @@ pub const ANY_NAMES: &[&str] = &[
"log Track Top-40 hits", "log Airplay Top-20 hits", "Danish", "group", "female", "career years (/50)", "debut unknown",
"Christmas in window",
];
pub const PA: usize = 21 + REL + WAVE + LONG;
pub const PA: usize = 21 + REL + WAVE + LONG + NMF;
/// Positions of the chart covariates (for the what-if): (t40 score, air score, recency 6w, recency 26w).
pub const CHART_IDX_CHANNEL: [usize; 4] = [10, 11, 12, 13];
pub const CHART_IDX_ANY: [usize; 4] = [9, 10, 11, 12];
@@ -162,7 +209,7 @@ fn run_and_since(flag: impl Fn(usize) -> bool, t: usize) -> (f64, f64) {
fn christmas_in_window(p: &Panel, t: usize, h: usize) -> bool { (t + h + 1..=t + h + W).any(|w| features::christmas(p, w)) }
pub fn row_channel(p: &Panel, r: &Rot, a: usize, c: usize, t: usize, h: usize, rel: Option<bool>, out: &mut [f64]) {
pub fn row_channel(p: &Panel, r: &Rot, a: usize, c: usize, t: usize, h: usize, sc: Scenario, out: &mut [f64]) {
features::row(p, a, c, t, t + h + 1, &mut out[..P]);
out[21] = christmas_in_window(p, t, h) as i32 as f64;
let n_on = groups_on(r, a, t);
@@ -174,12 +221,13 @@ pub fn row_channel(p: &Panel, r: &Rot, a: usize, c: usize, t: usize, h: usize, r
out[P..P + 10].copy_from_slice(&[f(on), f(r.any(a, t)), n_on / GROUPS as f64,
f(weeks == 1), f(weeks == 2), f((3..=5).contains(&weeks)), f(weeks >= 6),
f(!on && ratio >= 0.5), ratio.min(3.0) / 3.0, since]);
out[P + 10..P + 10 + REL].copy_from_slice(&rel_feats(p, a, t, h, rel, on));
out[P + 10..P + 10 + REL].copy_from_slice(&rel_feats(p, a, t, h, sc, on));
out[P + 10 + REL..P + 10 + REL + WAVE].copy_from_slice(&wave_feats(p, r, a, t));
out[P + 10 + REL + WAVE..PC].copy_from_slice(&long_feats(p, a, t));
out[P + 10 + REL + WAVE..P + 10 + REL + WAVE + LONG].copy_from_slice(&long_feats(p, a, t));
out[P + 10 + REL + WAVE + LONG..PC].copy_from_slice(&nmf_feats(p, a, t, h, sc));
}
pub fn row_any(p: &Panel, r: &Rot, a: usize, t: usize, h: usize, rel: Option<bool>, out: &mut [f64]) {
pub fn row_any(p: &Panel, r: &Rot, a: usize, t: usize, h: usize, sc: Scenario, out: &mut [f64]) {
let tot = |w: usize| p.group_total(a, w, None);
let mean = |n: usize| (0..n).map(|i| tot(t - i)).sum::<f64>() / n as f64;
let max12 = (0..LAGS).map(|i| tot(t - i)).fold(0.0, f64::max);
@@ -200,9 +248,10 @@ pub fn row_any(p: &Panel, r: &Rot, a: usize, t: usize, h: usize, rel: Option<boo
christmas_in_window(p, t, h) as i32 as f64,
];
out[..21].copy_from_slice(&v);
out[21..21 + REL].copy_from_slice(&rel_feats(p, a, t, h, rel, r.any(a, t)));
out[21..21 + REL].copy_from_slice(&rel_feats(p, a, t, h, sc, r.any(a, t)));
out[21 + REL..21 + REL + WAVE].copy_from_slice(&wave_feats(p, r, a, t));
out[21 + REL + WAVE..PA].copy_from_slice(&long_feats(p, a, t));
out[21 + REL + WAVE..21 + REL + WAVE + LONG].copy_from_slice(&long_feats(p, a, t));
out[21 + REL + WAVE + LONG..PA].copy_from_slice(&nmf_feats(p, a, t, h, sc));
}
/// Logit models for one h, trained on origins whose whole window is observed by `last_week`.
@@ -242,7 +291,7 @@ impl AddModel {
let (mut dc, mut da) = (Design::new(2), Design::new(2));
for t in c0..=hi {
for a in 0..p.a() {
let (pc, pa) = inner.predict(p, r, a, t, None, None);
let (pc, pa) = inner.predict(p, r, a, t, Scenario::ACTUAL, None);
for c in 0..C {
if pc[c] < 1e-4 && !r.add_in_window(a, c, t, h) { continue; } // certain zeros carry no information
dc.push(&[1.0, logit(pc[c])], r.add_in_window(a, c, t, h) as i32 as f64, 1.0);
@@ -268,10 +317,10 @@ impl AddModel {
// Channels where the artist hasn't been heard for 12 weeks: keep a quarter, weighted 4.
let quiet = (0..LAGS).all(|i| p.y(a, c, t - i) == 0.0);
let wgt = if quiet { if mix((a as u64) << 32 ^ (c as u64) << 20 ^ t as u64 ^ 0xadd) % 4 != 0 { continue; } 4.0 } else { 1.0 };
row_channel(p, r, a, c, t, h, None, &mut x);
row_channel(p, r, a, c, t, h, Scenario::ACTUAL, &mut x);
dc.push(&x, r.add_in_window(a, c, t, h) as i32 as f64, wgt);
}
row_any(p, r, a, t, h, None, &mut xa);
row_any(p, r, a, t, h, Scenario::ACTUAL, &mut xa);
da.push(&xa, r.add_any_in_window(a, t, h) as i32 as f64, 1.0);
}
}
@@ -279,17 +328,17 @@ impl AddModel {
}
/// (P(add) per channel, P(add on any channel)) from origin t.
/// `rel`: None = whether the artist actually released in the window (backtests), Some = scenario.
pub fn predict(&self, p: &Panel, r: &Rot, a: usize, t: usize, rel: Option<bool>, chart: Option<&dyn Fn(&mut [f64], [usize; 4])>) -> ([f64; C], f64) {
/// `sc`: what the window holds (Scenario::ACTUAL in backtests).
pub fn predict(&self, p: &Panel, r: &Rot, a: usize, t: usize, sc: Scenario, chart: Option<&dyn Fn(&mut [f64], [usize; 4])>) -> ([f64; C], f64) {
let mut x = vec![0.0; PC];
let mut pc = [0.0; C];
for c in 0..C {
row_channel(p, r, a, c, t, self.h, rel, &mut x);
row_channel(p, r, a, c, t, self.h, sc, &mut x);
if let Some(f) = chart { f(&mut x, CHART_IDX_CHANNEL); }
pc[c] = platt(self.platt_channel, glm::sigmoid(glm::dot(&self.channel, &x)));
}
let mut xa = vec![0.0; PA];
row_any(p, r, a, t, self.h, rel, &mut xa);
row_any(p, r, a, t, self.h, sc, &mut xa);
if let Some(f) = chart { f(&mut xa, CHART_IDX_ANY); }
// Coherence: "any station" can't exceed what the stations allow together (P4 counted once,
// as its best region): at most 1 - prod(1 - p) over the station groups.
@@ -379,7 +428,7 @@ pub fn backtest(p: &Panel, tau: &[u32; C], horizons: &[usize], start: usize, md:
}
let (mut lc, mut la) = ([0.0; 3], [0.0; 3]);
for a in 0..p.a() {
let (pc, pa) = m.predict(p, &r, a, t0, None, None);
let (pc, pa) = m.predict(p, &r, a, t0, Scenario::ACTUAL, None);
let mut tot = 0.0;
for c in 0..C {
let y = r.add_in_window(a, c, t0, h);
@@ -432,9 +481,9 @@ pub fn backtest(p: &Panel, tau: &[u32; C], horizons: &[usize], start: usize, md:
let m = AddModel::fit(p, &r, 0, p.w() - 1);
writeln!(md, "## Coefficients, window starting now (h = 0), all closed windows\n\n| Any-channel feature | Coefficient |\n|---|---|")?;
for (n, b) in ANY_NAMES.iter().zip(&m.any) { writeln!(md, "| {n} | {b:+.3} |")?; }
for (n, b) in REL_NAMES.iter().chain(WAVE_NAMES).chain(LONG_NAMES).zip(&m.any[21..]) { writeln!(md, "| {n} | {b:+.3} |")?; }
for (n, b) in REL_NAMES.iter().chain(WAVE_NAMES).chain(LONG_NAMES).chain(NMF_NAMES).zip(&m.any[21..]) { writeln!(md, "| {n} | {b:+.3} |")?; }
writeln!(md, "\n| Per-channel feature | Coefficient |\n|---|---|")?;
for (n, b) in features::NAMES.iter().chain(EXTRA).chain(REL_NAMES).chain(WAVE_NAMES).chain(LONG_NAMES).zip(&m.channel) { writeln!(md, "| {n} | {b:+.3} |")?; }
for (n, b) in features::NAMES.iter().chain(EXTRA).chain(REL_NAMES).chain(WAVE_NAMES).chain(LONG_NAMES).chain(NMF_NAMES).zip(&m.channel) { writeln!(md, "| {n} | {b:+.3} |")?; }
writeln!(md)?;
Ok(rows)
}
+39 -1
View File
@@ -79,6 +79,8 @@ pub struct Panel {
/// dates of DR's real rotation lists (2023–) that had the artist on any channel.
pub archive_rate: Vec<f64>,
pub listed_on: Vec<Vec<NaiveDate>>,
/// New Music Friday Denmark placements per artist: (Friday, position if known), sorted.
pub nmf_dk: Vec<Vec<(NaiveDate, Option<i64>)>>,
}
impl Panel {
@@ -144,7 +146,8 @@ impl Panel {
let list_tau = super::calibrate::stored(db)?;
let releases = load_releases(db, &id_of, artists.len())?;
let (archive_rate, listed_on) = load_long_history(db, &id_of, &names, artists.len())?;
Ok(Panel { weeks, artists, y, chart, list_tau, releases, archive_rate, listed_on })
let nmf_dk = load_nmf_dk(db, &id_of, &artists)?;
Ok(Panel { weeks, artists, y, chart, list_tau, releases, archive_rate, listed_on, nmf_dk })
}
}
@@ -272,3 +275,38 @@ fn load_long_history(db: &Db, id_of: &HashMap<String, usize>, names: &HashMap<St
for v in listed.iter_mut() { v.sort(); v.dedup(); }
Ok((rate, listed))
}
/// NMF Denmark placements per DR artist: the artist's SuperFridayChart placements, confirmed on the
/// track page (country Denmark, with position), or for Danish acts not yet read on the track page,
/// one playlist the size of NMF Denmark that Friday.
fn load_nmf_dk(db: &Db, id_of: &HashMap<String, usize>, artists: &[Artist]) -> Result<Vec<Vec<(NaiveDate, Option<i64>)>>> {
let mut out: Vec<Vec<(NaiveDate, Option<i64>)>> = vec![Vec::new(); artists.len()];
let has = |t: &str| -> Result<bool> { Ok(db.conn.query_row("SELECT count(*) FROM sqlite_master WHERE name = ?1", [t], |r| r.get::<_, i64>(0))? > 0) };
if !has("nmf_artist_tracks")? || !has("nmf_additions")? { return Ok(out); }
let mut confirmed: HashMap<(String, String), Option<i64>> = HashMap::new();
let mut read: std::collections::HashSet<(String, String)> = std::collections::HashSet::new();
let mut size: HashMap<String, f64> = HashMap::new();
for r in db.conn.prepare("SELECT isrc, friday, country, position, followers FROM nmf_additions")?
.query_map([], |r| Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?, r.get::<_, String>(2)?, r.get::<_, Option<i64>>(3)?, r.get::<_, Option<f64>>(4)?)))? {
let (isrc, f, country, pos, fo) = r?;
read.insert((isrc.clone(), f.clone()));
if country.eq_ignore_ascii_case("denmark") {
if let Some(fo) = fo { let e = size.entry(f.clone()).or_insert(0.0); *e = e.max(fo); }
confirmed.insert((isrc, f), pos);
}
}
for r in db.conn.prepare(
"SELECT l.dr_artist_id, t.isrc, t.friday, coalesce(t.playlists, 0), coalesce(t.followers, 0)
FROM nmf_artist_tracks t JOIN nmf_artist_lookups l ON l.spotify_artist_id = t.spotify_artist_id")?
.query_map([], |r| Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?, r.get::<_, String>(2)?, r.get::<_, i64>(3)?, r.get::<_, f64>(4)?)))? {
let (id, isrc, f, pl, fo) = r?;
let Some(&a) = id_of.get(&id) else { continue };
let Ok(d) = NaiveDate::parse_from_str(&f, "%Y-%m-%d") else { continue };
let key = (isrc, f.clone());
if let Some(pos) = confirmed.get(&key) { out[a].push((d, *pos)); continue; }
if read.contains(&key) { continue; } // read: not on NMF Denmark
if artists[a].danish && pl == 1 && size.get(&f).is_some_and(|n| (fo - n).abs() <= 0.02 * n) { out[a].push((d, None)); }
}
for v in out.iter_mut() { v.sort(); v.dedup_by_key(|x| x.0); }
Ok(out)
}
+11 -3
View File
@@ -7,7 +7,7 @@
//! independent. Not yet conditioned on a release at t + h: that needs the release-effect curve
//! from release dates (Deezer backfill). Until then this ranks expected airplay given current form.
use super::adds::{AddModel, Rot};
use super::adds::{AddModel, Rot, Scenario};
use super::common::{thresholds, Direct, Rng};
use super::features::P;
use super::panel::{Panel, C, CHANNELS};
@@ -123,8 +123,14 @@ pub fn run(db: &Db, p: &Panel, max_h: usize, show_h: usize) -> Result<Vec<Line>>
let id = &p.artists[a].id;
// If they release in the window, if they don't, and blended by their release rate.
let t0 = p.w() - 1;
let (pc1, pa1) = m.predict(p, &rot, a, t0, Some(true), None);
let (pc0, pa0) = m.predict(p, &rot, a, t0, Some(false), None);
// If they release: placed on NMF Denmark or not, weighted by how often the artist's
// releases were placed; if they don't release: no NMF placement.
let q = super::adds::p_nmf(p, a, t0);
let (pcn, pan) = m.predict(p, &rot, a, t0, Scenario::new(true, true), None);
let (pcr, par) = m.predict(p, &rot, a, t0, Scenario::new(true, false), None);
let (pc0, pa0) = m.predict(p, &rot, a, t0, Scenario::new(false, false), None);
let pc1: [f64; C] = std::array::from_fn(|c| q * pcn[c] + (1.0 - q) * pcr[c]);
let pa1 = q * pan + (1.0 - q) * par;
let pr = super::adds::p_release(p, a, t0);
let mix = |p1: f64, p0: f64| match pr { Some(r) => r * p1 + (1.0 - r) * p0, None => p0 };
let none = Option::<f64>::None;
@@ -132,6 +138,8 @@ pub fn run(db: &Db, p: &Panel, max_h: usize, show_h: usize) -> Result<Vec<Line>>
origin.to_string(), start, prob, none, none, none]);
put("add_any", "any", mix(pa1, pa0))?;
put("add_any_rel", "any", pa1)?;
put("add_any_nmf", "any", pan)?;
put("p_nmf", "any", q)?;
if let Some(r) = pr { put("p_release", "any", r)?; }
for c in 0..C {
put("add", CHANNELS[c], mix(pc1[c], pc0[c]))?;
+9 -4
View File
@@ -187,6 +187,8 @@ fn fold(e: &mut Value, target: &str, ch: &str, prob: Option<f64>, mean: Option<f
"add_any" => { e["add_any"] = r3(prob); }
"add" => { e["add"][ch] = r3(prob); }
"add_any_rel" => { e["add_any_rel"] = r3(prob); }
"add_any_nmf" => { e["add_any_nmf"] = r3(prob); }
"p_nmf" => { e["p_nmf"] = r3(prob); }
"add_rel" => { if !e["add_rel"].is_object() { e["add_rel"] = json!({}); } e["add_rel"][ch] = r3(prob); }
"p_release" => { e["p_release"] = r3(prob); }
"now" => { e["now"][ch] = json!({"on": prob.unwrap_or(0.0) > 0.5, "plays": mean}); }
@@ -234,7 +236,7 @@ fn rank(st: &State, h: usize) -> Result<Reply> {
}
let mut rows: HashMap<String, Value> = HashMap::new();
let mut st_ = c.prepare("SELECT key, target, prob, mean, lo, hi, target_week FROM forecasts WHERE made_at = ?1 AND model_version = ?2
AND ((target IN ('rotation', 'any') AND horizon = ?3) OR (target IN ('add', 'add_any', 'add_rel', 'add_any_rel', 'p_release') AND horizon = ?4) OR target = 'now')")?;
AND ((target IN ('rotation', 'any') AND horizon = ?3) OR (target IN ('add', 'add_any', 'add_rel', 'add_any_rel', 'add_any_nmf', 'p_nmf', 'p_release') AND horizon = ?4) OR target = 'now')")?;
let it = st_.query_map(params![made, VERSION, h.max(1) as i64, h as i64], |r| Ok((r.get::<_, String>(0)?, r.get::<_, String>(1)?, r.get::<_, Option<f64>>(2)?,
r.get::<_, Option<f64>>(3)?, r.get::<_, Option<f64>>(4)?, r.get::<_, Option<f64>>(5)?, r.get::<_, String>(6)?)))?;
let mut window = String::new();
@@ -299,7 +301,7 @@ fn artist(st: &State, id: &str) -> Result<Reply> {
let ch = key.split_once(':').map(|x| x.1).unwrap_or("");
if target == "now" { fold(&mut now, "now", ch, prob, mean, lo, hi); continue; }
let e = byh.entry(h).or_insert_with(|| json!({"h": h, "p": {}, "plays": {}, "add": {}}));
if target.starts_with("add") || target == "p_release" { e["window_start"] = json!(week); } else { e["week"] = json!(week); }
if target.starts_with("add") || target.starts_with("p_") { e["window_start"] = json!(week); } else { e["week"] = json!(week); }
fold(e, &target, ch, prob, mean, lo, hi);
}
now_out = now["now"].clone();
@@ -357,11 +359,13 @@ fn whatif(st: &State, q: &HashMap<String, String>) -> Result<Reply> {
let (t40, air) = (num("t40"), num("air"));
// The scenario: a release in the window (default) or none.
let release = q.get("release").map(|v| v != "no").unwrap_or(true);
// ...and whether that release is placed on New Music Friday Denmark (default: no).
let nmf = q.get("nmf").is_some_and(|v| v == "yes");
let over = move |x: &mut [f64], idx: [usize; 4]| set_chart(x, idx, entry, t40, air);
let run = |overrides: bool| -> Value {
let f: Option<&dyn Fn(&mut [f64], [usize; 4])> = if overrides { Some(&over) } else { None };
let (add_c, add_any) = am.predict(p, &m.rot, a, t0, Some(release), f);
let (add_c, add_any) = am.predict(p, &m.rot, a, t0, crate::model::adds::Scenario::new(release, release && nmf), f);
let mut x = vec![0.0; P];
let mut mu = [0.0; C];
for c in 0..C {
@@ -383,7 +387,8 @@ fn whatif(st: &State, q: &HashMap<String, String>) -> Result<Reply> {
};
let now: serde_json::Map<String, Value> = (0..C).filter(|&c| m.rot.on(a, c, t0)).map(|c| (CHANNELS[c].to_string(), json!(p.y(a, c, t0)))).collect();
ok(json!({
"h": h, "release": release, "p_release": crate::model::adds::p_release(p, a, t0),
"h": h, "release": release, "nmf": nmf, "p_release": crate::model::adds::p_release(p, a, t0), "p_nmf": crate::model::adds::p_nmf(p, a, t0),
"nmf_history": p.nmf_dk[a].iter().rev().take(8).map(|(d, pos)| json!({"friday": d.to_string(), "position": pos})).collect::<Vec<_>>(),
"window_start": (p.weeks[t0] + chrono::Duration::weeks(h as i64 + 1)).to_string(),
"current": {"t40": chart.t40_rank, "air": chart.air_rank,
"entry": if chart.weeks_since_entry == u16::MAX { Value::Null } else { json!(chart.weeks_since_entry) },
+4 -3
View File
@@ -335,7 +335,7 @@ async function showArtist(id) {
<div class="kpi"><div>Any DR station</div><b>${pct(fc.add_any)}</b></div>
${GROUPS.map(([g, l]) => `<div class="kpi"><div>${l}</div><b>${pct(chanP(fc, g))}</b></div>`).join('')}
</div>
${fc.p_release != null ? `<p class="note">Chance they release something in the window (from their release rate): <b>${pct(fc.p_release)}</b>. If they release: ${pct(fc.add_any_rel)} · at their usual rate: ${pct(fc.add_any)}.</p>` : '<p class="note">No discography found on Deezer for this artist, so release timing isn\'t in the forecast.</p>'}
${fc.p_release != null ? `<p class="note">Chance they release something in the window (from their release rate): <b>${pct(fc.p_release)}</b>. If they release: ${pct(fc.add_any_rel)} · at their usual rate: ${pct(fc.add_any)}.${fc.add_any_nmf != null ? ` If the release is placed on New Music Friday Denmark: <b>${pct(fc.add_any_nmf)}</b> (their releases were placed about ${pct(fc.p_nmf)} of the time).` : ''}</p>` : '<p class="note">No discography found on Deezer for this artist, so release timing isn\'t in the forecast.</p>'}
<p class="note">In rotation now: ${nowChips(a.now, Object.fromEntries(Object.entries(a.bauer || {}).map(([s, v]) => [s, v.plays])))} · plays in the week of ${fp.week ? fmtDate(fp.week) : '—'}: <b>${Math.round(fp.mean ?? 0)}</b> (${Math.round(fp.lo ?? 0)}–${Math.round(fp.hi ?? 0)})</p>
</div>
<div class="sec"><h3>Weekly plays, last 51 weeks and next 12</h3>${historyChart(a)}</div>
@@ -352,6 +352,7 @@ function whatIfUI(a, h) {
box.innerHTML = `<div class="wi">
<div class="row"><label for="wi-h">Window starts in (weeks)</label><input type="range" id="wi-h" min="0" max="${meta.horizons.length}" value="${h}"><b id="wi-hv"></b></div>
<div class="row"><label for="wi-r">Releases a song in the window</label><input type="checkbox" id="wi-r" checked><b></b></div>
<div class="row"><label for="wi-n">…placed on New Music Friday Denmark</label><input type="checkbox" id="wi-n"><b></b></div>
<div class="row"><label for="wi-e">New chart entry, weeks ago</label><input type="range" id="wi-e" min="-1" max="26" value="-1"><b id="wi-ev"></b></div>
<div class="row"><label for="wi-t">Track Top-40 position now</label><input type="range" id="wi-t" min="0" max="40" value="0"><b id="wi-tv"></b></div>
<div class="row"><label for="wi-a">Airplay Top-20 position now</label><input type="range" id="wi-a" min="0" max="20" value="0"><b id="wi-av"></b></div>
@@ -368,7 +369,7 @@ function whatIfUI(a, h) {
const run = async () => {
label();
let qs = `id=${encodeURIComponent(a.id)}&h=${$('wi-h').value}`;
qs += `&release=${$('wi-r').checked ? 'yes' : 'no'}`;
qs += `&release=${$('wi-r').checked ? 'yes' : 'no'}&nmf=${$('wi-n').checked ? 'yes' : 'no'}`;
if (init) qs += `&entry=${$('wi-e').value < 0 ? 'none' : $('wi-e').value}&t40=${$('wi-t').value}&air=${$('wi-a').value}`;
const r = await fetch(BASE + '/api/whatif?' + qs);
if (r.status === 503) { $('wi-out').innerHTML = '<p class="note">The model is still loading on the server; try again in a minute.</p>'; return; }
@@ -387,7 +388,7 @@ function whatIfUI(a, h) {
<div class="kpis">${rows.map(([l, b, s]) => `<div class="kpi"><div>${l}</div><b>${pct(s)}</b>${d(b, s, 'p')}<div>was ${pct(b)}</div></div>`).join('')}
<div class="kpi"><div>Plays, first week</div><b>${Math.round(w.scenario.mean)}</b>${d(w.base.mean, w.scenario.mean, 'n')}<div>was ${Math.round(w.base.mean)}</div></div></div>`;
};
['wi-h', 'wi-r', 'wi-e', 'wi-t', 'wi-a'].forEach(id => $(id).addEventListener('input', () => { label(); clearTimeout(timer); timer = setTimeout(run, 120); }));
['wi-h', 'wi-r', 'wi-n', 'wi-e', 'wi-t', 'wi-a'].forEach(id => $(id).addEventListener('input', () => { label(); clearTimeout(timer); timer = setTimeout(run, 120); }));
run();
}