Knowable pool (artists enter only from their first release/play/chart/list/NMF), pooled out-of-sample recalibration (Platt vs isotonic; the site uses pooled Platt per channel, isotonic any channel), release-timing baseline, 8-week bootstrap blocks, surge-rule precision/recall vs the lists with a constant grid (docs/ADDCHECK.md); backtests restructured to score all variants with calibration tables
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,23 @@
|
||||
# Add definitions vs DR's own lists
|
||||
|
||||
Truth: artists with a song new on a list vs the previous list of the channel. Recall over all list pairs (24), and recall and precision over pairs at most 8 weeks apart (15). Precision is a lower bound: songs added and dropped between lists, or added off-list, count against it.
|
||||
|
||||
| Definition | Recall, all pairs | Recall, close pairs | Precision, close pairs | F1, close pairs | Adds detected | New songs |
|
||||
|---|---|---|---|---|---|---|
|
||||
| onset | 64% | 55% | 22% | 0.32 | 351 | 177 |
|
||||
| surge [0.6, 0.5, 0.5, 0.25] | 77% | 68% | 25% | 0.36 | 420 | 177 |
|
||||
|
||||
Surge constants tried (floor × τ, jump × τ, jump × baseline, held next week), best F1 on close pairs:
|
||||
|
||||
| Constants | Recall | Precision | F1 |
|
||||
|---|---|---|---|
|
||||
| [0.4, 0.3, 1.0, 0.5] | 65% | 26% | 0.369 |
|
||||
| [0.4, 0.3, 0.5, 0.5] | 68% | 25% | 0.368 |
|
||||
| [0.6, 0.3, 1.0, 0.5] | 64% | 26% | 0.368 |
|
||||
| [0.4, 0.3, 0.25, 0.5] | 69% | 25% | 0.367 |
|
||||
| [0.6, 0.5, 0.25, 0.25] | 70% | 25% | 0.365 |
|
||||
| [0.6, 0.3, 0.5, 0.5] | 67% | 25% | 0.365 |
|
||||
| [0.6, 0.5, 0.25, 0.5] | 68% | 25% | 0.365 |
|
||||
| [0.6, 0.3, 0.25, 0.5] | 69% | 25% | 0.364 |
|
||||
|
||||
The grid's best F1 (0.369) is within noise of the current constants (0.36), so the defaults stay. Surge beats onset-only on both recall and precision.
|
||||
Reference in New Issue
Block a user