You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
Music/docs/superpowers/plans/2026-05-30-fix-zero-bitrate.md

607 lines
24 KiB

# Fix `bitrate = 0` Tracks Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Stop the importer from ever storing a bitrate of `0`, and backfill the real bitrate onto existing tracks where it is `0` or `NULL`.
**Architecture:** Two independent pieces. (1) A pure, unit-tested Swift function `ScannerService.resolveBitrate` that derives kbps from AVFoundation's estimate with a file-size/duration fallback, returning `nil` (never `0`) when nothing is derivable; wired into `extractMetadata`. (2) A stdlib-only Python backfill script `scripts/backfill_bitrate.py` (modeled on the existing `backfill_itunes_dates.py`) that recomputes bitrate via `ffprobe`, falling back to the same formula, with dry-run default and `--apply` + timestamped backup.
**Tech Stack:** Swift (AVFoundation, swift-testing), Python 3 stdlib (`sqlite3`, `subprocess`), `ffprobe` (optional external tool).
**Project conventions to respect:**
- User's global rule: **never auto-commit.** Each "Commit checkpoint" step means *stop and run the `/commit` skill* — do not run `git commit` directly.
- Tests use swift-testing (`import Testing`, `@Test`, `#expect`), one struct per file, every test carries a step-by-step comment.
- Swift static utilities in `ScannerService` are `nonisolated static`.
**Spec:** `docs/superpowers/specs/2026-05-30-fix-zero-bitrate-design.md`
---
### Task 1: `ScannerService.resolveBitrate` pure function (Swift, TDD)
**Files:**
- Test: `MusicTests/ScannerServiceTests.swift` (add tests to the existing `ScannerServiceTests` struct)
- Modify: `Music/Services/ScannerService.swift` (add the static function)
- [ ] **Step 1: Write the failing tests**
Add these five tests inside the existing `struct ScannerServiceTests { ... }` in `MusicTests/ScannerServiceTests.swift`, just before the closing brace:
```swift
// Verifies resolveBitrate uses the OS estimate when it is positive.
// 1. Passes a positive estimatedDataRate in bits/sec (320450).
// 2. Expects it rounded to kbps (320450/1000 = 320.45 -> 320), ignoring size/duration.
@Test func resolveBitrateUsesEstimateWhenPositive() {
let kbps = ScannerService.resolveBitrate(estimatedDataRate: 320_450,
fileSizeBytes: 5_000_000,
durationSeconds: 200)
#expect(kbps == 320)
}
// Verifies the size/duration fallback when the OS estimate is 0 (the AVFoundation bug).
// 1. Passes estimatedDataRate 0 with a real file size and duration.
// 2. Expects 230_358_479 * 8 / 7198.54 / 1000 -> ~256.0 -> 256 kbps (matches ffprobe).
@Test func resolveBitrateFallsBackToSizeAndDuration() {
let kbps = ScannerService.resolveBitrate(estimatedDataRate: 0,
fileSizeBytes: 230_358_479,
durationSeconds: 7198.5371428571425)
#expect(kbps == 256)
}
// Verifies nil (never 0) when the estimate is 0 and duration is unusable.
// 1. Zero duration cannot yield a value -> nil.
// 2. A NaN duration (CMTimeGetSeconds can return NaN) is also nil, not 0.
@Test func resolveBitrateReturnsNilWhenNoDuration() {
#expect(ScannerService.resolveBitrate(estimatedDataRate: 0,
fileSizeBytes: 230_358_479,
durationSeconds: 0) == nil)
#expect(ScannerService.resolveBitrate(estimatedDataRate: 0,
fileSizeBytes: 230_358_479,
durationSeconds: .nan) == nil)
}
// Verifies nil when the estimate is 0 and there is no file size.
// 1. Missing fileSizeBytes with estimate 0 -> nil (never 0).
@Test func resolveBitrateReturnsNilWhenNoFileSize() {
#expect(ScannerService.resolveBitrate(estimatedDataRate: 0,
fileSizeBytes: nil,
durationSeconds: 200) == nil)
}
// Verifies the core invariant: no input combination ever yields 0.
// 1. All-zero inputs return nil so the UI renders "—" instead of "0 kbps".
@Test func resolveBitrateNeverReturnsZero() {
#expect(ScannerService.resolveBitrate(estimatedDataRate: 0,
fileSizeBytes: 0,
durationSeconds: 0) == nil)
}
```
- [ ] **Step 2: Run the tests to verify they fail**
Run:
```bash
xcodebuild test -project /Users/laurentmorvillier/code/Music/Music.xcodeproj \
-scheme Music -destination 'platform=macOS' \
-only-testing:MusicTests/ScannerServiceTests 2>&1 | tail -30
```
Expected: **compile failure**`type 'ScannerService' has no member 'resolveBitrate'`.
(If the destination errors, list options with `xcodebuild -showdestinations -project Music.xcodeproj -scheme Music` and use the macOS one.)
- [ ] **Step 3: Write the minimal implementation**
In `Music/Services/ScannerService.swift`, add this method right after the `discoverAudioFiles` static function (after line 35, inside the class):
```swift
/// Resolve a track's bitrate in kbps from the OS estimate, falling back to a
/// size/duration average. Returns nil when nothing can be derived — never 0,
/// so the UI shows "—" instead of a meaningless "0 kbps".
///
/// AVFoundation's `estimatedDataRate` returns 0 for some files (observed on
/// long/VBR MP3s); for those we compute the true average bitrate from the
/// file size and duration, which matches ffprobe to the kbps.
nonisolated static func resolveBitrate(estimatedDataRate: Double,
fileSizeBytes: Int64?,
durationSeconds: Double?) -> Int? {
if estimatedDataRate > 0 {
return Int((estimatedDataRate / 1000).rounded())
}
// NaN-safe: `dur > 0` is false for .nan, so we return nil rather than 0.
if let size = fileSizeBytes, size > 0,
let dur = durationSeconds, dur > 0 {
return Int((Double(size) * 8 / dur / 1000).rounded())
}
return nil
}
```
- [ ] **Step 4: Run the tests to verify they pass**
Run:
```bash
xcodebuild test -project /Users/laurentmorvillier/code/Music/Music.xcodeproj \
-scheme Music -destination 'platform=macOS' \
-only-testing:MusicTests/ScannerServiceTests 2>&1 | tail -30
```
Expected: **TEST SUCCEEDED**, all `ScannerServiceTests` pass (the new five plus the existing `discoverAudioFiles`).
- [ ] **Step 5: Commit checkpoint**
Stop and run the `/commit` skill (do not `git commit` directly). Suggested message: `feat: add ScannerService.resolveBitrate with size/duration fallback`.
---
### Task 2: Wire `resolveBitrate` into `extractMetadata`
**Files:**
- Modify: `Music/Services/ScannerService.swift:146-162` (the bitrate block inside `extractMetadata`)
- [ ] **Step 1: Move the file-stats computation above the bitrate block**
In `extractMetadata`, `TrackFileStats.compute` currently runs at line 162, *after* the bitrate block. Move it up so its `fileSize` is available to `resolveBitrate`. Replace the current block that spans from the duration load through the bitrate computation (lines 146-160):
```swift
let duration = try await asset.load(.duration)
let durationSeconds = CMTimeGetSeconds(duration)
var bitrate: Int?
var sampleRate: Int?
if let audioTrack = try await asset.loadTracks(withMediaType: .audio).first {
let estimatedRate = try await audioTrack.load(.estimatedDataRate)
bitrate = Int(estimatedRate / 1000)
let descriptions = try await audioTrack.load(.formatDescriptions)
if let desc = descriptions.first {
if let asbd = CMAudioFormatDescriptionGetStreamBasicDescription(desc) {
sampleRate = Int(asbd.pointee.mSampleRate)
}
}
}
```
with this:
```swift
let duration = try await asset.load(.duration)
let durationSeconds = CMTimeGetSeconds(duration)
let stats = try TrackFileStats.compute(for: url)
var bitrate: Int?
var sampleRate: Int?
if let audioTrack = try await asset.loadTracks(withMediaType: .audio).first {
let estimatedRate = try await audioTrack.load(.estimatedDataRate)
bitrate = Self.resolveBitrate(estimatedDataRate: estimatedRate,
fileSizeBytes: stats.fileSize,
durationSeconds: durationSeconds)
let descriptions = try await audioTrack.load(.formatDescriptions)
if let desc = descriptions.first {
if let asbd = CMAudioFormatDescriptionGetStreamBasicDescription(desc) {
sampleRate = Int(asbd.pointee.mSampleRate)
}
}
} else {
// No audio track loaded — still attempt the size/duration fallback
// so we never silently lose the bitrate.
bitrate = Self.resolveBitrate(estimatedDataRate: 0,
fileSizeBytes: stats.fileSize,
durationSeconds: durationSeconds)
}
```
- [ ] **Step 2: Remove the now-duplicate `TrackFileStats.compute` call**
The original line 162 `let stats = try TrackFileStats.compute(for: url)` (now appearing just above the `return Track(` call) is a duplicate — delete that single line. The `Track(...)` initializer still references `stats.fileSize`, `stats.dateModified`, and `stats.fileHash`, which now come from the moved-up computation. Confirm exactly one `let stats = try TrackFileStats.compute(for: url)` remains in the function.
- [ ] **Step 3: Build to verify it compiles**
Run:
```bash
xcodebuild build -project /Users/laurentmorvillier/code/Music/Music.xcodeproj \
-scheme Music -destination 'platform=macOS' 2>&1 | tail -15
```
Expected: **BUILD SUCCEEDED**.
- [ ] **Step 4: Re-run the full test target to confirm no regressions**
Run:
```bash
xcodebuild test -project /Users/laurentmorvillier/code/Music/Music.xcodeproj \
-scheme Music -destination 'platform=macOS' \
-only-testing:MusicTests 2>&1 | tail -20
```
Expected: **TEST SUCCEEDED** for the whole `MusicTests` target.
- [ ] **Step 5: Commit checkpoint**
Stop and run the `/commit` skill. Suggested message: `fix: importer derives bitrate via resolveBitrate instead of storing 0`.
---
### Task 3: Backfill script — pure helpers + self-test (Python, TDD)
**Files:**
- Create: `scripts/backfill_bitrate.py`
- [ ] **Step 1: Create the script with helpers and a failing self-test**
Create `scripts/backfill_bitrate.py` with exactly this content:
```python
#!/usr/bin/env python3
"""One-time backfill of real bitrate onto tracks stored with bitrate 0 or NULL.
ScannerService writes `bitrate = Int(estimatedDataRate / 1000)` at scan time.
AVFoundation's estimatedDataRate returns 0 for some files (long/VBR MP3s), so a
literal 0 gets stored; other tracks were imported before bitrate existed and are
NULL. This script recomputes bitrate for those rows using ffprobe, falling back
to fileSize*8/duration (the same average the app's importer now uses) when
ffprobe is unavailable or can't determine a value.
Dry-run by default. Pass --apply to write (a timestamped backup is made first).
Usage:
python3 backfill_bitrate.py [--db <path>] [--apply]
python3 backfill_bitrate.py --self-test
Stdlib only; uses ffprobe if present on PATH (optional).
"""
import argparse
import os
import shutil
import sqlite3
import subprocess
import sys
import unicodedata
from datetime import datetime
from urllib.parse import unquote
# Default DB path for the sandboxed app (bundle id com.staxriver.mu). Computed from
# $HOME so it resolves to the right user on whichever Mac the script runs on.
DEFAULT_DB = os.path.expanduser(
"~/Library/Containers/com.staxriver.mu/Data/Library/"
"Application Support/Music/db.sqlite"
)
def norm_path(u):
"""Reduce a file:// URL (or bare path) to a comparable, on-disk POSIX path.
The app stores `fileURL` as Foundation's url.absoluteString (a percent-encoded
file URL). Decode it, drop the file:// (or file://localhost) prefix, NFC-
normalize, and strip a trailing slash so it can be stat'd on APFS.
"""
s = u
if s.startswith("file://"):
s = s[len("file://"):]
if s.startswith("localhost/"):
s = s[len("localhost"):] # leaves the leading "/"
s = unquote(s)
s = unicodedata.normalize("NFC", s)
if len(s) > 1 and s.endswith("/"):
s = s[:-1]
return s
def parse_ffprobe_bitrate(stdout):
"""Parse ffprobe's bit_rate stdout (bits/sec) into integer kbps, or None.
Returns None for empty output, 'N/A', or any non-integer text so the caller
falls back to the formula.
"""
s = stdout.strip()
if not s or s == "N/A":
return None
try:
return round(int(s) / 1000)
except ValueError:
return None
def kbps_from_ffprobe(path):
"""Return integer kbps from ffprobe's format bit_rate, or None if unavailable.
None on: ffprobe not installed, ffprobe error, or N/A/empty/non-integer output.
"""
try:
out = subprocess.run(
["ffprobe", "-v", "error", "-show_entries", "format=bit_rate",
"-of", "default=nw=1:nk=1", path],
capture_output=True, text=True, timeout=30,
)
except (FileNotFoundError, subprocess.SubprocessError):
return None
return parse_ffprobe_bitrate(out.stdout)
def kbps_from_formula(file_size, duration):
"""Average kbps from size (bytes) and duration (seconds): size*8/duration/1000.
Returns None when inputs can't yield a meaningful value (missing size, or
non-positive/missing duration).
"""
if not file_size or not duration or duration <= 0:
return None
return round(file_size * 8 / duration / 1000)
def resolve_bitrate(path, duration):
"""Best available kbps for an on-disk file: ffprobe first, formula fallback.
`duration` is the DB's stored seconds; file size is read from disk. Returns
None if neither method can produce a positive value.
"""
kbps = kbps_from_ffprobe(path)
if kbps and kbps > 0:
return kbps
try:
size = os.path.getsize(path)
except OSError:
size = None
return kbps_from_formula(size, duration)
def ffprobe_available():
return shutil.which("ffprobe") is not None
def self_test():
"""Fast smoke check of the pure helpers (no DB, no ffprobe needed)."""
# ffprobe stdout parsing
assert parse_ffprobe_bitrate("256005\n") == 256
assert parse_ffprobe_bitrate("N/A") is None
assert parse_ffprobe_bitrate("") is None
assert parse_ffprobe_bitrate("garbage") is None
# formula: 230_358_479 bytes over 7198.54 s -> 256 kbps (matches ffprobe sample)
assert kbps_from_formula(230_358_479, 7198.5371428571425) == 256
assert kbps_from_formula(None, 100) is None
assert kbps_from_formula(1000, 0) is None
assert kbps_from_formula(1000, None) is None
# path normalization (NFD vs NFC accents, percent-encoding, localhost host)
nfc = norm_path("file:///Users/x/Mu%CC%81sica/Cafe%CC%81.mp3")
nfd = norm_path("file://localhost/Users/x/M%C3%BAsica/Caf%C3%A9.mp3")
assert nfc == nfd == "/Users/x/Música/Café.mp3", (nfc, nfd)
assert norm_path("file:///a/b%20c%23d.mp3") == "/a/b c#d.mp3"
print("self-test OK")
def main(argv=None):
p = argparse.ArgumentParser(description=__doc__)
p.add_argument("--db", default=DEFAULT_DB, help=f"App DB path (default: {DEFAULT_DB})")
p.add_argument("--apply", action="store_true", help="Write changes (default: dry run).")
p.add_argument("--self-test", action="store_true", help="Run the built-in smoke test.")
args = p.parse_args(argv)
if args.self_test:
self_test()
return 0
if not os.path.exists(args.db):
p.error(f"DB not found: {args.db}")
run(args.db, args.apply)
return 0
if __name__ == "__main__":
sys.exit(main())
```
Note: `run(...)` is referenced by `main` but not yet defined — Task 4 adds it. The self-test does not call `run`, so `--self-test` works now.
- [ ] **Step 2: Run the self-test to verify the helpers pass**
Run:
```bash
python3 /Users/laurentmorvillier/code/Music/scripts/backfill_bitrate.py --self-test
```
Expected: `self-test OK`.
- [ ] **Step 3: Verify the dry-run path fails cleanly (run undefined)**
Run:
```bash
python3 /Users/laurentmorvillier/code/Music/scripts/backfill_bitrate.py --db /nonexistent.sqlite
```
Expected: argparse error `DB not found: /nonexistent.sqlite` (exit 2) — confirms arg handling before `run` is implemented.
- [ ] **Step 4: Commit checkpoint**
Stop and run the `/commit` skill. Suggested message: `feat: add backfill_bitrate.py helpers + self-test`.
---
### Task 4: Backfill script — DB wiring, dry-run report, `--apply`
**Files:**
- Modify: `scripts/backfill_bitrate.py` (add `fetch_rows`, `build_updates`, `backup_db`, `apply_updates`, `run`)
- [ ] **Step 1: Add the DB + reporting functions**
Insert these functions into `scripts/backfill_bitrate.py` immediately before `def main(`:
```python
def fetch_rows(db_path):
"""Return candidate rows: (id, fileURL, duration, bitrate) where bitrate is 0/NULL."""
con = sqlite3.connect(db_path)
try:
return con.execute(
"SELECT id, fileURL, duration, bitrate FROM tracks "
"WHERE bitrate = 0 OR bitrate IS NULL"
).fetchall()
finally:
con.close()
def build_updates(rows):
"""Resolve a new bitrate for each candidate row.
Returns (updates, missing, undeterminable):
- updates: list of {id, file_url, old, new} where new is a positive kbps
- missing: (id, path) for rows whose file is not on disk (left untouched)
- undeterminable: (id, path) for on-disk files whose bitrate couldn't be found
"""
updates, missing, undeterminable = [], [], []
for row_id, file_url, duration, old in rows:
path = norm_path(file_url)
if not os.path.exists(path):
missing.append((row_id, path))
continue
new = resolve_bitrate(path, duration)
if not new or new <= 0:
undeterminable.append((row_id, path))
continue
updates.append({"id": row_id, "file_url": file_url, "old": old, "new": new})
return updates, missing, undeterminable
def backup_db(db_path):
"""Copy db.sqlite (+ -wal, -shm) under backups/<timestamp>/ next to the DB."""
stamp = datetime.now().strftime("%Y%m%d-%H%M%S")
backup_dir = os.path.join(os.path.dirname(db_path), "backups", stamp)
os.makedirs(backup_dir, exist_ok=True)
for suffix in ("", "-wal", "-shm"):
src = db_path + suffix
if os.path.exists(src):
shutil.copy2(src, os.path.join(backup_dir, os.path.basename(src)))
return backup_dir
def apply_updates(db_path, updates):
"""Write bitrate updates in a single transaction, then checkpoint the WAL."""
con = sqlite3.connect(db_path)
try:
con.execute("BEGIN")
con.executemany("UPDATE tracks SET bitrate=:new WHERE id=:id", updates)
con.commit()
con.execute("PRAGMA wal_checkpoint(TRUNCATE)")
finally:
con.close()
def run(db_path, apply):
rows = fetch_rows(db_path)
updates, missing, undeterminable = build_updates(rows)
print(f"Candidate rows (bitrate 0 or NULL): {len(rows)}")
print(f"Resolvable (will set): {len(updates)}")
print(f"Skipped — file missing on disk: {len(missing)}")
print(f"Skipped — could not determine: {len(undeterminable)}")
if not ffprobe_available():
print("NOTE: ffprobe not on PATH — used the filesize/duration formula for all rows.")
print()
for u in updates[:15]:
name = os.path.basename(norm_path(u["file_url"]))
old = "NULL" if u["old"] is None else u["old"]
print(f" • {name}")
print(f" bitrate {old} -> {u['new']} kbps")
if len(updates) > 15:
print(f" ... and {len(updates) - 15} more")
print()
if missing[:5]:
print("Sample of skipped (file missing on disk, left untouched):")
for row_id, path in missing[:5]:
print(f" - [{row_id}] {os.path.basename(path)}")
print()
if undeterminable[:5]:
print("Sample of skipped (could not determine bitrate, left untouched):")
for row_id, path in undeterminable[:5]:
print(f" - [{row_id}] {os.path.basename(path)}")
print()
if not apply:
print("DRY RUN — nothing written. Re-run with --apply to commit these changes.")
return
if not updates:
print("Nothing to apply.")
return
backup_dir = backup_db(db_path)
print(f"Backup written to: {backup_dir}")
apply_updates(db_path, updates)
print(f"Applied {len(updates)} bitrate updates to {db_path}")
```
- [ ] **Step 2: Re-run the self-test (ensure the new code didn't break helpers)**
Run:
```bash
python3 /Users/laurentmorvillier/code/Music/scripts/backfill_bitrate.py --self-test
```
Expected: `self-test OK`.
- [ ] **Step 3: Dry-run against a temp DB to verify end-to-end wiring**
This builds a tiny throwaway DB with one real bitrate=0 row, so the run path is exercised without touching the app DB:
```bash
python3 - <<'PY'
import sqlite3, os, tempfile
d = tempfile.mkdtemp()
db = os.path.join(d, "db.sqlite")
con = sqlite3.connect(db)
con.execute("CREATE TABLE tracks (id INTEGER PRIMARY KEY, fileURL TEXT, duration REAL, bitrate INTEGER)")
# A file that does not exist -> should be reported as 'missing', not crash.
con.execute("INSERT INTO tracks (fileURL, duration, bitrate) VALUES (?,?,?)",
("file:///no/such/file.mp3", 100.0, 0))
con.commit(); con.close()
print(db)
PY
```
Take the printed path and run:
```bash
python3 /Users/laurentmorvillier/code/Music/scripts/backfill_bitrate.py --db <printed-path>
```
Expected: a summary with `Candidate rows ... : 1`, `Skipped — file missing on disk: 1`, and `DRY RUN — nothing written.` (no traceback).
- [ ] **Step 4: Commit checkpoint**
Stop and run the `/commit` skill. Suggested message: `feat: backfill_bitrate.py DB wiring, dry-run report, --apply`.
---
### Task 5: Verify against the real library (manual, no code change)
**Files:** none.
- [ ] **Step 1: Dry-run against the real app DB**
Quit the Music app first (avoids WAL/lock contention). Then:
```bash
python3 /Users/laurentmorvillier/code/Music/scripts/backfill_bitrate.py
```
Expected: a non-zero `Resolvable (will set)` count and a sample of `bitrate 0 -> NNN kbps` lines. Eyeball a few values for plausibility (typical 128–320 kbps).
- [ ] **Step 2: Apply**
```bash
python3 /Users/laurentmorvillier/code/Music/scripts/backfill_bitrate.py --apply
```
Expected: `Backup written to: .../backups/<timestamp>` then `Applied N bitrate updates`.
- [ ] **Step 3: Confirm the DB no longer has 0/NULL bitrates (except undeterminable)**
```bash
DB="$HOME/Library/Containers/com.staxriver.mu/Data/Library/Application Support/Music/db.sqlite"
sqlite3 "$DB" "SELECT COUNT(*) AS still_zero_or_null FROM tracks WHERE bitrate = 0 OR bitrate IS NULL;"
```
Expected: `0`, or the small count the dry-run reported as "could not determine"/"file missing".
- [ ] **Step 4: Reopen the app and spot-check**
Open a previously-0 track's Get Info (or the Bit Rate column) and confirm it now shows a real value.
---
## Self-Review Notes
- **Spec coverage:** Root cause + invariant → Tasks 1–2 (`resolveBitrate`, never stores 0). Backfill (0 and NULL, ffprobe→formula, missing-file skip, dry-run/backup/apply) → Tasks 3–4. Manual verification → Task 5. All spec sections covered.
- **Type consistency:** `resolveBitrate(estimatedDataRate:fileSizeBytes:durationSeconds:)` is defined identically in Task 1 and called identically in Task 2; `stats.fileSize` is `Int64` (matches `fileSizeBytes: Int64?`). Python helper names (`parse_ffprobe_bitrate`, `kbps_from_ffprobe`, `kbps_from_formula`, `resolve_bitrate`, `fetch_rows`, `build_updates`, `backup_db`, `apply_updates`, `run`) are defined once and referenced consistently.
- **No placeholders:** every code step shows complete code; every run step shows the command and expected output.