wgpu: per-chunk OOM cooldown + continue past blocked candidates

Fixes two coupled streaming pathologies on working-set > pool scenes:

1) The candidate loop used `break` when a candidate couldn't fit even
after eviction. Comment justified it with "sorted by priority, lower
candidates can't beat it either" — true for *priority* eviction, but
the failure is *size-based fitting*. A 31 MB candidate that doesn't
fit in 24 MB largest-free was starving the entire per-frame budget,
including smaller candidates that would have fit happily. Replaced
with `continue`.

2) Same blocked candidate re-entered the candidate list every frame
forever, spamming `[blocked]` and (worse, on web) paying for the same
byte-range fetch over and over when apply-time OOM happened. Added
`blocked_cooldown_until_frame_idx` on the chunk: when OOM strikes at
enqueue *or* apply, the chunk is skipped from candidate gathering for
~3s. Web-friendly cap of one wasted fetch per 3s per chronic chunk
instead of per-frame. Cooldown expires naturally; if the pool layout
changes within the window (other chunks evicted, fragmentation
coalesces) the chunk re-enters automatically.

Also added eviction-attribution + chunk thrash detection (gated behind
WGPU_STREAM_EVICT_LOG=1) to confirm A→B→A 2-cycles vs simple
sacrificial-victim cycles. Quietened the steady-state stream debug
dump — moved the verbose multi-line "missing/resident/bottom" snapshot
behind WGPU_STREAM_DEEP_DEBUG=1 with a wider 300-frame interval, and
added a single-line `[stream]` health summary every ~5s in interactive
mode. Removed the hardcoded one-off "brace.ifc bracing all
chunks" dump that was investigation scaffolding for a now-closed bug.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Dion Moult
2026-06-01 10:01:10 +10:00
parent 699f22b502
commit e80897886d
2 changed files with 176 additions and 44 deletions
+23
View File
@@ -211,6 +211,29 @@ struct WgpuModelGpuData {
// with load_count >> 1 has been cycling — used by the stream
// debug log (WGPU_STREAM_DEBUG=1) to surface thrash.
uint32_t load_count = 0;
// Eviction attribution — who pushed this chunk out the last
// time? Filled by evict_lowest_priority_than when the chunk is
// unloaded. Read by the cycle-detection logger when this chunk
// re-enters as a candidate so we can spot A→B→A 2-cycles. Zero
// for chunks that were never evicted or were LRU-evicted (the
// latter doesn't have an obvious "evictor" — just a slot
// pressure event).
uint32_t last_evicted_by_model_id = 0;
uint32_t last_evicted_by_chunk_idx = UINT32_MAX;
float last_evicted_by_priority = 0.0f;
// Frame at which this chunk was most recently evicted, so the
// cycle log only fires when re-entry is "soon" (cache thrash)
// rather than "minutes later" (legitimate camera move).
uint64_t last_evicted_frame_idx = 0;
// Cooldown frame: if streaming_frame_idx_ < this, skip the
// chunk in the candidate gather. Set when a candidate is
// blocked OOM (eviction exhausted, still doesn't fit) OR when
// applyStreamedChunk fails on the drained worker result. Caps
// web bandwidth waste at one fetch per cooldown for chunks
// that genuinely can't fit in the current pool state; the
// cooldown expires naturally so the chunk re-enters when
// pool layout has had a chance to change.
uint64_t blocked_cooldown_until_frame_idx = 0;
// Per-frame instance-aware priority. Sum of px² projected
// contributions of every instance owned by this chunk —
// captures the chunk's actual on-screen footprint, not the