mirror of
https://github.com/IfcOpenShell/IfcOpenShell.git
synced 2026-09-20 06:58:56 +00:00
ifcviewer-web: stop large-model network streaming thrash (grow before fetch)
Battle-testing real sidecars over HTTP Range exposed severe thrash: a
531 MB model re-fetched 2.25 GB (4×) and never converged — viewAll puts
the whole model in frustum, so every chunk wants to be resident, and the
web async path made it worse two ways:
- A web load only consumes pool space when it COMPLETES (async), so the
per-frame issuance over-committed the pool; completions then failed
applyStreamedChunk on a full pool, the chunk re-candidated with no
cooldown, and re-fetched every frame.
- Pool growth is itself async on web (provisional sub-buffers validated
off the JS event loop), so even fetched chunks failed to alloc until
the pool caught up, and re-fetched.
Fix: gate web chunk issuance on VALIDATED free space + in-flight
reservation, and grow the pool BEFORE fetching:
- streaming_web_inflight_bytes_ reserves each in-flight load's footprint
so we never have more bytes in flight than the pool can place.
- When a visible chunk doesn't fit validated free, don't fetch — call
pool_.requestGrowth() (BufferPool: drives the async provisional grow
without allocating) and short-back-off; the chunk is fetched once,
after space exists. When the pool is saturated (model > GPU memory),
long-cooldown so a never-fitting chunk isn't re-fetched. Gating before
the evictor also kills phase-2 visible↔visible swap thrash.
- On async load failure, cool down (short if the pool can still grow,
long if saturated) instead of re-candidating next frame.
Result (manual battle tool, host.mjs + real files): 531 MB now loads
23/23 chunks, 322 MB loads 14/14 — resident climbs monotonically with
ZERO thrash warnings and a stable resident set, vs the old re-fetch loop.
The whole model resides on the GPU and stays. (Remaining ~3× ramp
over-fetch — per-chunk re-loads during the async-growth ramp + read
amplification from chunk byte-locality — is a separate efficiency
follow-up, not thrash.) 6/6 web smoke + 107/107 unit pass; desktop
unaffected (the gate is web-only; requestGrowth is a no-op wrapper there).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -111,6 +111,13 @@ public:
|
||||
// could rescue them, or whether eviction is the only path.
|
||||
bool can_grow() const { return !growth_disabled_ && per_sub_buffer_capacity_ > 0; }
|
||||
|
||||
// Proactively add a sub-buffer (no allocation). On web this kicks off the
|
||||
// async provisional-validation cycle so validated free space appears a
|
||||
// frame or two later — letting the streaming driver grow the pool BEFORE
|
||||
// fetching a chunk's bytes, instead of fetching, failing the alloc on a
|
||||
// not-yet-grown pool, and re-fetching. No-op if growth is pending/disabled.
|
||||
bool requestGrowth() { return addSubBuffer(); }
|
||||
|
||||
// Test-only seam. Production code populates sub-pools lazily through
|
||||
// alloc() → addSubBuffer() → wgpuDeviceCreateBuffer; that path needs a
|
||||
// real WGPUDevice and is impractical to exercise from a unit test.
|
||||
|
||||
Reference in New Issue
Block a user