Many Vulkan drivers cap a single VkDeviceMemory allocation at exactly
maxStorageBufferBindingSize (NVIDIA: 2 GB on consumer GeForce) or
refuse big contiguous allocations once heap is fragmented. The old
addSubBuffer gave up at the first refusal, latching growth_disabled_
— so on a 4 GB GeForce we extracted 2 GB and called it done.
The wgpu-mem-probe tool (feab05650) showed the driver actually grants
~3 GB total across multiple sub-buffers — invariant under allocation
pattern (2+1+small, 3×1 GB, 6×512 MB, 12×256 MB all land at 3 GB).
The cap is the hardware/desktop, not the request size.
addSubBuffer now starts at last_growth_size_ (initially
per_sub_buffer_capacity_, decays as the driver refuses larger sizes)
and halves on failure inside a single call. Stops at a 64 MB floor;
below that the per-sub-buffer bookkeeping cost (free list, bind
groups) isn't worth it. growth_disabled_ now latches only when even
64 MB is refused — a true hardware ceiling, not just "the first
attempt didn't fit."
pool_can_fit gains a next_growth_size_bytes() accessor to stay
honest about how big a future sub-buffer can be after the driver
has refused larger sizes.
Measured (big federation, --streaming, close camera):
pool capacity: 2048 MB → 2688 MB (2 GB + 512 MB + 128 MB)
VRAM resident: 2155 MB → 2800 MB (whole scene fits, no eviction)
avg fps: 42 → 53
stream time: 2.5 ms → 0.1 ms (no churn — working set is stable)
On larger GPUs (8 / 16 / 24 GB workstations) the same code extracts
proportionally more (e.g. 4 × 2 GB on a 10 GB+ card).
The GL backend's higher "4+ GB resident" claim is overcommit into
host RAM — explicit Vulkan/wgpu memory management deliberately
doesn't paper over that, and the wgpu-mem-probe data confirms it
isn't recoverable.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Five interlocking fixes that take --streaming on the big federation
scene from "5 fps + endless flicker + infinite cold-load" to a
stable 35-49 fps with a converged working set.
1. Multi-sub-buffer WgpuBufferPool. Pool now grows lazily by adding
sub-buffers of per_sub_buffer_capacity_ when alloc demand exceeds
existing free runs. Each Slice carries (buffer, offset, size,
sub_idx). On driver refusal of addSubBuffer, growth_disabled_
latches so subsequent allocs don't keep retrying and log-spamming.
pool_can_fit consults can_grow() to know when growth could rescue
a candidate vs when eviction is the only path.
2. Split cull / stream benchmark timers. The previous "cull[wall]"
metric was actually cull + driveStreamingLoads, blaming the wrong
subsystem (~170 ms of "cull" was synchronous disk I/O).
3. frustum_visible_count on Chunk, populated in cullModelCpuCompute
right after the per-instance aabbInFrustum check. driveStreamingLoads
now keys residency on this instead of total_visible_draws (which
includes contribution + HiZ). HiZ visibility flips frame-to-frame
as occluders shift; using it for residency caused chunks to be
evicted then immediately re-loaded, every frame, even with a
stationary camera — both the perf cliff and the visible flicker.
4. Distance-sorted candidates in driveStreamingLoads. Walk the
non-resident frustum-visible chunks in distance order (closest
first). With sorted processing, evict_farthest_than converges
monotonically: each swap replaces a far resident with a closer
candidate; once the next candidate is farther than every
remaining resident, the loop exits. Without sorting the loader
visited candidates in model/chunk-id order, swapping random
chunks every frame without ever converging.
5. 10% eviction hysteresis (EVICT_DIST2_RATIO = 1.21). On scenes
where many chunks are clustered at similar distance from the
camera (e.g. several chunks all ~370 m away), naive
"evict any resident strictly farther than candidate" triggers
sub-meter swaps every frame, never resting. Requiring the victim
to be 10% farther in linear distance kills these cycles while
still allowing genuine "much closer" candidates to evict.
Plus: latched bench_warm_done_ on the cold-load gate, with a
5-frames-of-zero-loads convergence test (default-camera big scene
converges in 20 frames) and a 600-frame timeout fallback that prints
exactly once.
Measured on the test federation (111 sidecars, ~3 GB raw, 1 M
instances) with the user's close-in camera:
- avg 35 fps (was 5), median 49 fps (was 7)
- cull 19 ms (now the bottleneck), stream 5-8 ms (was 172)
- p99 184 ms — occasional big-chunk load on the render thread;
background-thread I/O would smooth that out as a follow-up.
With the default wide camera:
- avg 40 fps, converges in 20 frames, residency grows naturally
from 59 → 76 chunks as orbit shifts the frustum.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Drops the per-machine "guess the OOM ceiling" budget knob in favour of
a single buffer pool whose capacity is *probed* at device-init time.
The runtime answers the question: descend from min(maxBufferSize, 4 GB)
through OOM error scopes, accept the largest size that allocates
cleanly. On a desktop wgpu-native v29 box this lands at 2 GB; on
browser-class platforms it'll land at 256 MB – 1 GB depending on the
implementation. Same code path either way.
Architecture:
- WgpuBufferPool (new): single WGPUBuffer + free-list sub-allocator
with adjacent-range coalescing and first-fit. 256 B alignment for
storage-binding offsets.
- Chunks now hold (pool_vertex_offset, pool_vertex_size) and
(pool_index_offset, pool_index_size) instead of per-chunk WGPUBuffer
handles. Load = pool.alloc + queueWriteBuffer. Unload = pool.free.
- Bind groups bind pool_.buffer() at the chunk's specific (offset, size)
for both the vertex and index storage bindings.
- Eviction queries pool.largest_free_run_bytes() instead of a tracked
budget; the two-phase LRU/distance evictor's policy is unchanged.
What this fixes:
- No more gpu-alloc-rs fragmentation OOM: one VkDeviceMemory block
instead of N per-chunk blocks with rounding overhead. On the test
dataset (~3 GB on disk, 562 k visible instances) the wgpu backend
now runs through to render without OOM at any point.
- No --streaming-vram-mb knob, no hardcoded budget constant, no
per-machine calibration. The pool size adapts to whatever the
runtime grants.
Notes:
- Error scope probing: wgpu-native v29 classifies "Not enough memory
left" as WGPUErrorType_Validation, not OutOfMemory. We push both
filters (nested) and treat either firing as probe failure.
- The 4 GB probe cap is principled, not magic: above that, wgpu-native's
advertised maxBufferSize is sometimes a sentinel (1 TB) that just
forces wasteful halving steps. 4 GB is the largest buffer any
realistic WebGPU implementation will grant a single allocation today.
- Pool destroy()/release happens after model release in shutdown() so
the underlying buffer outlives every bind group that references it.
Follow-ups: spatial chunking (task #22) for finer eviction granularity;
cull perf needs work at 100+ models / 1M+ instances (separate from
streaming concerns).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Dion Moult