wgpu streaming: multi-pool growth, frustum-only residency, sorted convergence

Five interlocking fixes that take --streaming on the big federation
scene from "5 fps + endless flicker + infinite cold-load" to a
stable 35-49 fps with a converged working set.

1. Multi-sub-buffer WgpuBufferPool. Pool now grows lazily by adding
   sub-buffers of per_sub_buffer_capacity_ when alloc demand exceeds
   existing free runs. Each Slice carries (buffer, offset, size,
   sub_idx). On driver refusal of addSubBuffer, growth_disabled_
   latches so subsequent allocs don't keep retrying and log-spamming.
   pool_can_fit consults can_grow() to know when growth could rescue
   a candidate vs when eviction is the only path.

2. Split cull / stream benchmark timers. The previous "cull[wall]"
   metric was actually cull + driveStreamingLoads, blaming the wrong
   subsystem (~170 ms of "cull" was synchronous disk I/O).

3. frustum_visible_count on Chunk, populated in cullModelCpuCompute
   right after the per-instance aabbInFrustum check. driveStreamingLoads
   now keys residency on this instead of total_visible_draws (which
   includes contribution + HiZ). HiZ visibility flips frame-to-frame
   as occluders shift; using it for residency caused chunks to be
   evicted then immediately re-loaded, every frame, even with a
   stationary camera — both the perf cliff and the visible flicker.

4. Distance-sorted candidates in driveStreamingLoads. Walk the
   non-resident frustum-visible chunks in distance order (closest
   first). With sorted processing, evict_farthest_than converges
   monotonically: each swap replaces a far resident with a closer
   candidate; once the next candidate is farther than every
   remaining resident, the loop exits. Without sorting the loader
   visited candidates in model/chunk-id order, swapping random
   chunks every frame without ever converging.

5. 10% eviction hysteresis (EVICT_DIST2_RATIO = 1.21). On scenes
   where many chunks are clustered at similar distance from the
   camera (e.g. several chunks all ~370 m away), naive
   "evict any resident strictly farther than candidate" triggers
   sub-meter swaps every frame, never resting. Requiring the victim
   to be 10% farther in linear distance kills these cycles while
   still allowing genuine "much closer" candidates to evict.

Plus: latched bench_warm_done_ on the cold-load gate, with a
5-frames-of-zero-loads convergence test (default-camera big scene
converges in 20 frames) and a 600-frame timeout fallback that prints
exactly once.

Measured on the test federation (111 sidecars, ~3 GB raw, 1 M
instances) with the user's close-in camera:
- avg 35 fps (was 5), median 49 fps (was 7)
- cull 19 ms (now the bottleneck), stream 5-8 ms (was 172)
- p99 184 ms — occasional big-chunk load on the render thread;
  background-thread I/O would smooth that out as a follow-up.

With the default wide camera:
- avg 40 fps, converges in 20 frames, residency grows naturally
  from 59 → 76 chunks as orbit shifts the frustum.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Dion Moult
2026-05-28 14:08:16 +10:00
parent 502c29fbc2
commit c3a55d7f7b
5 changed files with 507 additions and 238 deletions
+17 -12
View File
@@ -30,6 +30,7 @@
#include "BvhAccel.h"
#include "InstancedGeometry.h"
#include "WgpuBufferPool.h"
// Per-model wgpu state. Mirrors the GL backend's ModelGpuData but with
// wgpu handles. Stage 2 only allocates and uploads the four core buffers;
@@ -77,16 +78,13 @@ struct WgpuModelGpuData {
// needs them. Non-streaming path always sets is_resident=true and
// populates pool ranges at applyCachedModel time.
struct Chunk {
// Pool-allocated vertex bytes. When resident, pool_vertex_size > 0
// and the range [pool_vertex_offset, pool_vertex_offset + pool_vertex_size)
// in WgpuViewportWindow::pool_ holds this chunk's vertex_storage.
// When non-resident, both are 0.
uint64_t pool_vertex_offset = 0;
uint64_t pool_vertex_size = 0;
// Pool-allocated index bytes. Same lifetime as the vertex range —
// either both resident or both freed.
uint64_t pool_index_offset = 0;
uint64_t pool_index_size = 0;
// Pool-allocated vertex + index bytes. Both slices land in the
// shared WgpuViewportWindow::pool_; the slice tells us which
// sub-buffer they live in (the pool may span multiple sub-buffers
// when scenes exceed wgpu's single-buffer cap). When non-resident,
// both .size are 0.
WgpuBufferPool::Slice vertex_slice;
WgpuBufferPool::Slice index_slice;
WGPUBuffer visible_draws_buffer = nullptr;
WGPUBuffer prefix_sums_buffer = nullptr;
@@ -98,8 +96,17 @@ struct WgpuModelGpuData {
size_t prefix_sums_capacity = 0;
// Per-frame, populated by cullModelCpuCompute and consumed by render().
// total_visible_* are post-frustum + contribution + HiZ — used to size
// the actual draw call. frustum_visible_count is bumped immediately
// after the frustum check (before contribution / HiZ), and is what
// driveStreamingLoads keys on for residency decisions. Streaming
// must NOT use the HiZ-post counters: HiZ visibility flips
// frame-to-frame as occluders shift, which would otherwise thrash
// the loader (evict-then-reload every frame even with the camera
// stationary, killing FPS and producing visible flicker).
uint32_t total_visible_vertices = 0;
uint32_t total_visible_draws = 0;
uint32_t frustum_visible_count = 0;
std::vector<VisibleDrawGpu> visible_draws_scratch;
std::vector<uint32_t> prefix_sums_scratch;
@@ -190,8 +197,6 @@ struct WgpuModelGpuData {
bool hidden = false;
};
class WgpuBufferPool;
// Release every wgpu handle in `m` (including per-chunk and per-model pool
// ranges via `pool.free()`) and clear its size mirrors. Safe to call
// repeatedly; idempotent on already-released entries.