wgpu streaming (3/4): --streaming scaffold + applyCachedModelStreaming

Wires the metadata-only reader (commit 1) through a parallel streaming
load path. With --streaming on:

  - loadSidecar routes through readSidecarMetadataOnly: reads header +
    mesh dict + instance dict + georef + elements upfront. Skips
    vertex bytes entirely.
  - applyCachedModelStreaming computes the same chunk plan as the
    non-streaming path, allocates the small per-chunk buffers
    (visible_draws + prefix_sums + per_chunk_uniform), allocates the
    model-shared mesh + instance + index buffers, but leaves each
    chunk's vertex_storage NULL and is_resident=false.
  - Stores streaming_file_path + vertex_section_offset on the model so
    the per-frame loader can range-read chunks later.
  - Computes per-chunk world AABB by walking instances → mesh → chunk;
    used by both cull (chunk-level frustum reject, future) and the
    streaming loader (proximity-prioritised fetch, future).

Index buffer is still loaded upfront in stage 1 (small relative to
vertex data: ~1/2 of vertex bytes on real scenes). Stage 2 may defer
it too if measurements suggest it's worth the extra plumbing.

Render + pick already gate on c.bind_group (null when non-resident),
so the existing guards correctly skip non-resident chunks without
further changes.

With this commit alone, --streaming mode shows an EMPTY scene (just
background colour) because no chunk ever becomes resident. Commit 4
adds the per-frame loader that triggers chunk load when cull marks
them visible — that's the commit where rendering kicks in and the
OOM fix actually lands.

Default behaviour (no --streaming): legacy synchronous full-load.
Pixel-identical to the prior commit on basic.ifc.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
Dion Moult
2026-05-28 09:15:53 +10:00
parent d368ee449d
commit f6d888d42b
3 changed files with 382 additions and 25 deletions
+8 -3
View File
@@ -58,13 +58,18 @@ int main(int argc, char* argv[]) {
"Enable BVH-walk cull. Off by default — currently a regression on "
"dense camera-looking-at-everything scenes; may help on sprawling "
"federations where most of the scene is off-screen."});
parser.addOption({"streaming",
"Enable streaming sidecar load. Reads metadata-only at load time; "
"vertex chunks are deferred and loaded on demand as they become "
"frustum-visible. Required for scenes that exceed GPU memory."});
parser.process(app);
auto* viewport = new WgpuViewportWindow;
viewport->resize(1280, 800);
if (parser.isSet("no-hiz")) viewport->hiz_enabled_ = false;
if (parser.isSet("web-limits")) viewport->web_limits_ = true;
if (parser.isSet("bvh")) viewport->bvh_enabled_ = true;
if (parser.isSet("no-hiz")) viewport->hiz_enabled_ = false;
if (parser.isSet("web-limits")) viewport->web_limits_ = true;
if (parser.isSet("bvh")) viewport->bvh_enabled_ = true;
if (parser.isSet("streaming")) viewport->streaming_enabled_ = true;
QWidget* container = QWidget::createWindowContainer(viewport);
container->setMinimumSize(320, 240);