ifcviewer: v14 chunk-contiguous sidecar + progressive network streaming

Makes large-model streaming over a network actually good — fixing read
amplification, then first-paint latency — building on the byte-range work.

v14 layout + TOC (SidecarLayout, pure + unit-tested)
  The loader chunks meshes by spatial Morton order, but the sidecar stored
  geometry in mesh-id order, so a chunk's meshes were scattered through the
  file: streaming one chunk meant either hundreds of tiny range requests or
  reading (and discarding) everything between them — a 113 MB model fetched
  ~340 MB, a 531 MB model 2.25 GB (4.2x). Fix: at bake, reorder meshes into
  the loader's chunk order and rebuild vertex/index(LOD0+LOD1)/instance
  sections so each chunk is one CONTIGUOUS byte range, and bake a chunk TOC
  ({first_mesh, mesh_count}). The loader builds chunks straight from the TOC
  rather than re-deriving the plan — the float Morton quantisation isn't
  bit-identical across toolchains (x86 baker vs wasm loader), so a re-derived
  plan scatters the chunks. Format bumped to v14 (regenerate sidecars). The
  reorder buckets instances by per-instance mesh_id (the baker never sets
  MeshInfo.first_instance — trusting it scrambled every transform → geometry
  at the origin). Multiset-verified on a 28,900-instance model: every
  instance's placement + geometry preserved. Result: 531 MB fetches 531 MB
  (1.0x) in 72 requests (was 2036).

Progressive streaming (concurrency cap + small chunks)
  Even at 1x, geometry appeared only after ~the whole model arrived: the
  browser multiplexes every in-flight Range request over one HTTP/2 conn, so
  unbounded concurrency (9 in flight) split the bandwidth and nothing finished
  until the end (measured: first paint after 113 of 118 MB / 35 s @ 24 Mbps).
  Cap concurrent chunk loads (kMaxWebInflightChunks=2): the priority-sorted
  top chunks finish and paint first, then the next → first paint 9 s. Chunk
  size dropped 16->4 MB (cheap now that each chunk is one read; matches Cesium
  3D Tiles / xeokit / SVF2) for smoother progression. First-paint is now
  metadata-bound (~10 MB tail) — the next lever.

111/111 unit (new test_sidecar_layout: geometry preserved, contiguous layout,
Morton-identity) + 6/6 web smoke pass; desktop bake (SceneLoader) reorders
before writeSidecar; embedded web sample regenerated to v14.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Dion Moult
2026-06-30 21:22:47 +10:00
parent 46a695266a
commit e1be2f208c
14 changed files with 559 additions and 29 deletions
+17 -1
View File
@@ -155,10 +155,26 @@ bool sidecarDataEqual(const SidecarData& a, const SidecarData& b) {
TEST_CASE("MeshInfo and InstanceCpu have stable layouts (sidecar wire format)", "[sidecar]") {
REQUIRE(sizeof(MeshInfo) == 56);
REQUIRE(sizeof(InstanceGpu) == 80);
REQUIRE(SIDECAR_VERSION == 13);
REQUIRE(SIDECAR_VERSION == 14);
REQUIRE(sizeof(SidecarChunk) == 8);
REQUIRE(SIDECAR_MAGIC == 0x49465657u);
}
TEST_CASE("writeSidecar/readSidecar round-trip the v14 chunk TOC", "[sidecar]") {
fs::path dir = makeScratchDir("chunks");
fs::path ifc = dir / "model.ifc";
SidecarData sd = buildFixture();
sd.chunks = { {0, 1}, {1, 1} }; // two chunks over the two meshes
REQUIRE(writeSidecar(ifc.string(), sd));
auto loaded = readSidecar(ifc.string());
REQUIRE(loaded.has_value());
REQUIRE(loaded->chunks.size() == 2);
REQUIRE(loaded->chunks[0].first_mesh == 0);
REQUIRE(loaded->chunks[0].mesh_count == 1);
REQUIRE(loaded->chunks[1].first_mesh == 1);
REQUIRE(loaded->chunks[1].mesh_count == 1);
}
TEST_CASE("writeSidecar then readSidecar round-trips the full fixture", "[sidecar]") {
fs::path dir = makeScratchDir("roundtrip");
fs::path ifc = dir / "model.ifc";