ifcviewer: v14 chunk-contiguous sidecar + progressive network streaming

Makes large-model streaming over a network actually good — fixing read
amplification, then first-paint latency — building on the byte-range work.

v14 layout + TOC (SidecarLayout, pure + unit-tested)
  The loader chunks meshes by spatial Morton order, but the sidecar stored
  geometry in mesh-id order, so a chunk's meshes were scattered through the
  file: streaming one chunk meant either hundreds of tiny range requests or
  reading (and discarding) everything between them — a 113 MB model fetched
  ~340 MB, a 531 MB model 2.25 GB (4.2x). Fix: at bake, reorder meshes into
  the loader's chunk order and rebuild vertex/index(LOD0+LOD1)/instance
  sections so each chunk is one CONTIGUOUS byte range, and bake a chunk TOC
  ({first_mesh, mesh_count}). The loader builds chunks straight from the TOC
  rather than re-deriving the plan — the float Morton quantisation isn't
  bit-identical across toolchains (x86 baker vs wasm loader), so a re-derived
  plan scatters the chunks. Format bumped to v14 (regenerate sidecars). The
  reorder buckets instances by per-instance mesh_id (the baker never sets
  MeshInfo.first_instance — trusting it scrambled every transform → geometry
  at the origin). Multiset-verified on a 28,900-instance model: every
  instance's placement + geometry preserved. Result: 531 MB fetches 531 MB
  (1.0x) in 72 requests (was 2036).

Progressive streaming (concurrency cap + small chunks)
  Even at 1x, geometry appeared only after ~the whole model arrived: the
  browser multiplexes every in-flight Range request over one HTTP/2 conn, so
  unbounded concurrency (9 in flight) split the bandwidth and nothing finished
  until the end (measured: first paint after 113 of 118 MB / 35 s @ 24 Mbps).
  Cap concurrent chunk loads (kMaxWebInflightChunks=2): the priority-sorted
  top chunks finish and paint first, then the next → first paint 9 s. Chunk
  size dropped 16->4 MB (cheap now that each chunk is one read; matches Cesium
  3D Tiles / xeokit / SVF2) for smoother progression. First-paint is now
  metadata-bound (~10 MB tail) — the next lever.

111/111 unit (new test_sidecar_layout: geometry preserved, contiguous layout,
Morton-identity) + 6/6 web smoke pass; desktop bake (SceneLoader) reorders
before writeSidecar; embedded web sample regenerated to v14.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Dion Moult
2026-06-30 21:22:47 +10:00
parent 46a695266a
commit e1be2f208c
14 changed files with 559 additions and 29 deletions
+25 -1
View File
@@ -65,9 +65,26 @@ static constexpr uint32_t SIDECAR_MAGIC = 0x49465657; // "IFVW"
// composition has reduced them to viewport-local float-sized values.
// v13 = Map unit scale in cached ModelGeoref is derived from
// IfcMapConversion.Scale, not IfcProjectedCRS.MapUnit.
static constexpr uint32_t SIDECAR_VERSION = 13;
// v14 = Geometry is laid out in streaming-chunk order (SidecarLayout) and a
// chunk table-of-contents (`chunks`) is appended. The loader builds its
// chunks from the TOC instead of re-deriving the Morton/greedy plan, so
// each chunk is one CONTIGUOUS byte range — fixing network read
// amplification. The plan can't be re-derived at load because the float
// Morton quantisation isn't bit-identical across toolchains (x86 baker vs
// wasm loader), so it must be baked in. No back-compat: v13 sidecars are
// rejected (regenerate them).
static constexpr uint32_t SIDECAR_VERSION = 14;
static constexpr uint32_t SIDECAR_ENDIAN = 0x01020304;
// Chunk table-of-contents entry (v14+). A chunk is a CONTIGUOUS range of
// meshes in the (reordered) meshes array — and therefore a contiguous span of
// vertex + index bytes, since the geometry is laid out in chunk order. The
// loader builds chunk `i` from meshes [first_mesh, first_mesh + mesh_count).
struct SidecarChunk {
uint32_t first_mesh;
uint32_t mesh_count;
};
// Fixed-size element record. Strings are stored as (offset, length) pairs
// into a separate string table.
struct PackedElementInfo {
@@ -111,6 +128,13 @@ struct SidecarData {
// Element tree metadata.
std::vector<PackedElementInfo> elements;
std::string string_table;
// Streaming chunk TOC. Always written on disk (v14); geometry is laid out
// in this chunk order (see SidecarLayout) so each chunk is one contiguous
// range and the loader builds chunks directly from it. Stays empty only for
// in-memory direct loads (finalizeModel), which don't stream and fall back
// to deriving the plan.
std::vector<SidecarChunk> chunks;
};
// Sidecar is keyed on the path stem: foo.ifc and foo.ifcdb/ both resolve to