ifcviewer: v14 chunk-contiguous sidecar + progressive network streaming

Makes large-model streaming over a network actually good — fixing read
amplification, then first-paint latency — building on the byte-range work.

v14 layout + TOC (SidecarLayout, pure + unit-tested)
  The loader chunks meshes by spatial Morton order, but the sidecar stored
  geometry in mesh-id order, so a chunk's meshes were scattered through the
  file: streaming one chunk meant either hundreds of tiny range requests or
  reading (and discarding) everything between them — a 113 MB model fetched
  ~340 MB, a 531 MB model 2.25 GB (4.2x). Fix: at bake, reorder meshes into
  the loader's chunk order and rebuild vertex/index(LOD0+LOD1)/instance
  sections so each chunk is one CONTIGUOUS byte range, and bake a chunk TOC
  ({first_mesh, mesh_count}). The loader builds chunks straight from the TOC
  rather than re-deriving the plan — the float Morton quantisation isn't
  bit-identical across toolchains (x86 baker vs wasm loader), so a re-derived
  plan scatters the chunks. Format bumped to v14 (regenerate sidecars). The
  reorder buckets instances by per-instance mesh_id (the baker never sets
  MeshInfo.first_instance — trusting it scrambled every transform → geometry
  at the origin). Multiset-verified on a 28,900-instance model: every
  instance's placement + geometry preserved. Result: 531 MB fetches 531 MB
  (1.0x) in 72 requests (was 2036).

Progressive streaming (concurrency cap + small chunks)
  Even at 1x, geometry appeared only after ~the whole model arrived: the
  browser multiplexes every in-flight Range request over one HTTP/2 conn, so
  unbounded concurrency (9 in flight) split the bandwidth and nothing finished
  until the end (measured: first paint after 113 of 118 MB / 35 s @ 24 Mbps).
  Cap concurrent chunk loads (kMaxWebInflightChunks=2): the priority-sorted
  top chunks finish and paint first, then the next → first paint 9 s. Chunk
  size dropped 16->4 MB (cheap now that each chunk is one read; matches Cesium
  3D Tiles / xeokit / SVF2) for smoother progression. First-paint is now
  metadata-bound (~10 MB tail) — the next lever.

111/111 unit (new test_sidecar_layout: geometry preserved, contiguous layout,
Morton-identity) + 6/6 web smoke pass; desktop bake (SceneLoader) reorders
before writeSidecar; embedded web sample regenerated to v14.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Dion Moult
2026-06-30 21:22:47 +10:00
parent 46a695266a
commit e1be2f208c
14 changed files with 559 additions and 29 deletions
+58
View File
@@ -0,0 +1,58 @@
/********************************************************************************
* *
* This file is part of IfcOpenShell. *
* *
* IfcOpenShell is free software: you can redistribute it and/or modify *
* it under the terms of the Lesser GNU General Public License as published by *
* the Free Software Foundation, either version 3.0 of the License, or *
* (at your option) any later version. *
* *
* IfcOpenShell is distributed in the hope that it will be useful, *
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
* Lesser GNU General Public License for more details. *
* *
* You should have received a copy of the Lesser GNU General Public License *
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
* *
********************************************************************************/
#ifndef SIDECARLAYOUT_H
#define SIDECARLAYOUT_H
#include "SidecarCache.h"
// Reorder a sidecar's geometry for streaming locality.
//
// The streaming loader chunks meshes by 3D Morton (Z-order) over their
// centroids, then greedy-packs them into ~16 MB chunks; a chunk is always a
// CONSECUTIVE run of that sorted order. But a freshly-baked sidecar stores
// vertices / indices / meshes in mesh-id (iterator) order, which has no
// relation to the spatial chunking — so a chunk's meshes are scattered through
// the file, and streaming one chunk over a network either issues hundreds of
// tiny range requests or reads (and discards) everything in between (~3×
// bandwidth amplification was measured on a 113 MB model).
//
// This pass permutes meshes into the loader's Morton order and rebuilds the
// vertex / index / instance sections to match, so that each spatial chunk
// becomes a CONTIGUOUS byte range. The loader then re-runs the same Morton
// sort, gets the identity permutation, and reads each chunk as one contiguous
// range — no amplification, ~1 request per chunk, and chunks appear
// progressively as they arrive.
//
// Pure transform (no Qt / no wgpu): meshes, vertices, indices (LOD0 + LOD1),
// and instances are all rebuilt in the new order with vbo/ebo/lod1 offsets,
// MeshInfo.first_instance, and InstanceCpu.mesh_id remapped consistently.
// Index values are mesh-local, so they move unchanged. Element/georef/string
// data is mesh-independent and untouched. No-op for < 2 meshes.
//
// Also populates `sd.chunks` (the v14 TOC): the loader must build chunks from
// this rather than re-deriving the plan, because the float Morton quantisation
// isn't bit-identical across toolchains (x86 baker vs wasm loader) — a re-derived
// plan disagrees on boundary meshes and the contiguity is lost.
//
// Run at bake (writeSidecar path) or as a one-shot migration over existing
// .ifcview files (read → reorder → write).
void reorderSidecarByMorton(SidecarData& sd);
#endif // SIDECARLAYOUT_H