mirror of
https://github.com/IfcOpenShell/IfcOpenShell.git
synced 2026-08-19 19:54:07 +00:00
ifcviewer: v14 chunk-contiguous sidecar + progressive network streaming
Makes large-model streaming over a network actually good — fixing read
amplification, then first-paint latency — building on the byte-range work.
v14 layout + TOC (SidecarLayout, pure + unit-tested)
The loader chunks meshes by spatial Morton order, but the sidecar stored
geometry in mesh-id order, so a chunk's meshes were scattered through the
file: streaming one chunk meant either hundreds of tiny range requests or
reading (and discarding) everything between them — a 113 MB model fetched
~340 MB, a 531 MB model 2.25 GB (4.2x). Fix: at bake, reorder meshes into
the loader's chunk order and rebuild vertex/index(LOD0+LOD1)/instance
sections so each chunk is one CONTIGUOUS byte range, and bake a chunk TOC
({first_mesh, mesh_count}). The loader builds chunks straight from the TOC
rather than re-deriving the plan — the float Morton quantisation isn't
bit-identical across toolchains (x86 baker vs wasm loader), so a re-derived
plan scatters the chunks. Format bumped to v14 (regenerate sidecars). The
reorder buckets instances by per-instance mesh_id (the baker never sets
MeshInfo.first_instance — trusting it scrambled every transform → geometry
at the origin). Multiset-verified on a 28,900-instance model: every
instance's placement + geometry preserved. Result: 531 MB fetches 531 MB
(1.0x) in 72 requests (was 2036).
Progressive streaming (concurrency cap + small chunks)
Even at 1x, geometry appeared only after ~the whole model arrived: the
browser multiplexes every in-flight Range request over one HTTP/2 conn, so
unbounded concurrency (9 in flight) split the bandwidth and nothing finished
until the end (measured: first paint after 113 of 118 MB / 35 s @ 24 Mbps).
Cap concurrent chunk loads (kMaxWebInflightChunks=2): the priority-sorted
top chunks finish and paint first, then the next → first paint 9 s. Chunk
size dropped 16->4 MB (cheap now that each chunk is one read; matches Cesium
3D Tiles / xeokit / SVF2) for smoother progression. First-paint is now
metadata-bound (~10 MB tail) — the next lever.
111/111 unit (new test_sidecar_layout: geometry preserved, contiguous layout,
Morton-identity) + 6/6 web smoke pass; desktop bake (SceneLoader) reorders
before writeSidecar; embedded web sample regenerated to v14.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,58 @@
|
||||
/********************************************************************************
|
||||
* *
|
||||
* This file is part of IfcOpenShell. *
|
||||
* *
|
||||
* IfcOpenShell is free software: you can redistribute it and/or modify *
|
||||
* it under the terms of the Lesser GNU General Public License as published by *
|
||||
* the Free Software Foundation, either version 3.0 of the License, or *
|
||||
* (at your option) any later version. *
|
||||
* *
|
||||
* IfcOpenShell is distributed in the hope that it will be useful, *
|
||||
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
|
||||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
|
||||
* Lesser GNU General Public License for more details. *
|
||||
* *
|
||||
* You should have received a copy of the Lesser GNU General Public License *
|
||||
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||||
* *
|
||||
********************************************************************************/
|
||||
|
||||
#ifndef SIDECARLAYOUT_H
|
||||
#define SIDECARLAYOUT_H
|
||||
|
||||
#include "SidecarCache.h"
|
||||
|
||||
// Reorder a sidecar's geometry for streaming locality.
|
||||
//
|
||||
// The streaming loader chunks meshes by 3D Morton (Z-order) over their
|
||||
// centroids, then greedy-packs them into ~16 MB chunks; a chunk is always a
|
||||
// CONSECUTIVE run of that sorted order. But a freshly-baked sidecar stores
|
||||
// vertices / indices / meshes in mesh-id (iterator) order, which has no
|
||||
// relation to the spatial chunking — so a chunk's meshes are scattered through
|
||||
// the file, and streaming one chunk over a network either issues hundreds of
|
||||
// tiny range requests or reads (and discards) everything in between (~3×
|
||||
// bandwidth amplification was measured on a 113 MB model).
|
||||
//
|
||||
// This pass permutes meshes into the loader's Morton order and rebuilds the
|
||||
// vertex / index / instance sections to match, so that each spatial chunk
|
||||
// becomes a CONTIGUOUS byte range. The loader then re-runs the same Morton
|
||||
// sort, gets the identity permutation, and reads each chunk as one contiguous
|
||||
// range — no amplification, ~1 request per chunk, and chunks appear
|
||||
// progressively as they arrive.
|
||||
//
|
||||
// Pure transform (no Qt / no wgpu): meshes, vertices, indices (LOD0 + LOD1),
|
||||
// and instances are all rebuilt in the new order with vbo/ebo/lod1 offsets,
|
||||
// MeshInfo.first_instance, and InstanceCpu.mesh_id remapped consistently.
|
||||
// Index values are mesh-local, so they move unchanged. Element/georef/string
|
||||
// data is mesh-independent and untouched. No-op for < 2 meshes.
|
||||
//
|
||||
// Also populates `sd.chunks` (the v14 TOC): the loader must build chunks from
|
||||
// this rather than re-deriving the plan, because the float Morton quantisation
|
||||
// isn't bit-identical across toolchains (x86 baker vs wasm loader) — a re-derived
|
||||
// plan disagrees on boundary meshes and the contiguity is lost.
|
||||
//
|
||||
// Run at bake (writeSidecar path) or as a one-shot migration over existing
|
||||
// .ifcview files (read → reorder → write).
|
||||
void reorderSidecarByMorton(SidecarData& sd);
|
||||
|
||||
#endif // SIDECARLAYOUT_H
|
||||
Reference in New Issue
Block a user