The .ifcview data is hugely redundant (repeated double instance matrices,
patterned indices) — measured 12x zstd whole-file. Server Content-Encoding
can't be used (it breaks HTTP Range), so compress PER-CHUNK into the format.
Format (v16): geometry becomes per-chunk zstd(vertices)+zstd(indices) frames —
each independently Range-fetchable, so streaming is intact — and the critical +
deferred metadata blocks are single zstd frames. SidecarChunk carries the
compressed blob offsets/sizes; applyStreamedChunk (render/upload) is UNCHANGED —
decompression slots into the fetch. Full readSidecar (test/tooling) reconstructs
by decompress+scatter. zstd: desktop links libzstd (also compresses at bake);
the web build (Emscripten has no zstd port) FetchContent's the pinned zstd
source and compiles its decompress-only subset for wasm — no vendored blob,
same version as desktop. New SidecarCompress wraps it (compress guarded off
under Emscripten). Both stream paths — desktop StreamingThread worker + sync
fallback (readChunkGeometryCompressed) and web beginWebChunkLoad — decompress;
readSidecarMetadataOnly / the web bootstrap / loadDeferredMetadataWeb decompress
the metadata blocks. streamingByteProgress reports COMPRESSED bytes. MEASURED: a
752 MB v15 federation → 75 MB v16 (10x; per-file 6.7-15.3x); PP-PLP 118→15 MB,
loads 13/13 chunks on web, 0 errors.
Three fixes found while testing big federations on a real server:
- Web-streamed race: streaming_from_web was set in the deferred-header callback
(a round-trip after the model+chunks exist), so driveStreamingLoads could take
the sync fopen path meanwhile → "failed to read/decompress chunk 0". Now set
immediately after applyCachedModel.
- OOM abort on 18 models: the pool grew unbounded until an alloc failed, but on
web that's an uncatchable bad_alloc abort. Cap total pool capacity
(setMaxTotalCapacity, 3 GB) so it stops before the heap ceiling, and raise
MAXIMUM_MEMORY 2→4 GB (wasm32 max) for headroom.
- Web never evicted (grow-or-block only). At the hard budget, fall through to the
LRU/priority evictor so a big federation stays navigable (highest-contribution
chunks win) instead of freezing with holes.
113/113 desktop + 6/6 web smoke pass. No back-compat: regenerate sidecars
(desktop bakes v16; scratch conv tool migrates v15→v16).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First-paint over a network is metadata-bound: the whole post-index metadata
(~10 MB on a 118 MB model) had to download before any geometry. But ~25% of
it — elements + string_table, the IFC element tree (names/GUIDs/hierarchy) —
is used only for UI/picking, never for rendering (ViewportCore never touches
it).
v15 splits the post-index metadata into a render-CRITICAL block (meshes,
instances, georef, chunk TOC) preceded by its byte length, then a DEFERRED
block (elements + string_table). The web loader reads only the critical block
before painting; the deferred block sits at a known, self-describing offset
([critical end, EOF)) and is fetched on demand. Desktop reads both (local).
Web on-demand path is wired and complete (not yet called — no UI consumer):
loadDeferredMetadataWeb(model_id) range-fetches + parses the deferred block
into ModelGpuData.elements/string_table, at most once; the first consumer
will be "show the selected object's name" on pick. No background prefetch —
view-only sessions never download the property data (saves 2.64 MB on this
model).
parseSidecarTail split into parseSidecarCritical + parseSidecarDeferred (pure,
unit-tested); StreamingSidecar gains the critical-block locator. Measured
(118 MB model): critical metadata 10.35 -> 7.71 MB, deferred 2.64 MB off the
path; first paint 10.5 -> 9.6 s @ 24 Mbps. (Instances still dominate the
critical block — the next metadata lever.) Format -> v15, no back-compat;
regenerate sidecars. 111/111 unit + 6/6 web smoke pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Makes large-model streaming over a network actually good — fixing read
amplification, then first-paint latency — building on the byte-range work.
v14 layout + TOC (SidecarLayout, pure + unit-tested)
The loader chunks meshes by spatial Morton order, but the sidecar stored
geometry in mesh-id order, so a chunk's meshes were scattered through the
file: streaming one chunk meant either hundreds of tiny range requests or
reading (and discarding) everything between them — a 113 MB model fetched
~340 MB, a 531 MB model 2.25 GB (4.2x). Fix: at bake, reorder meshes into
the loader's chunk order and rebuild vertex/index(LOD0+LOD1)/instance
sections so each chunk is one CONTIGUOUS byte range, and bake a chunk TOC
({first_mesh, mesh_count}). The loader builds chunks straight from the TOC
rather than re-deriving the plan — the float Morton quantisation isn't
bit-identical across toolchains (x86 baker vs wasm loader), so a re-derived
plan scatters the chunks. Format bumped to v14 (regenerate sidecars). The
reorder buckets instances by per-instance mesh_id (the baker never sets
MeshInfo.first_instance — trusting it scrambled every transform → geometry
at the origin). Multiset-verified on a 28,900-instance model: every
instance's placement + geometry preserved. Result: 531 MB fetches 531 MB
(1.0x) in 72 requests (was 2036).
Progressive streaming (concurrency cap + small chunks)
Even at 1x, geometry appeared only after ~the whole model arrived: the
browser multiplexes every in-flight Range request over one HTTP/2 conn, so
unbounded concurrency (9 in flight) split the bandwidth and nothing finished
until the end (measured: first paint after 113 of 118 MB / 35 s @ 24 Mbps).
Cap concurrent chunk loads (kMaxWebInflightChunks=2): the priority-sorted
top chunks finish and paint first, then the next → first paint 9 s. Chunk
size dropped 16->4 MB (cheap now that each chunk is one read; matches Cesium
3D Tiles / xeokit / SVF2) for smoother progression. First-paint is now
metadata-bound (~10 MB tail) — the next lever.
111/111 unit (new test_sidecar_layout: geometry preserved, contiguous layout,
Morton-identity) + 6/6 web smoke pass; desktop bake (SceneLoader) reorders
before writeSidecar; embedded web sample regenerated to v14.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Use IfcMapConversion.Scale as the source of truth for converting map coordinates to metres, instead of deriving that scale from IfcProjectedCRS.MapUnit. Bump the sidecar version because cached georef matrices and unit scales may differ under the new interpretation.
Generated with the assistance of an AI coding tool.
Keep placement transformations in double precision through streaming, sidecar caching, and viewport recomposition so large coordinates can be cancelled before the final GPU float upload.
Generated with the assistance of an AI coding tool.
Previously, applyCoordinateOperationToViewport — which pushes both
CoordinateOperation and ModelTransformation — was only called on
paths that required the IFC source to be loaded
(onLoadedFromStream and onDataSourceReady). Sidecar-only loads
(loadDataSource off, or no .ifc/.rdb sibling) silently lost both
stages.
Cache the per-model georef + unit scales in the sidecar itself so
the IFC source isn't needed to apply them:
SidecarData gains
coordinate_operation_meters[16] // column-major
project_length_to_meters
map_unit_to_meters
has_coordinate_operation
148 B fixed block written/read between instances and elements.
SIDECAR_VERSION 10 -> 11; existing sidecars rebuild on next load.
MainWindow::writeSidecarForModel populates the block from
loader_->modelGeoref(mid) before writeSidecar.
SceneLoader::applySidecarData restores it into the model's
ModelGeoref + sets has_georef = true, so subsequent
loader_->modelGeoref(mid) calls return the cached data without
needing the IFC.
MainWindow::onLoadedFromSidecar now calls
applyCoordinateOperationToViewport(mid) directly — both
CoordinateOperation and ModelTransformation land at sidecar-load
time, no longer waiting on a possibly-never-arriving data source.
Edits to the IFC's IfcMapConversion don't invalidate the cache —
delete the .ifcview manually if the source's georef changes. This
matches the existing cache-invalidation contract.
Tests: round-trip the new fields through the existing sidecar
fixture; assert SIDECAR_VERSION == 11.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Covers the pure-logic modules with no Qt event loop or GL context: BVH
build, LOD decimation, sidecar round-trip, instanced-geometry layout
constants, and Federation save/load + relative-path policy. Each test
binary compiles only the production source(s) under test, so the unit
tier doesn't pull Qt/OpenCASCADE/IfcGeom into the test build.
Gated behind BUILD_IFCVIEWER_TESTS=OFF; default builds remain offline.
Catch2 v3.5.4 is fetched on demand via FetchContent.
Dion Moult