Rename the streamer/sidecar transfer and record types to describe what
they are rather than how they move:
MeshChunk -> StreamedMesh
InstanceChunk -> StreamedInstance
InstanceCpu -> InstanceInfo
PackedElementInfo -> ElementTableRecord
uploadMeshChunk -> uploadStreamedMesh
uploadInstanceChunk -> uploadStreamedInstance
buildMeshChunk -> buildStreamedMesh
and the two post-index sidecar metadata blocks:
"critical" metadata -> "geometry" metadata (meshes/instances/georef/TOC)
"deferred" metadata -> "element" metadata (elements + string table)
parseSidecarCritical -> parseSidecarGeometryMetadata
parseSidecarDeferred -> parseSidecarElementMetadata
The one behavioural change: the element hierarchy (parent_id) was
carried through ElementInfo, ElementTableRecord, and the sidecar element
table but never consumed, so drop it and bump SIDECAR_VERSION 16 -> 17.
No back-compat: regenerate sidecars. sample.ifcview is regenerated at v17.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rename short local variables and parameters in the viewer loading, sidecar, and BonsaiViewer command paths to make their responsibilities clearer.\n\nGenerated with the assistance of an AI coding tool.
The .ifcview data is hugely redundant (repeated double instance matrices,
patterned indices) — measured 12x zstd whole-file. Server Content-Encoding
can't be used (it breaks HTTP Range), so compress PER-CHUNK into the format.
Format (v16): geometry becomes per-chunk zstd(vertices)+zstd(indices) frames —
each independently Range-fetchable, so streaming is intact — and the critical +
deferred metadata blocks are single zstd frames. SidecarChunk carries the
compressed blob offsets/sizes; applyStreamedChunk (render/upload) is UNCHANGED —
decompression slots into the fetch. Full readSidecar (test/tooling) reconstructs
by decompress+scatter. zstd: desktop links libzstd (also compresses at bake);
the web build (Emscripten has no zstd port) FetchContent's the pinned zstd
source and compiles its decompress-only subset for wasm — no vendored blob,
same version as desktop. New SidecarCompress wraps it (compress guarded off
under Emscripten). Both stream paths — desktop StreamingThread worker + sync
fallback (readChunkGeometryCompressed) and web beginWebChunkLoad — decompress;
readSidecarMetadataOnly / the web bootstrap / loadDeferredMetadataWeb decompress
the metadata blocks. streamingByteProgress reports COMPRESSED bytes. MEASURED: a
752 MB v15 federation → 75 MB v16 (10x; per-file 6.7-15.3x); PP-PLP 118→15 MB,
loads 13/13 chunks on web, 0 errors.
Three fixes found while testing big federations on a real server:
- Web-streamed race: streaming_from_web was set in the deferred-header callback
(a round-trip after the model+chunks exist), so driveStreamingLoads could take
the sync fopen path meanwhile → "failed to read/decompress chunk 0". Now set
immediately after applyCachedModel.
- OOM abort on 18 models: the pool grew unbounded until an alloc failed, but on
web that's an uncatchable bad_alloc abort. Cap total pool capacity
(setMaxTotalCapacity, 3 GB) so it stops before the heap ceiling, and raise
MAXIMUM_MEMORY 2→4 GB (wasm32 max) for headroom.
- Web never evicted (grow-or-block only). At the hard budget, fall through to the
LRU/priority evictor so a big federation stays navigable (highest-contribution
chunks win) instead of freezing with holes.
113/113 desktop + 6/6 web smoke pass. No back-compat: regenerate sidecars
(desktop bakes v16; scratch conv tool migrates v15→v16).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First-paint over a network is metadata-bound: the whole post-index metadata
(~10 MB on a 118 MB model) had to download before any geometry. But ~25% of
it — elements + string_table, the IFC element tree (names/GUIDs/hierarchy) —
is used only for UI/picking, never for rendering (ViewportCore never touches
it).
v15 splits the post-index metadata into a render-CRITICAL block (meshes,
instances, georef, chunk TOC) preceded by its byte length, then a DEFERRED
block (elements + string_table). The web loader reads only the critical block
before painting; the deferred block sits at a known, self-describing offset
([critical end, EOF)) and is fetched on demand. Desktop reads both (local).
Web on-demand path is wired and complete (not yet called — no UI consumer):
loadDeferredMetadataWeb(model_id) range-fetches + parses the deferred block
into ModelGpuData.elements/string_table, at most once; the first consumer
will be "show the selected object's name" on pick. No background prefetch —
view-only sessions never download the property data (saves 2.64 MB on this
model).
parseSidecarTail split into parseSidecarCritical + parseSidecarDeferred (pure,
unit-tested); StreamingSidecar gains the critical-block locator. Measured
(118 MB model): critical metadata 10.35 -> 7.71 MB, deferred 2.64 MB off the
path; first paint 10.5 -> 9.6 s @ 24 Mbps. (Instances still dominate the
critical block — the next metadata lever.) Format -> v15, no back-compat;
regenerate sidecars. 111/111 unit + 6/6 web smoke pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Makes large-model streaming over a network actually good — fixing read
amplification, then first-paint latency — building on the byte-range work.
v14 layout + TOC (SidecarLayout, pure + unit-tested)
The loader chunks meshes by spatial Morton order, but the sidecar stored
geometry in mesh-id order, so a chunk's meshes were scattered through the
file: streaming one chunk meant either hundreds of tiny range requests or
reading (and discarding) everything between them — a 113 MB model fetched
~340 MB, a 531 MB model 2.25 GB (4.2x). Fix: at bake, reorder meshes into
the loader's chunk order and rebuild vertex/index(LOD0+LOD1)/instance
sections so each chunk is one CONTIGUOUS byte range, and bake a chunk TOC
({first_mesh, mesh_count}). The loader builds chunks straight from the TOC
rather than re-deriving the plan — the float Morton quantisation isn't
bit-identical across toolchains (x86 baker vs wasm loader), so a re-derived
plan scatters the chunks. Format bumped to v14 (regenerate sidecars). The
reorder buckets instances by per-instance mesh_id (the baker never sets
MeshInfo.first_instance — trusting it scrambled every transform → geometry
at the origin). Multiset-verified on a 28,900-instance model: every
instance's placement + geometry preserved. Result: 531 MB fetches 531 MB
(1.0x) in 72 requests (was 2036).
Progressive streaming (concurrency cap + small chunks)
Even at 1x, geometry appeared only after ~the whole model arrived: the
browser multiplexes every in-flight Range request over one HTTP/2 conn, so
unbounded concurrency (9 in flight) split the bandwidth and nothing finished
until the end (measured: first paint after 113 of 118 MB / 35 s @ 24 Mbps).
Cap concurrent chunk loads (kMaxWebInflightChunks=2): the priority-sorted
top chunks finish and paint first, then the next → first paint 9 s. Chunk
size dropped 16->4 MB (cheap now that each chunk is one read; matches Cesium
3D Tiles / xeokit / SVF2) for smoother progression. First-paint is now
metadata-bound (~10 MB tail) — the next lever.
111/111 unit (new test_sidecar_layout: geometry preserved, contiguous layout,
Morton-identity) + 6/6 web smoke pass; desktop bake (SceneLoader) reorders
before writeSidecar; embedded web sample regenerated to v14.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Splits the v13 metadata wire-format knowledge out of the FILE*-bound
streaming reader into pure, buffer-based functions so the web byte-range
path (#88) can reuse it without loading the whole sidecar into the wasm
heap:
- parseSidecarHead — validates the 16-byte head, yields num_vertex_bytes
- parseSidecarTail — parses meshes/instances/georef/elements/strings
from an in-memory tail buffer, bounds-checked
- planSidecarReadRanges + SidecarReadPlan — the range-coalescing /
scatter planner, promoted out of the anonymous namespace
readSidecarMetadataOnly and the range readers now call these; desktop
behaviour is unchanged (head + tail are small, the bulk is still skipped
via seek). The metadata tail is split from the head around the bulk
sections, so a blob-backed loader just slices those two regions and
hands the bytes to the same parsers.
Closes a coverage gap: StreamingLoader had no unit tests. Adds
test_streaming_loader.cpp (7 cases: metadata round-trip, corrupt/truncated
rejection, vertex+index range scatter, head validation, tail truncation,
read-plan coalescing). 107/107 unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The GL backend is gone (task #53). The wgpu/non-wgpu folder split and
the Wgpu* class prefix were both disambiguation artefacts from the
overlap period — now pure dead weight.
## Folder + library merge
* `src/ifcviewer-wgpu/` → folded into `src/ifcviewer/` (git mv tracks
every file as a rename so blame/log history survives).
* `src/ifcviewer-wgpu-minimal/` → `src/ifcviewer-minimal/` (the exe was
already named `IfcViewerMinimal`; this just brings the folder + CMake
target name into line).
* `src/ifcviewer-wgpu/tests/test_wgpu_{selection,visibility}.cpp` →
`src/ifcviewer/tests/test_{selection,visibility}.cpp`, folded into
the existing `add_ifcviewer_unit_test(...)` helper.
* The `IfcViewerWgpu` static library is dissolved — its sources become
part of the unified `IfcViewer` static library, which now bundles
scene/loader + renderer in one target. The pre-merge circular
dependency (IfcViewer linking IfcViewerWgpu just to get the
ViewportWindow.h include path that SceneLoader.h needs) goes away.
* The wgpu-native FetchContent block, the Cocoa/QuartzCore link on
Apple, the OBJCXX-enabled `.mm` source, and the wgpu-native runtime
install all move into `src/ifcviewer/CMakeLists.txt` unchanged.
## Type renames (Wgpu prefix dropped from every Wgpu* identifier)
WgpuAreaMeasurement → AreaMeasurement
WgpuBufferPool → BufferPool
WgpuLengthMeasurement → LengthMeasurement
WgpuMetalSurface → MetalSurface
WgpuModelGpuData → ModelGpuData
WgpuOverlayFrame → OverlayFrame
WgpuOverlayRenderer → OverlayRenderer
WgpuSectionPlane → SectionPlane
WgpuSelectionState → SelectionState
WgpuStreamingLoader → StreamingLoader
WgpuStreamingThread → StreamingThread
WgpuViewportWindow → ViewportWindow
WgpuVisibilityState → VisibilityState
CMake target IfcViewerWgpuMinimal → IfcViewerMinimal (exe name was
already this since wgpu shipped as default).
Deliberately kept: `onWgpuLog` (wgpu-native log callback — names a
binding to an external API, not one of *our* types), and the WGPU*
enum/struct prefixes from wgpu-native's own headers. `WgpuMemProbe`
lives in the separate `src/wgpu-mem-probe/` standalone diagnostic
project and isn't touched.
## Include-path updates
Every `#include "../ifcviewer-wgpu/Wgpu<X>.h"` → `"../ifcviewer/<X>.h"`,
every in-directory `#include "Wgpu<X>.h"` → `"<X>.h"`. Includes from
sibling subdirectories (modules/, etc.) are updated to point at
`../../../ifcviewer/` instead of `../../../ifcviewer-wgpu/`.
## cmake/CMakeLists.txt simplification
The redundant `add_subdirectory(ifcviewer-wgpu)` blocks (one inside
the BUILD_BONSAIVIEWER fan-in, one in the BONSAIVIEWER-less standalone
block) collapse into a single unconditional
`add_subdirectory(../src/ifcviewer ifcviewer)`. The standalone block
keeps only `wgpu-mem-probe` (the diagnostic tool, unrelated to the
viewer lib).
## Verification
* Full build green: `IfcViewer` static lib, `IfcViewerMinimal` exe,
`BonsaiViewer` exe, all four pre-existing ifcviewer unit tests, and
the two new-location tests (`test_selection`, `test_visibility`).
* No stray `Wgpu<X>` identifier remains across `src/ifcviewer/`,
`src/bonsaiviewer/`, `src/ifcviewer-minimal/` (verified by grep).
* Renames tracked by git as `R` entries — `git log --follow` on
ViewportWindow.cpp etc. continues to show history through the move.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Dion Moult