The .ifcview data is hugely redundant (repeated double instance matrices,
patterned indices) — measured 12x zstd whole-file. Server Content-Encoding
can't be used (it breaks HTTP Range), so compress PER-CHUNK into the format.
Format (v16): geometry becomes per-chunk zstd(vertices)+zstd(indices) frames —
each independently Range-fetchable, so streaming is intact — and the critical +
deferred metadata blocks are single zstd frames. SidecarChunk carries the
compressed blob offsets/sizes; applyStreamedChunk (render/upload) is UNCHANGED —
decompression slots into the fetch. Full readSidecar (test/tooling) reconstructs
by decompress+scatter. zstd: desktop links libzstd (also compresses at bake);
the web build (Emscripten has no zstd port) FetchContent's the pinned zstd
source and compiles its decompress-only subset for wasm — no vendored blob,
same version as desktop. New SidecarCompress wraps it (compress guarded off
under Emscripten). Both stream paths — desktop StreamingThread worker + sync
fallback (readChunkGeometryCompressed) and web beginWebChunkLoad — decompress;
readSidecarMetadataOnly / the web bootstrap / loadDeferredMetadataWeb decompress
the metadata blocks. streamingByteProgress reports COMPRESSED bytes. MEASURED: a
752 MB v15 federation → 75 MB v16 (10x; per-file 6.7-15.3x); PP-PLP 118→15 MB,
loads 13/13 chunks on web, 0 errors.
Three fixes found while testing big federations on a real server:
- Web-streamed race: streaming_from_web was set in the deferred-header callback
(a round-trip after the model+chunks exist), so driveStreamingLoads could take
the sync fopen path meanwhile → "failed to read/decompress chunk 0". Now set
immediately after applyCachedModel.
- OOM abort on 18 models: the pool grew unbounded until an alloc failed, but on
web that's an uncatchable bad_alloc abort. Cap total pool capacity
(setMaxTotalCapacity, 3 GB) so it stops before the heap ceiling, and raise
MAXIMUM_MEMORY 2→4 GB (wasm32 max) for headroom.
- Web never evicted (grow-or-block only). At the hard budget, fall through to the
LRU/priority evictor so a big federation stays navigable (highest-contribution
chunks win) instead of freezing with holes.
113/113 desktop + 6/6 web smoke pass. No back-compat: regenerate sidecars
(desktop bakes v16; scratch conv tool migrates v15→v16).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First-paint over a network is metadata-bound: the whole post-index metadata
(~10 MB on a 118 MB model) had to download before any geometry. But ~25% of
it — elements + string_table, the IFC element tree (names/GUIDs/hierarchy) —
is used only for UI/picking, never for rendering (ViewportCore never touches
it).
v15 splits the post-index metadata into a render-CRITICAL block (meshes,
instances, georef, chunk TOC) preceded by its byte length, then a DEFERRED
block (elements + string_table). The web loader reads only the critical block
before painting; the deferred block sits at a known, self-describing offset
([critical end, EOF)) and is fetched on demand. Desktop reads both (local).
Web on-demand path is wired and complete (not yet called — no UI consumer):
loadDeferredMetadataWeb(model_id) range-fetches + parses the deferred block
into ModelGpuData.elements/string_table, at most once; the first consumer
will be "show the selected object's name" on pick. No background prefetch —
view-only sessions never download the property data (saves 2.64 MB on this
model).
parseSidecarTail split into parseSidecarCritical + parseSidecarDeferred (pure,
unit-tested); StreamingSidecar gains the critical-block locator. Measured
(118 MB model): critical metadata 10.35 -> 7.71 MB, deferred 2.64 MB off the
path; first paint 10.5 -> 9.6 s @ 24 Mbps. (Instances still dominate the
critical block — the next metadata lever.) Format -> v15, no back-compat;
regenerate sidecars. 111/111 unit + 6/6 web smoke pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Splits the v13 metadata wire-format knowledge out of the FILE*-bound
streaming reader into pure, buffer-based functions so the web byte-range
path (#88) can reuse it without loading the whole sidecar into the wasm
heap:
- parseSidecarHead — validates the 16-byte head, yields num_vertex_bytes
- parseSidecarTail — parses meshes/instances/georef/elements/strings
from an in-memory tail buffer, bounds-checked
- planSidecarReadRanges + SidecarReadPlan — the range-coalescing /
scatter planner, promoted out of the anonymous namespace
readSidecarMetadataOnly and the range readers now call these; desktop
behaviour is unchanged (head + tail are small, the bulk is still skipped
via seek). The metadata tail is split from the head around the bulk
sections, so a blob-backed loader just slices those two regions and
hands the bytes to the same parsers.
Closes a coverage gap: StreamingLoader had no unit tests. Adds
test_streaming_loader.cpp (7 cases: metadata round-trip, corrupt/truncated
rejection, vertex+index range scatter, head validation, tail truncation,
read-plan coalescing). 107/107 unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The GL backend is gone (task #53). The wgpu/non-wgpu folder split and
the Wgpu* class prefix were both disambiguation artefacts from the
overlap period — now pure dead weight.
## Folder + library merge
* `src/ifcviewer-wgpu/` → folded into `src/ifcviewer/` (git mv tracks
every file as a rename so blame/log history survives).
* `src/ifcviewer-wgpu-minimal/` → `src/ifcviewer-minimal/` (the exe was
already named `IfcViewerMinimal`; this just brings the folder + CMake
target name into line).
* `src/ifcviewer-wgpu/tests/test_wgpu_{selection,visibility}.cpp` →
`src/ifcviewer/tests/test_{selection,visibility}.cpp`, folded into
the existing `add_ifcviewer_unit_test(...)` helper.
* The `IfcViewerWgpu` static library is dissolved — its sources become
part of the unified `IfcViewer` static library, which now bundles
scene/loader + renderer in one target. The pre-merge circular
dependency (IfcViewer linking IfcViewerWgpu just to get the
ViewportWindow.h include path that SceneLoader.h needs) goes away.
* The wgpu-native FetchContent block, the Cocoa/QuartzCore link on
Apple, the OBJCXX-enabled `.mm` source, and the wgpu-native runtime
install all move into `src/ifcviewer/CMakeLists.txt` unchanged.
## Type renames (Wgpu prefix dropped from every Wgpu* identifier)
WgpuAreaMeasurement → AreaMeasurement
WgpuBufferPool → BufferPool
WgpuLengthMeasurement → LengthMeasurement
WgpuMetalSurface → MetalSurface
WgpuModelGpuData → ModelGpuData
WgpuOverlayFrame → OverlayFrame
WgpuOverlayRenderer → OverlayRenderer
WgpuSectionPlane → SectionPlane
WgpuSelectionState → SelectionState
WgpuStreamingLoader → StreamingLoader
WgpuStreamingThread → StreamingThread
WgpuViewportWindow → ViewportWindow
WgpuVisibilityState → VisibilityState
CMake target IfcViewerWgpuMinimal → IfcViewerMinimal (exe name was
already this since wgpu shipped as default).
Deliberately kept: `onWgpuLog` (wgpu-native log callback — names a
binding to an external API, not one of *our* types), and the WGPU*
enum/struct prefixes from wgpu-native's own headers. `WgpuMemProbe`
lives in the separate `src/wgpu-mem-probe/` standalone diagnostic
project and isn't touched.
## Include-path updates
Every `#include "../ifcviewer-wgpu/Wgpu<X>.h"` → `"../ifcviewer/<X>.h"`,
every in-directory `#include "Wgpu<X>.h"` → `"<X>.h"`. Includes from
sibling subdirectories (modules/, etc.) are updated to point at
`../../../ifcviewer/` instead of `../../../ifcviewer-wgpu/`.
## cmake/CMakeLists.txt simplification
The redundant `add_subdirectory(ifcviewer-wgpu)` blocks (one inside
the BUILD_BONSAIVIEWER fan-in, one in the BONSAIVIEWER-less standalone
block) collapse into a single unconditional
`add_subdirectory(../src/ifcviewer ifcviewer)`. The standalone block
keeps only `wgpu-mem-probe` (the diagnostic tool, unrelated to the
viewer lib).
## Verification
* Full build green: `IfcViewer` static lib, `IfcViewerMinimal` exe,
`BonsaiViewer` exe, all four pre-existing ifcviewer unit tests, and
the two new-location tests (`test_selection`, `test_visibility`).
* No stray `Wgpu<X>` identifier remains across `src/ifcviewer/`,
`src/bonsaiviewer/`, `src/ifcviewer-minimal/` (verified by grep).
* Renames tracked by git as `R` entries — `git log --follow` on
ViewportWindow.cpp etc. continues to show history through the move.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Dion Moult