The .ifcview data is hugely redundant (repeated double instance matrices,
patterned indices) — measured 12x zstd whole-file. Server Content-Encoding
can't be used (it breaks HTTP Range), so compress PER-CHUNK into the format.
Format (v16): geometry becomes per-chunk zstd(vertices)+zstd(indices) frames —
each independently Range-fetchable, so streaming is intact — and the critical +
deferred metadata blocks are single zstd frames. SidecarChunk carries the
compressed blob offsets/sizes; applyStreamedChunk (render/upload) is UNCHANGED —
decompression slots into the fetch. Full readSidecar (test/tooling) reconstructs
by decompress+scatter. zstd: desktop links libzstd (also compresses at bake);
the web build (Emscripten has no zstd port) FetchContent's the pinned zstd
source and compiles its decompress-only subset for wasm — no vendored blob,
same version as desktop. New SidecarCompress wraps it (compress guarded off
under Emscripten). Both stream paths — desktop StreamingThread worker + sync
fallback (readChunkGeometryCompressed) and web beginWebChunkLoad — decompress;
readSidecarMetadataOnly / the web bootstrap / loadDeferredMetadataWeb decompress
the metadata blocks. streamingByteProgress reports COMPRESSED bytes. MEASURED: a
752 MB v15 federation → 75 MB v16 (10x; per-file 6.7-15.3x); PP-PLP 118→15 MB,
loads 13/13 chunks on web, 0 errors.
Three fixes found while testing big federations on a real server:
- Web-streamed race: streaming_from_web was set in the deferred-header callback
(a round-trip after the model+chunks exist), so driveStreamingLoads could take
the sync fopen path meanwhile → "failed to read/decompress chunk 0". Now set
immediately after applyCachedModel.
- OOM abort on 18 models: the pool grew unbounded until an alloc failed, but on
web that's an uncatchable bad_alloc abort. Cap total pool capacity
(setMaxTotalCapacity, 3 GB) so it stops before the heap ceiling, and raise
MAXIMUM_MEMORY 2→4 GB (wasm32 max) for headroom.
- Web never evicted (grow-or-block only). At the hard budget, fall through to the
LRU/priority evictor so a big federation stays navigable (highest-contribution
chunks win) instead of freezing with holes.
113/113 desktop + 6/6 web smoke pass. No back-compat: regenerate sidecars
(desktop bakes v16; scratch conv tool migrates v15→v16).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Makes large-model streaming over a network actually good — fixing read
amplification, then first-paint latency — building on the byte-range work.
v14 layout + TOC (SidecarLayout, pure + unit-tested)
The loader chunks meshes by spatial Morton order, but the sidecar stored
geometry in mesh-id order, so a chunk's meshes were scattered through the
file: streaming one chunk meant either hundreds of tiny range requests or
reading (and discarding) everything between them — a 113 MB model fetched
~340 MB, a 531 MB model 2.25 GB (4.2x). Fix: at bake, reorder meshes into
the loader's chunk order and rebuild vertex/index(LOD0+LOD1)/instance
sections so each chunk is one CONTIGUOUS byte range, and bake a chunk TOC
({first_mesh, mesh_count}). The loader builds chunks straight from the TOC
rather than re-deriving the plan — the float Morton quantisation isn't
bit-identical across toolchains (x86 baker vs wasm loader), so a re-derived
plan scatters the chunks. Format bumped to v14 (regenerate sidecars). The
reorder buckets instances by per-instance mesh_id (the baker never sets
MeshInfo.first_instance — trusting it scrambled every transform → geometry
at the origin). Multiset-verified on a 28,900-instance model: every
instance's placement + geometry preserved. Result: 531 MB fetches 531 MB
(1.0x) in 72 requests (was 2036).
Progressive streaming (concurrency cap + small chunks)
Even at 1x, geometry appeared only after ~the whole model arrived: the
browser multiplexes every in-flight Range request over one HTTP/2 conn, so
unbounded concurrency (9 in flight) split the bandwidth and nothing finished
until the end (measured: first paint after 113 of 118 MB / 35 s @ 24 Mbps).
Cap concurrent chunk loads (kMaxWebInflightChunks=2): the priority-sorted
top chunks finish and paint first, then the next → first paint 9 s. Chunk
size dropped 16->4 MB (cheap now that each chunk is one read; matches Cesium
3D Tiles / xeokit / SVF2) for smoother progression. First-paint is now
metadata-bound (~10 MB tail) — the next lever.
111/111 unit (new test_sidecar_layout: geometry preserved, contiguous layout,
Morton-identity) + 6/6 web smoke pass; desktop bake (SceneLoader) reorders
before writeSidecar; embedded web sample regenerated to v14.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Splits the v13 metadata wire-format knowledge out of the FILE*-bound
streaming reader into pure, buffer-based functions so the web byte-range
path (#88) can reuse it without loading the whole sidecar into the wasm
heap:
- parseSidecarHead — validates the 16-byte head, yields num_vertex_bytes
- parseSidecarTail — parses meshes/instances/georef/elements/strings
from an in-memory tail buffer, bounds-checked
- planSidecarReadRanges + SidecarReadPlan — the range-coalescing /
scatter planner, promoted out of the anonymous namespace
readSidecarMetadataOnly and the range readers now call these; desktop
behaviour is unchanged (head + tail are small, the bulk is still skipped
via seek). The metadata tail is split from the head around the bulk
sections, so a blob-backed loader just slices those two regions and
hands the bytes to the same parsers.
Closes a coverage gap: StreamingLoader had no unit tests. Adds
test_streaming_loader.cpp (7 cases: metadata round-trip, corrupt/truncated
rejection, vertex+index range scatter, head validation, tail truncation,
read-plan coalescing). 107/107 unit tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace qInfo() growth-event logging with fprintf(stderr,...) so
BufferPool.cpp has no Qt touchpoints. Lets the test target drop its
Qt6::Core link too. Prerequisite for the IfcViewerCore library boundary
the web target will link against.
The chunk planner (Morton sort + greedy pack) and instance composition
(federation × placement matrix chain + world-AABB derive) were inline
helpers in ViewportWindow.cpp. Pulled both out as free-function modules
so the math + lookup logic can be exercised without a Qt window or a
wgpu device. ViewportWindow now delegates; InstanceLookup is a using-
alias to InstanceCompose::InstanceLookup.
Also added an addSubBufferForTesting / clearSubPoolsForTesting seam to
BufferPool so the sub-allocator invariants can be pinned with fake
WGPUBuffer handles. The fakes are never dereferenced; the guard drops
the sub-pools before destructor would call wgpuBufferRelease.
Three new test binaries under src/ifcviewer/tests/, 33 cases / 173
assertions: BufferPool first-fit + alignment + coalescing + multi-
sub-pool isolation; ChunkPlanner Morton split / interleave / stable
sort / greedy-pack monotonicity and single-mesh-oversize; InstanceCompose
identity / translation / order-of-multiplication / large-placement
cancellation against federation false origin / column-major writeback /
findInstance lookup paths.
Unit / Geolocation / Placement are schema-agnostic IFC helpers ported
from ifcopenshell.util.{unit,geolocation,placement}. Nothing about
them is viewer-specific: pure IfcParse + Eigen, no Qt, no IfcGeom, no
renderer. Living under src/ifcviewer/ implies an unwanted dependency
direction every time a non-viewer caller (test_federation, the bonsai
SettingsView georef readout, a future standalone IFC tool) wants to
use them.
Move them to a new `src/ifcutil/` static lib (IfcUtil). The lib has
PUBLIC `target_include_directories(${CMAKE_CURRENT_SOURCE_DIR})` so
callers that link IfcUtil can keep `#include "Unit.h"` etc. without
relative-path adjustments — the include dir propagates transitively
via IfcViewer's PUBLIC link.
## Changes
* `git mv src/ifcviewer/{Geolocation,Placement,Unit}.{h,cpp}
→ src/ifcutil/` (history follows the rename).
* `src/ifcutil/CMakeLists.txt`: IfcUtil static lib, PUBLIC links
IfcParse + Eigen3::Eigen, PUBLIC include dir.
* `cmake/CMakeLists.txt`: `add_subdirectory(../src/ifcutil ifcutil)`
before ifcviewer/ so the link target exists when IfcViewer's
CMakeLists runs.
* `src/ifcviewer/CMakeLists.txt`: IfcUtil added to IfcViewer's PUBLIC
link_libraries.
* `src/ifcviewer/tests/CMakeLists.txt`: test_federation drops the
explicit `${IFCVIEWER_SRC}/{Unit,Geolocation,Placement}.cpp`
source list and links `IfcUtil` instead (matches how production
code resolves the symbols).
* `src/bonsaiviewer/modules/models/SettingsView.cpp`: the two
explicit `#include "../../../ifcviewer/{Geolocation,Unit}.h"`
paths swap to `../../../ifcutil/…`. All other callers use bare
`#include "Unit.h"` style and continue to work via the propagated
include dir.
## Verification
* `ninja -C build-viewer` builds clean: IfcUtil + IfcViewer +
IfcViewerMinimal + BonsaiViewer + all four pre-existing
ifcviewer tests + the two from-wgpu tests.
* `test_federation` runs green: 226 assertions in 22 test cases
pass with IfcUtil linked instead of the explicit-source compile.
* `git log --follow` traces e.g. `Geolocation.cpp` back through the
rename to its prior location in src/ifcviewer/.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The GL backend is gone (task #53). The wgpu/non-wgpu folder split and
the Wgpu* class prefix were both disambiguation artefacts from the
overlap period — now pure dead weight.
## Folder + library merge
* `src/ifcviewer-wgpu/` → folded into `src/ifcviewer/` (git mv tracks
every file as a rename so blame/log history survives).
* `src/ifcviewer-wgpu-minimal/` → `src/ifcviewer-minimal/` (the exe was
already named `IfcViewerMinimal`; this just brings the folder + CMake
target name into line).
* `src/ifcviewer-wgpu/tests/test_wgpu_{selection,visibility}.cpp` →
`src/ifcviewer/tests/test_{selection,visibility}.cpp`, folded into
the existing `add_ifcviewer_unit_test(...)` helper.
* The `IfcViewerWgpu` static library is dissolved — its sources become
part of the unified `IfcViewer` static library, which now bundles
scene/loader + renderer in one target. The pre-merge circular
dependency (IfcViewer linking IfcViewerWgpu just to get the
ViewportWindow.h include path that SceneLoader.h needs) goes away.
* The wgpu-native FetchContent block, the Cocoa/QuartzCore link on
Apple, the OBJCXX-enabled `.mm` source, and the wgpu-native runtime
install all move into `src/ifcviewer/CMakeLists.txt` unchanged.
## Type renames (Wgpu prefix dropped from every Wgpu* identifier)
WgpuAreaMeasurement → AreaMeasurement
WgpuBufferPool → BufferPool
WgpuLengthMeasurement → LengthMeasurement
WgpuMetalSurface → MetalSurface
WgpuModelGpuData → ModelGpuData
WgpuOverlayFrame → OverlayFrame
WgpuOverlayRenderer → OverlayRenderer
WgpuSectionPlane → SectionPlane
WgpuSelectionState → SelectionState
WgpuStreamingLoader → StreamingLoader
WgpuStreamingThread → StreamingThread
WgpuViewportWindow → ViewportWindow
WgpuVisibilityState → VisibilityState
CMake target IfcViewerWgpuMinimal → IfcViewerMinimal (exe name was
already this since wgpu shipped as default).
Deliberately kept: `onWgpuLog` (wgpu-native log callback — names a
binding to an external API, not one of *our* types), and the WGPU*
enum/struct prefixes from wgpu-native's own headers. `WgpuMemProbe`
lives in the separate `src/wgpu-mem-probe/` standalone diagnostic
project and isn't touched.
## Include-path updates
Every `#include "../ifcviewer-wgpu/Wgpu<X>.h"` → `"../ifcviewer/<X>.h"`,
every in-directory `#include "Wgpu<X>.h"` → `"<X>.h"`. Includes from
sibling subdirectories (modules/, etc.) are updated to point at
`../../../ifcviewer/` instead of `../../../ifcviewer-wgpu/`.
## cmake/CMakeLists.txt simplification
The redundant `add_subdirectory(ifcviewer-wgpu)` blocks (one inside
the BUILD_BONSAIVIEWER fan-in, one in the BONSAIVIEWER-less standalone
block) collapse into a single unconditional
`add_subdirectory(../src/ifcviewer ifcviewer)`. The standalone block
keeps only `wgpu-mem-probe` (the diagnostic tool, unrelated to the
viewer lib).
## Verification
* Full build green: `IfcViewer` static lib, `IfcViewerMinimal` exe,
`BonsaiViewer` exe, all four pre-existing ifcviewer unit tests, and
the two new-location tests (`test_selection`, `test_visibility`).
* No stray `Wgpu<X>` identifier remains across `src/ifcviewer/`,
`src/bonsaiviewer/`, `src/ifcviewer-minimal/` (verified by grep).
* Renames tracked by git as `R` entries — `git log --follow` on
ViewportWindow.cpp etc. continues to show history through the move.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
LodBuilder.cpp guards its real body behind #ifdef WITH_MESH_OPTIMIZER
(the stub is `return;`). The IfcViewer static lib propagates the
define via target_compile_definitions, but test_lod_builder compiles
LodBuilder.cpp standalone (it doesn't link IfcViewer), so the test
silently exercised the no-op path. summariseLods and buildLods cases
asserted on the post-build state and saw zero LOD1 output.
Pre-existing regression since 884e7ba32 ("Make meshoptim optional");
adds the define to the test target directly so the real build path
runs. 5/5 LOD cases pass after.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Bonsai now drives the wgpu viewport for both sidecar and direct-IFC
loads. The GL viewer and its supporting state classes are gone.
SceneLoader rewire:
- Takes WgpuViewportWindow* instead of ViewportWindow*.
- Sidecar path reads metadata only (readSidecarMetadataOnly) and hands
the StreamingSidecar off to the new applyCachedModel. Field accesses
inside applySidecarData go through .meta.
- Direct-IFC path uses the wgpu A-path (upload{Mesh,Instance}Chunk +
finalizeModel). The applyLodExtension call is dropped — wgpu has no
live LOD1 splice; LOD1 still lands in the on-disk sidecar for the
next open.
Bonsai migration:
- ViewportWindow → WgpuViewportWindow across MainWindow, Measurement,
SessionState, and every modules/*/{Commands,Panel,View}.{h,cpp} —
116 sites total. Same s/OverlayRenderer::/WgpuOverlayRenderer::/
rename, 12 sites.
- Includes flipped from ../ifcviewer/ViewportWindow.h to
../ifcviewer-wgpu/WgpuViewportWindow.h. OverlayRenderer.h include
dropped (transitively reached via the viewport header).
- BonsaiViewer links IfcViewerWgpu in addition to IfcViewer for the
duration of the migration; the GL-side IfcViewer also publicly links
IfcViewerWgpu so SceneLoader can resolve WgpuViewportWindow.
GL backend deletion:
- src/ifcviewer/ViewportWindow.{cpp,h}, BvhAccel.*, OverlayRenderer.*,
Selection.*, Visibility.* all gone.
- src/ifcviewer-minimal/ removed entirely (MinimalWindow drove the GL
viewport).
- src/ifcviewer/tests: test_bvh_accel, test_selection, test_visibility
removed. The first has no replacement (wgpu doesn't use a per-instance
BVH); the latter two are ported separately. test_lod_builder,
test_sidecar_cache, test_instanced_geometry, test_federation remain
(backend-agnostic).
- IfcViewer's CMakeLists drops OpenGL, Qt::OpenGL, Qt::Widgets — none
of the surviving translation units reach for them.
Build flag plumbing:
- BUILD_BONSAIVIEWER now auto-enables BUILD_BONSAIVIEWER_WGPU since
SceneLoader requires the wgpu lib for its WgpuViewportWindow* arg.
- The wgpu subprojects add_subdirectory ahead of the GL one so
IfcViewerWgpu exists when IfcViewer's link evaluates.
- src/ifcviewer-minimal subdir reference removed from cmake/CMakeLists.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
test_instanced_geometry previously re-implemented vertex quantization
inline, with a stale comment claiming the helpers still lived in
ViewportWindow.cpp. They now live in VertexQuantization.h, so route the
test through the real quantizeVertex/octEncodeNormal and add coverage
for the degenerate-axis path, octahedral normal round-trip, the i8
normal error bound (~0.78 deg worst observed), and color passthrough.
Add test_visibility and test_selection: Tier-1 coverage of the two
per-object viewport state machines. Both are QObjects for their
changed() signal but touch no GL on the construction/mutation path, so
the tests exercise the pure CPU logic without a context.
Suite goes from 39 to 61 cases.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Keep the IfcViewer test CMake in sync with the optional meshoptimizer dependency so test_lod_builder is only added when the package is enabled.
Generated with the assistance of an AI coding tool.
Commit f7add7f4 split getAxis2Placement out of an anonymous helper in
Geolocation.cpp into a shared Placement.{h,cpp}, but the test_federation
target's source list wasn't updated. The test binary failed to link
with `undefined reference to getAxis2Placement(express::Base const&)`
from Geolocation::getWcs. Add Placement.cpp to the explicit-source
list — it has no Qt dependency, only ifcparse.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds ModelGeoref { ModelUnits units; Eigen::Matrix4d stage2_meters; bool
has_stage2; } and computeModelGeoref(file*) in Federation.{h,cpp}. The
helper reads the project length unit, IfcProjectedCRS.MapUnit, helmert
parameters and WCS, and reduces them to a metres-in/metres-out stage 2
matrix using the existing Geolocation + Unit primitives. When the model
has no IfcMapConversion it returns an identity stage_2 with has_stage2
== false, so the upload pipeline can branch cheaply.
SceneLoader::Model gains a cached ModelGeoref; SceneLoader::modelGeoref
(uint32_t mid) computes lazily on first call (returns nullptr when the
IFC file isn't available yet — happens on the sidecar-hit path before
the data-source thread populates the streamer) and serves from cache
afterwards.
Not yet consumed by the upload pipeline; that's the next commit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds the structs that were briefly in src/ifcviewer/Federation.{h,cpp}
two commits ago, now folded into the merged Federation alongside the
file persistence layer:
- FederationConfig: federation-wide unit ({prefix, name}). Default
METRE; one-of an IfcSIUnit name with optional prefix or an
IfcConversionBasedUnit name.
- FederationOrigin: stage 3 — XYZ in federation unit + Z-rot.
Composes to R_z · T(-xyz_meters), nominating a point as origin.
- AFrame + ModelTransform: stage 4 intent — A (model project or
map unit, per a_frame), B and pivot (federation unit), full
intrinsic-XYZ Euler rotation in degrees.
- ModelUnits: per-model project_length_to_meters / map_unit_to_meters
cached at load time.
Free functions composeFederationOrigin and composeModelTransform
return Eigen::Matrix4d in metres. composeModelTransform takes the
model's stage-2 georef matrix so it can lift `a` into metres when
authored in ModelLocal.
Federation gains config_, origin_ members + setters that emit
dirtyChanged. Each Model carries a transform_intent. JSON I/O
emits config / origin always; transform_intent only when non-default.
Schema stays "ifcfed/1" — additive, optional, sane defaults.
Five new tests: round-trip of the new fields, default-omission
behaviour, two compose smoke tests for FederationOrigin, and one
verifying the "pivot at B preserves A→B" invariant of
composeModelTransform. All 36 ctest cases pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Move src/ifcviewer-full/Federation.{h,cpp} (and its tests) into
src/ifcviewer/ so the lib stays the single source of truth for the
federation data model. Restores the original "agnostic lib usable
from ifcviewer-full and ifcviewer-minimal alike" framing.
Drop the unused per-model transform[16] / has_transform field — it
was round-trip-only with no UI to author it, and is being replaced
by an intent-based ModelTransform in the next commit. No real
.ifcfed in the wild populated this field; old files still load
(unknown JSON keys ignored), they just lose the unused transform.
Replaces the pure-data-model Federation.{h,cpp} that was added a
few commits earlier — that file's structs and compose helpers
return as part of the merged Federation in commit 6.
ifcviewer-full's per-app tests dir is removed (test_federation was
the only one); BUILD_IFCVIEWER_TESTS now wires test_federation in
under src/ifcviewer/tests/, with the Qt6::Core/Gui/Test dependency
declared inline since unlike the other Tier-1 tests it has to pull
Qt in. All 31 tests pass.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Covers the pure-logic modules with no Qt event loop or GL context: BVH
build, LOD decimation, sidecar round-trip, instanced-geometry layout
constants, and Federation save/load + relative-path policy. Each test
binary compiles only the production source(s) under test, so the unit
tier doesn't pull Qt/OpenCASCADE/IfcGeom into the test build.
Gated behind BUILD_IFCVIEWER_TESTS=OFF; default builds remain offline.
Catch2 v3.5.4 is fetched on demand via FetchContent.
Dion Moult