mirror of
https://github.com/IfcOpenShell/IfcOpenShell.git
synced 2026-08-30 16:43:00 +00:00
0774398d4e02937f15aa721372159072bb605c3b
20 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dion Moult
|
0774398d4e |
wgpu: volume measurement tool (V hotkey, selection-driven HUD + per-object labels)
Ports Bonsai's volumeOfObjects + volumesPerObject onto WgpuViewportWindow. Mesh-local volumes are precomputed at applyCachedModel via signed- tetrahedra-from-origin (dequantising positions from the 12 B/vertex GPU layout); per-instance volume is just the cached local × |det(placement)|. No GPU readback — measurement is O(K) in the selection size. Streaming path computes volumes per-chunk as they arrive — fills any mesh whose chunk just delivered, then re-runs updateVolumeReadout if the user is staring at a Volume readout while the geometry pages in. UX matches GL: V toggles, Esc exits, selection-driven (LMB pick / marquee / Shift/Ctrl set ops all funnel into updateVolumeReadout). HUD shows total + count; one overlay label per object at its AABB centre, capped at 200 to keep the label-texture cache bounded on large marquees. Side fixes layered on the label overlay: - O(1) AABB lookup via object_id_to_instance instead of linear-scanning every model's instance list per selected object. - Label texture cache evicts entries not touched this frame, so churning through "X.XXXX m³" strings doesn't pin GPU memory. - DrawRec stores the WGPUBindGroup handle by value rather than a LabelTexture* pointer into the QHash — getOrCreateLabelTexture can rehash the table and invalidate every captured pointer, which crashed large marquee selections with BindGroup-no-longer-alive. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
72af067c9a |
wgpu: streaming + HiZ correctness fixes
Three correctness bugs found and fixed, plus an unrelated fly-mode deadlock surfaced along the way. HiZ false-rejection at the bottom of the screen ------------------------------------------------ The mip-pyramid sizing floored when halving — for a 256×70 mip 0 the level-3 mip is 32×8, but mip-0 row 69 maps to ly = 69>>3 = 8, which is out of bounds for an 8-row mip. ly1 then clamps down to 7 while ly0 stays at 8, the sampling loop runs zero times, max_d retains its initial 0.0, and `min_z > 0` rejects every AABB whose projected y range touches the bottom row. Same class for the right edge on very wide viewports. Fix: ceil rather than floor when halving mip dimensions so every parent row has a covering child texel, plus std::clamp on both lookup endpoints as belt-and-suspenders for any future mip-sizing change. Surfaced after the user added more sidecars and saw "anything near the bottom of the screen, no matter close or far" disappear ~0.5 s after camera stops — that delay was the strict-VP gate + readback latency opening the HiZ window. Found via WGPU_HIZ_TRACE rejection logs that showed every rejection had `max_d=0` and `ly0 > ly1`. Streaming priority lets the ocean starve out the bracing --------------------------------------------------------- Per-instance projected screen footprint was estimated as bounding- sphere radius squared. BIM geometry is overwhelmingly thin-in-one-axis (slabs, pipes, columns, windows) and a flat ocean plane viewed nearly edge-on gets a sphere projection ~250× larger than its actual screen rect. Its chunk dominated the priority ranking and evicted the brace chunks despite the braces being one of the closest visible things. Fix: per-instance priority is now the screen-space AABB rectangle area (world AABB extents projected onto the camera right/up basis vectors, divided by view-z). Sphere radius is retained for the contribution cull and LOD pick because conservative-over is the right failure mode there. Stale-VP HiZ gate ----------------- HiZ resolves into an async ping-pong of staging buffers, so the pyramid resident at cull time was typically captured one or two frames ago. During camera motion the captured VP differs from vp_this_frame and AABBs end up sampling depth taken for what was at slightly-different screen positions in the old view. Strict by default now: HiZ engages only when hiz_vp_ == vp_this_frame. WGPU_HIZ_MOTION=1 trusts the stale pyramid (matches GL's default). Fly mode Shift+Q deadlock -------------------------- keyPressEvent requested a redraw only on the first key of a new held set (was_empty). Pressing Shift first then Q never satisfied that condition because Shift had already populated the set, so the render loop never ticked. Now every relevant keypress calls requestUpdate unconditionally. HiZ stays opt-in behind WGPU_HIZ=1 for one release while the fix bakes; WGPU_HIZ_TRACE=1 keeps the per-rejection diagnostic available for future bugs. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
1f8f6bffc4 |
wgpu cull: refactor per-mesh chunk lookups to per-instance
Preparatory refactor for spatial instance bucketing (#55). Cull previously routed through per-mesh tables (mesh_chunk_idx, mesh_chunk_local_base_vertex, _ebo_first_u32, _lod1_first_u32) to find the chunk and chunk-local offsets for each instance. That assumes a mesh lives in EXACTLY ONE chunk — the assumption holds under the current mesh-keyed planner but breaks under spatial bucketing, where the same mesh can be duplicated across multiple buckets if its instances are scattered. Adds four per-instance arrays (instance_chunk_idx, instance_base_vertex, instance_ebo_first_u32, instance_lod1_first_u32) populated at planning time. The current mesh-keyed planner derives them by translation: instance_chunk_idx[i] = mesh_chunk_idx[instances[i].mesh_id] The spatial-bucket planner (next commit) will populate them directly, allowing the same mesh_id to map to different chunks for different instances. cullModelCpuCompute now reads the per-instance arrays: - frustum_visible_count uses chunks[instance_chunk_idx[i]] - VisibleDrawGpu uses instance_base_vertex / instance_ebo_first_u32 / instance_lod1_first_u32 - LOD-select branch and use_lod1 logic unchanged. Per-mesh tables stay (used by makeChunkRequest, debug logs, the planner itself). Memory cost: 16 bytes × N instances ≈ 16 MB on a 1M-instance scene. Pixel-identical on basic.ifc in both non-streaming and streaming modes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
a63bc63439 |
wgpu: diagnostic instrumentation for cull/streaming perf
Adds three knobs and three new heartbeat numbers to support the ongoing perf-parity work. None affect behaviour in default runs. cull[wall|compute|upload] split timer The existing cull_timer wrapped both the parallel std::async dispatch and the sequential cullModelCpuUpload loop (queueWriteBuffer × 3 per resident chunk × ~120 chunks ≈ 360 wgpu calls per frame). Splitting them ruled upload out as the bottleneck on a 51-model federation scene: compute ≈ 16-17 ms, upload ≈ 1 ms. WGPU_CULL_THREADS=0 — force sequential cull std::async-per-model was already in place; this env var disables it so we can compare wall time vs sequential and confirm parallelism is working. On the federation scene with 52 models: sequential 74 ms vs parallel 17 ms = 4.4× speedup. Confirmed; the 17 ms floor is not a parallelism failure, it's the cost of culling the largest single model (model 43, 114k instances) ÷ no parallelism within that model. WGPU_STREAM_DEBUG=1 — per-frame [stream-debug] log Surfaces cands/enq/drained/ev_lru/ev_pri/blocked/resident/cycled/ max_load each frame from driveStreamingLoads. The "cycled" / "max_load" pair makes thrash vs eviction-churn vs just-loading distinguishable. Off by default; opt-in via the env var. Bench-warm timeout dump When [bench warm] times out (600 frames without 0-loads streak), prints a structured summary: resident/missing/total chunks, cycled count, pool usage, largest free run, avg missing chunk size, and an auto-classifier diagnosis (POOL FRAGMENTED vs WORKING SET > POOL vs FEW-CHUNK CYCLE vs still-loading). Caught a real fragmentation pattern (18 MB largest free run vs ~100 MB typical chunk) on a 51-model run where the dumb classifier would have called it a load-budget problem. LOD1 firing counter "lod1 X/Y (saved Z tris, N no-lod1)" suffix on the [frame] log. X = LOD1-selected this frame, Y = LOD1-eligible, Z = tris not drawn vs always-LOD0, N = visible instances with no baked LOD1 (mesh below IFC_LOD_MIN_TRIS). Confirmed LOD1 path is genuinely firing post the per-chunk LOD1-storage commit, and exposed that ~90% of instances in real scenes are no-lod1 meshes — relevant to the future LOD-tier-residency design. Chunk.load_count + Chunk.lod0/1 layout bookkeeping Per-chunk reload counter for the thrash detector. lod0/1 layout_count fields prep the data model for distance-tiered residency (Phase B of #31) but aren't acted on yet. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
10f8fc88e8 |
wgpu chunks: pack LOD1 indices alongside LOD0; cull picks per-instance
Sidecar LOD1 is per-mesh, index-only — meshoptimizer-baked decimated index slices that share the LOD0 VBO. Before this change the wgpu chunk path force-disabled LOD1 (effective_lod1 = false; tagged in a comment as "until per-chunk LOD1 storage lands"), so it had to walk every visible instance at full LOD0 even when the per-instance LOD selection said the projected radius was below the LOD1 threshold. Per-chunk layout: append LOD1 indices for the chunk's meshes after all LOD0 indices in the same pool slice. A new mesh_chunk_local_lod1_first_u32 array records each mesh's LOD1 starting offset (in u32s) within the chunk's index slice; LOD0 offsets stay where they were. The cull's emit then sets VisibleDrawGpu.ebo_first_u32 to whichever side matches the per- instance use_lod1 decision the prior #8 commit already computed. Vertex pulling is oblivious to the LOD split — it just reads the indices the cull pointed it at. c.index_count is repurposed as the LOD0+LOD1 total so the pool allocation, eviction's pool-fit check, and the VRAM accounting all scale automatically. c.lod1_index_count exposes the LOD1 portion for stats. makeChunkRequest appends LOD1 byte ranges to req.i_ranges in the same per-mesh order; the streaming worker concatenates ranges in order, so the assembled idx blob lands LOD0-first / LOD1-second which matches the chunk-local packing. On a 10-model regen with meshoptimizer-baked sidecars, the bench [frame] heartbeat shows lod1 firing on ~80-90% of LOD1-eligible instances and saving 10-30M tris per frame versus the prior LOD0-only ceiling. Same camera/scene on basic.ifc is still pixel-identical (no mesh in basic.ifc is large enough to bake a LOD1, so the cull just follows the LOD0 path it always did). Temporary debug counters (lod1_dbg_count_ et al.) print "lod1 X/Y (saved Z tris, N no-lod1)" in both interactive and benchmark [frame] heartbeats — kept on while LOD1 correctness gets confirmed across more scenes, will come out once trust is built. The non-streaming applyCachedModel path runs the same LOD1 plumbing but no longer fits the full federation scene in pool (extra index bytes push past the 2 GB single-buffer cap); that mode was already streaming-only on that scene before. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
c7fba1abb8 |
wgpu streaming: screen-space AABB priority + grace period + interactive heartbeat
The chunk priority metric is now the 2D projected pixel area of the chunk's AABB on screen — 8 corners projected through view-projection, 2D axis-aligned bbox of the projected points, clamped to viewport. This replaces the prior bounding-sphere-radius² metric, which was a 3D approximation: it treated a 322 × 55 × 5 m slab as a 163 m sphere, giving it the same huge priority face-on or edge-on. The new metric genuinely answers "what would this chunk's AABB cover if rendered solid given the current camera and viewport." Newly-loaded chunks get a 30-frame grace period at full priority (visibility_history floor temporarily forced to 1.0). Without it, just-loaded chunks crashed to history=0 → effective priority = pri × 0.05 → immediately reverse-swapped by the chunk they displaced. Cycle starved the per-frame load budget so candidates ranked below the cyclers never got attempted. 30 frames = HISTORY_ALPHA's time constant — enough for visibility_history to develop meaningfully. EVICT_PRIORITY_RATIO bumped 1.21 → 2.0 to suppress more swap noise between similar-priority chunks. Interactive heartbeat log added: every render in non-bench mode prints [frame] with fps, ms, obj, sub_draws, hiz_rej, cull, stream, chunks breakdown (resident/frustum/total + missing count), VRAM, model count. Every 30 frames when something's missing, also dumps: - top 8 models by missing chunk count - top 20 missing chunks by priority (with AABBs) - bottom 5 residents by effective priority - all chunks of brace.ifc (one-off diagnostic, hardcoded for the brace-visibility investigation) The heartbeat made the streaming bug visible: a brace model that isolation-loads correctly is missing in the full set because slabs covering more pixels win the priority contest. Per-model fairness or manual pinning are the remaining options if pixel-area + grace + hysteresis isn't enough — left for follow-up so the user can decide based on real testing. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
3d368e0079 |
wgpu chunks: 3D Morton-code spatial sort (tight voxel chunks)
The previous chunk-plan sorted meshes by lexicographic (z, y, x) centroid — effectively a 1D Z-slab traversal. On a typical IFC building (50 × 50 × 100 m), a 16-MB chunk's 80-ish meshes spanned roughly 50 × 50 × 0.5 m. On a city federation it was much worse: the first chunk grouped ground-floor stuff from every building, spanning the entire scene horizontally. Per-chunk AABBs that wide make frustum / contribution / HiZ rejection useless (every chunk "overlaps the frustum" by virtue of spanning the whole scene). 3D Morton (Z-order) interleaves bits of quantised (x, y, z) centroids, so consecutive items in the sorted order cluster in all 3 axes — chunks become tight 3D voxels of the model. Prerequisite for the contribution-aware eviction priority (task #25) to actually discriminate near and far chunks. 21 bits per axis = ~2 M bins per axis, sub-millimetre precision on a kilometre-scale scene. Both apply paths (streaming and non- streaming) share the same sortMeshIdsByMorton helper. Benchmark unchanged (~47 fps avg, 20 ms cull, 0.3 ms stream) — the distance-based evictor still keys on chunk centres, which moved slightly under Morton but not enough to materially shift residency. The user-visible win comes from the next commit, which switches priority to screen-space contribution × HiZ history — both of which need today's tight AABBs to mean anything. Pixel-identical to non-streaming on basic.ifc on both paths. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
dcc2bf1c01 |
wgpu streaming: background-thread chunk I/O kills render-thread stutters
The sync chunk-read on the render thread was causing 100-300 ms spikes during orbit whenever a new chunk needed to scatter-gather its mesh bytes from disk. p99 was 326 ms on the close-camera benchmark. New WgpuStreamingThread: one worker thread with a condvar-protected request/result queue. driveStreamingLoads becomes drain-then-enqueue: 1. Drain any results the worker pushed since last frame. For each, pool-allocate slices + queueWriteBuffer + build the chunk bind group (still main-thread because wgpu queue ops aren't thread-safe). 2. Walk visible non-resident chunks (sorted by distance), evict to make pool room, and enqueue the request. Chunk gains is_loading flag to prevent re-enqueueing while in flight. loadChunkBytesAndUploadGpu becomes the sync fallback path, used only when a screenshot is pending — the deferred-capture wait would otherwise let the window manager re-layout the window between frames and the test framework would capture at the wrong size. Normal streaming always goes through the worker. Bench warm-gate / requestUpdate gating updated to consider streaming_thread_.inFlightApprox() so we don't declare "converged" while a worker read is still in flight, and the render loop stays alive until the worker queue is empty. Refactored loadChunkBytesAndUploadGpu into two helpers: - makeChunkRequest: builds the worker request from chunk metadata - applyStreamedChunk: pool.alloc + queueWriteBuffer + bind group Both the sync and async paths share applyStreamedChunk. Benchmark (big federation, --streaming): close camera: avg 24 fps p99 47 ms (was 27/326) default camera: avg 24 fps p99 46 ms (was 31/186) stream time: ~2 ms (was 8-12) cull is now the bottleneck (20 ms median) — task #17 (GPU compute cull) is the next frontier. Pixel-identical to non-streaming on basic.ifc on both paths. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
6f66d08bee |
wgpu cull: chunk-level frustum cull replaces BVH walk
cullModelCpuCompute previously had two paths: a flat linear scan over
all instances (default), or a BVH-stack walk (--bvh, gated off because
it regressed on dense scenes — the BVH built per instance but its
interior-node AABBs spanned huge chunks of model so most subtrees
straddled the frustum and the walk overhead beat the rejection win).
With spatial chunk planning (commit
|
||
|
Dion Moult
|
4d36174200 |
wgpu streaming: spatial chunk planning + coalesced multi-range reads
Chunks are now grouped by world-space centroid instead of mesh-id
range, so each chunk's AABB tightly bounds its geometry instead of
spanning the whole model. Distance-based eviction can finally
distinguish the near corner of a skyscraper from the far corner.
Algorithm:
1. Compute each mesh's centroid = mean of its instances' world AABB
centres.
2. Sort mesh indices lexicographically by (z, y, x) centroid. Stable
sort keeps mesh-id order as tiebreaker for instanced repeats.
3. Greedy-pack sorted meshes into chunks ≤ WGPU_CHUNK_VERTEX_BYTES_LIMIT.
4. Each Chunk stores its mesh_ids list; the per-mesh layout (chunk_local
base_vertex / ebo_first_u32) is computed by walking the list at plan
time.
Loader: chunk vertex/index bytes are no longer file-contiguous, so
streaming uses new multi-range read paths
(readSidecarVertexRanges / readSidecarIndexRanges). Each range list
is sorted by file offset and adjacent ranges coalesced with a 64 KB
gap tolerance — on the close-camera benchmark this brings the
per-chunk seek count back down to ~mesh-id-grouping levels, so the
spatial sort costs ~nothing on I/O while delivering tighter AABBs.
Non-streaming applyCachedModel mirrors the spatial plan but gathers
from in-memory data.vertices / data.indices via per-mesh
queueWriteBuffer calls at chunk-local offsets.
Chunk struct drops vertex_byte_offset and index_first_u32 (no longer
meaningful — each chunk is N scattered ranges). vertex_byte_size and
index_count stay as aggregates for pool sizing + eviction math.
Tuning: kept WGPU_CHUNK_VERTEX_BYTES_LIMIT at 128 MB. Tried 8 MB and
32 MB; both gave tighter AABBs but the scatter-gather I/O cost blew
up because the per-frame load count grows linearly as chunks shrink
(orbit shifts the working set faster across finer chunks). 128 MB +
coalescing is the empirical sweet spot pre-v14. Once sidecar v14
re-orders bytes on disk to match spatial chunks, we can drop the
limit to ~8 MB for sharp eviction without re-paying the seek cost.
Benchmarks (big federation, --streaming):
close camera: avg 36 fps median 53 (was 35/49) — parity
default camera: avg 33 fps median 47 (was 40/49) — small regression
likely from increased coalesce overhead on more-
scattered orbit traversals; will resolve with v14.
Pixel-identical to non-streaming on basic.ifc on both paths.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Dion Moult
|
c3a55d7f7b |
wgpu streaming: multi-pool growth, frustum-only residency, sorted convergence
Five interlocking fixes that take --streaming on the big federation scene from "5 fps + endless flicker + infinite cold-load" to a stable 35-49 fps with a converged working set. 1. Multi-sub-buffer WgpuBufferPool. Pool now grows lazily by adding sub-buffers of per_sub_buffer_capacity_ when alloc demand exceeds existing free runs. Each Slice carries (buffer, offset, size, sub_idx). On driver refusal of addSubBuffer, growth_disabled_ latches so subsequent allocs don't keep retrying and log-spamming. pool_can_fit consults can_grow() to know when growth could rescue a candidate vs when eviction is the only path. 2. Split cull / stream benchmark timers. The previous "cull[wall]" metric was actually cull + driveStreamingLoads, blaming the wrong subsystem (~170 ms of "cull" was synchronous disk I/O). 3. frustum_visible_count on Chunk, populated in cullModelCpuCompute right after the per-instance aabbInFrustum check. driveStreamingLoads now keys residency on this instead of total_visible_draws (which includes contribution + HiZ). HiZ visibility flips frame-to-frame as occluders shift; using it for residency caused chunks to be evicted then immediately re-loaded, every frame, even with a stationary camera — both the perf cliff and the visible flicker. 4. Distance-sorted candidates in driveStreamingLoads. Walk the non-resident frustum-visible chunks in distance order (closest first). With sorted processing, evict_farthest_than converges monotonically: each swap replaces a far resident with a closer candidate; once the next candidate is farther than every remaining resident, the loop exits. Without sorting the loader visited candidates in model/chunk-id order, swapping random chunks every frame without ever converging. 5. 10% eviction hysteresis (EVICT_DIST2_RATIO = 1.21). On scenes where many chunks are clustered at similar distance from the camera (e.g. several chunks all ~370 m away), naive "evict any resident strictly farther than candidate" triggers sub-meter swaps every frame, never resting. Requiring the victim to be 10% farther in linear distance kills these cycles while still allowing genuine "much closer" candidates to evict. Plus: latched bench_warm_done_ on the cold-load gate, with a 5-frames-of-zero-loads convergence test (default-camera big scene converges in 20 frames) and a 600-frame timeout fallback that prints exactly once. Measured on the test federation (111 sidecars, ~3 GB raw, 1 M instances) with the user's close-in camera: - avg 35 fps (was 5), median 49 fps (was 7) - cull 19 ms (now the bottleneck), stream 5-8 ms (was 172) - p99 184 ms — occasional big-chunk load on the render thread; background-thread I/O would smooth that out as a follow-up. With the default wide camera: - avg 40 fps, converges in 20 frames, residency grows naturally from 59 → 76 chunks as orbit shifts the frustum. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
502c29fbc2 |
wgpu: probed-size pool replaces per-chunk createBuffer
Drops the per-machine "guess the OOM ceiling" budget knob in favour of a single buffer pool whose capacity is *probed* at device-init time. The runtime answers the question: descend from min(maxBufferSize, 4 GB) through OOM error scopes, accept the largest size that allocates cleanly. On a desktop wgpu-native v29 box this lands at 2 GB; on browser-class platforms it'll land at 256 MB – 1 GB depending on the implementation. Same code path either way. Architecture: - WgpuBufferPool (new): single WGPUBuffer + free-list sub-allocator with adjacent-range coalescing and first-fit. 256 B alignment for storage-binding offsets. - Chunks now hold (pool_vertex_offset, pool_vertex_size) and (pool_index_offset, pool_index_size) instead of per-chunk WGPUBuffer handles. Load = pool.alloc + queueWriteBuffer. Unload = pool.free. - Bind groups bind pool_.buffer() at the chunk's specific (offset, size) for both the vertex and index storage bindings. - Eviction queries pool.largest_free_run_bytes() instead of a tracked budget; the two-phase LRU/distance evictor's policy is unchanged. What this fixes: - No more gpu-alloc-rs fragmentation OOM: one VkDeviceMemory block instead of N per-chunk blocks with rounding overhead. On the test dataset (~3 GB on disk, 562 k visible instances) the wgpu backend now runs through to render without OOM at any point. - No --streaming-vram-mb knob, no hardcoded budget constant, no per-machine calibration. The pool size adapts to whatever the runtime grants. Notes: - Error scope probing: wgpu-native v29 classifies "Not enough memory left" as WGPUErrorType_Validation, not OutOfMemory. We push both filters (nested) and treat either firing as probe failure. - The 4 GB probe cap is principled, not magic: above that, wgpu-native's advertised maxBufferSize is sometimes a sentinel (1 TB) that just forces wasteful halving steps. 4 GB is the largest buffer any realistic WebGPU implementation will grant a single allocation today. - Pool destroy()/release happens after model release in shutdown() so the underlying buffer outlives every bind group that references it. Follow-ups: spatial chunking (task #22) for finer eviction granularity; cull perf needs work at 100+ models / 1M+ instances (separate from streaming concerns). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
71e61dd8a5 |
wgpu streaming (5/4): per-chunk indices + LRU/distance eviction (stopgap)
Defers index buffers per-chunk (alongside vertex bytes) so streaming fully delivers on its "don't load until visible" contract — the previous per-model index buffer was upfront-loaded and tipped scenes >~1.5 GB into allocator OOM at frame 1. Adds residency tracking + a two-phase evictor: (1) drop LRU non-visible chunks first, (2) if everything resident is visible-this-frame, drop the farthest-from-eye chunk only when the candidate to load is closer. This gives monotonic convergence to "closest visible chunks fit the budget" instead of "first 4 win, rest never load." Default budget set to 1 GB — explicitly a stopgap, documented inline. The per-machine OOM ceiling on wgpu-native (caused by allocator fragmentation from one VkDeviceMemory per createBuffer call) cannot be solved by tuning this knob. The proper fix is a probed single-pool buffer with sub-allocation, tracked under task #16. Caveat: LOD1 indices are now force-disabled when chunking — per-chunk buffers only carry LOD0. Re-enabling needs LOD1 to participate in the chunk plan. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
d368ee449d |
wgpu streaming (2/4): per-chunk residency fields on WgpuModelGpuData
Foundation for streaming. Adds to each Chunk:
- is_resident (default true; streaming flips false initially)
- vertex_byte_offset / vertex_byte_size in the sidecar file
- aabb_min / aabb_max world-space chunk bounds (used by future cull
and streaming priority)
Plus on the model:
- streaming_file_path (non-empty = streaming path was used)
- streaming_vertex_section_offset (where the chunks live in the file)
All fields default to backward-compatible values: is_resident=true,
streaming_file_path empty. The existing non-streaming applyCachedModel
sets up a Chunk with is_resident=true (implicit) and ignores the
streaming fields, so no behaviour changes yet.
Commit 3/4 wires the metadata-only reader from (1/4) through a new
applyCachedModelStreaming path that flips is_resident=false initially;
commit 4/4 adds the per-frame loader that brings chunks resident on
demand. This commit is verified pixel-identical to the previous render
on basic.ifc.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Dion Moult
|
a1693259b8 |
wgpu backend: BVH cull (opt-in via --bvh, default off)
Stage 15 implementation lands but doesn't pay off as default-on. On a 562k-instance / 18-model scene with a centred camera, the BVH walk adds ~10 ms of cull cost without rejecting enough subtrees to compensate — every interior node's AABB straddles the frustum, so descents go all the way to leaves anyway. Linear scan beats it by that 10 ms. GL's BVH works better mainly because they do full cull (frustum + HiZ + contribution) at every node — their per-test cost is lower (likely SIMD-vectorised) and they get more subtree rejections. My current impl does frustum-only at interior nodes (HiZ there cost more than it saved on the smaller dataset). For now, gate the whole BVH walk behind --bvh, default off. The infrastructure (BvhAccel build at applyCachedModel, walk in cull, release) stays in place so it's a one-flag toggle to measure either side. Real default-on requires further tuning — see updated task #15. Measured on 562k-instance scene: --bvh on → 25.9ms total (cull 25.4ms) --bvh off → 15.4ms total (cull 14.5ms) ← default For comparison, GL on the same scene + camera: GL → 18.2ms total (cull 8.5ms wall, multi-threaded BVH) Net: wgpu beats GL by ~3ms total despite slower cull, because the GPU side (no edge-pass cost, async HiZ readback, lean main pipeline) gives back more than the cull deficit. Also added task #17 (GPU compute-shader cull) as the asymptotic answer — both backends hit CPU cull as the ceiling on ≥500k scenes; moving it to a compute shader drops it to sub-ms regardless. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
7dc13eb104 |
wgpu backend: chunk vertex storage to fit browser limits + settle frame
Two pieces: 1. Per-chunk vertex storage (stage 13) WebGPU mandates maxStorageBufferBindingSize ≥ 128 MB. Real BIM models routinely exceed that (one of yours is 139 MB vertex). Without chunking, every browser load would fail with "exceeds max_storage_buffer_binding_size". Strategy: each model's vertex data is split into ≤ 128 MB chunks at applyCachedModel time. Each chunk gets its own vertex_storage buffer, visible_draws / prefix_sums buffers, per_chunk_uniform, and bind group. Index buffer, instance storage, and mesh storage stay single-per-model (they fit well under the cap on every scene we've seen). Mesh-to-chunk assignment is bake-time-deterministic (walks meshes in order, opens a new chunk when adding the next would overflow). Cull buckets visible instances by their mesh's chunk; render issues one drawcall per non-empty chunk per model. WGSL is unchanged — the binary-search vertex pulling works identically per chunk because base_vertex is now CHUNK-LOCAL (the chunk's bind group binds its own vertex_storage). Single code path: chunking is ALWAYS on at 128 MB regardless of target. Cost on desktop is a handful of extra drawcalls per frame (1 per non-empty chunk; typical models = 1-3 chunks). Negligible. A mesh whose vertex range is itself > 128 MB can't fit in any chunk and would need splitting — typical IFC meshes are nowhere near that (hundreds of verts), and applyCachedModel warns loudly if one ever appears. --web-limits CLI flag requests the WebGPU mandatory floor limits (128 MB max storage binding, 256 MB max buffer) instead of the adapter's actual max. Used to verify chunking actually fits through browser constraints — turns "trust me, web will work" into a hard test. The 139 MB scene loads cleanly with --web-limits. 2. Settle frame after motion (bug fix) Reported regression: after orbiting, sub-pixel instances dropped by motion-mode contribution culling stayed missing after the camera stopped. Event-driven rendering means no frame is scheduled after mouse-up, so the cull never re-ran at the still threshold. Fix: track last_cull_was_motion_. If this frame used the motion threshold, requestUpdate() after present to schedule one settle frame. Next frame: camera_moved = false → still threshold → small instances reappear. Matches GL's last_cull_was_motion_ behaviour. Verified pixel-identical on basic.ifc; loads the user's dense scene successfully under --web-limits (chunks=2 on the 139 MB model, chunks=1 on the others). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |
||
|
Dion Moult
|
51dc31a50b |
wgpu backend: 10× perf — megadraw, async HiZ, parallel cull, motion mode
Closes the perf gap to the GL backend on real BIM benchmarks. On a 10-
sidecar / 380k-instance corpus at a fixed --camera the wgpu binary went
from 110.6 ms to 11.6 ms (vs GL's 23 ms — half the frame time, but
note GL is doing extra work the wgpu backend hasn't ported yet; see
the caveats list at the bottom). Bundled because the pieces interlock
and shipping any of them without the others reintroduces the same wall.
1. Cross-mesh vertex pulling (single mega-draw per model)
The previous one-drawIndexed-per-(mesh × LOD-bucket) loop was costing
~13ms on a 27k-mesh scene. CPU now emits a flat visible_draws[]
(16 B per visible (mesh,lod,instance)) plus a prefix_sums[] table.
WGSL binary-searches prefix_sums by @builtin(vertex_index) to find
the entry, then manually fetches the mesh-local index from a
storage-bound indices[] and pulls the packed 12 B vertex. No
setIndexBuffer; the shader reads everything from storage. Bind
group grew from 4 to 7 entries (vertices, meshes, instances,
indices, visible_draws, prefix_sums, per-model uniform) — well
under WebGPU's mandatory 8 storage / 12 uniform floor.
2. Async HiZ readback via ping-pong staging buffers
Sync wait via wgpuInstanceProcessEvents was costing ~37 ms on a
real scene (GPU drain). Two staging slots now ping-pong: frame N
kicks a non-blocking mapAsync on slot K, frame N+1's first action
is one processEvents drain. Pyramid is 1-2 frames stale — matches
the "slightly-stale depth, fine" pattern the GL backend already
documents. encodeHizResolve returns -1 (skip) if both slots are
in flight; cull keeps using the most recent pyramid.
3. Cull reorder: contribution before HiZ
HiZ projection is ~10× more expensive than the contribution
check, yet most contribution-survivors would be HiZ-rejected
anyway on dense scenes. Computing projected_px first lets
contribution short-circuit ~80% of HiZ tests with no rejection-
quality loss. Saved ~34 ms on the dense bench.
4. Motion-mode contribution threshold
AppSettings::motionMinPixelRadius parity. While the camera is
changing (orbit/pan/zoom/--benchmark sweep), drop instances
below 10 px instead of 2 px. Halves visible_objects during
motion with no perceived quality loss.
5. Parallel cull (std::async across models)
Per-model cullModelCpu split into Compute (CPU-only, thread-safe)
+ Upload (main-thread wgpu queue writes). std::async fan-outs the
compute across models; main-thread joins and uploads. Wall-clock
cull on the 10-model corpus drops from ~17 ms single-threaded to
~9 ms across cores.
6. --no-hiz CLI flag + per-phase benchmark timings
Benchmark now also prints "per-frame avg ms: cull=X
hiz_readback=Y" so future regressions can be attributed without
guesswork. --no-hiz toggles the master switch from the CLI.
Honest caveats — wgpu is currently faster mostly because GL is doing
work we haven't ported yet:
- Edge silhouette pass (stage 9) will add ~3-5 ms back to wgpu.
- GL's HiZ uses the BVH so it rejects whole subtrees (1.7k vs
our 358 rejects on the same scene). BVH for HiZ is future work
(task #13 / a new task) — until then we draw more sub-pixel
geometry that's behind closer surfaces. Visually correct, perf
cost paid. Stage 4+5 are unaffected.
Verified pixel-identical on basic.ifc through every change. Real-scene
visual diff against GL pending the --screenshot flag on the GL minimal
(task #10's other half).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Dion Moult
|
61726e00a4 |
wgpu backend: CPU frustum cull + per-mesh draw compaction
Stage 6 of the wgpu port. Replaces the one-draw-per-(mesh, instance) loop
with a CPU cull pass that survives one drawIndexed per non-empty mesh
with packed instanceCount.
Adds to WgpuModelGpuData:
- visible_buffer: u32[] storage SSBO, pre-sized to instance_count at
applyCachedModel so the bind group reference never invalidates.
Re-uploaded each frame via wgpuQueueWriteBuffer.
- mesh_draws: per-mesh schedule (first_instance, instance_count,
first_index, base_vertex, index_count). instance_count==0 means the
mesh contributed nothing this frame and the draw is elided entirely.
cullModelCpu per-frame:
- Extract 6 frustum planes from the same VP we write into the uniform.
WebGPU clip-space z is [0, 1], so near plane = matrix row 2 (not
row 3 + row 2 as in GL); rest of the derivation is standard.
- Per-instance AABB-vs-frustum test using the p-vertex shortcut
(cheapest correct early-out for AABBs).
- Bucket survivors by mesh_id; flatten into a contiguous u32 list;
upload via wgpuQueueWriteBuffer. Per-mesh slice is [first_instance,
first_instance + instance_count).
WGSL adds @group(1) @binding(3) var<storage, read> visible: array<u32>
and an extra indirection: instance_idx = visible[iid]; the rest of the
shader is unchanged. firstInstance on each drawIndexed offsets into
visible[], so each mesh reads its own slice.
Verified two ways:
1. basic.ifc (3 instances, all on-screen) renders pixel-identically
to pre-stage-6 — proves cull keeps everything it should.
2. basic.ifc + a synthetic instance placed at (100, 100, 100) is
culled cleanly: only the cube renders, the far quad is rejected
by the frustum test. Proves cull actually rejects out-of-frustum
geometry rather than passing everything through.
Contribution culling, HiZ, and LOD selection arrive in stages 7 and 8;
they all hook into the same cullModelCpu seam.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Dion Moult
|
bbf2bfde92 |
wgpu backend: vertex-pulling main render pass
Stage 3 of the wgpu port. Replaces the clear-only render loop with the
full main shading pass:
- WGSL port of the GL main shader. Vertex-pulling: the vertex storage
buffer is read as array<u32> in the shader, with pos/normal/color
decoded manually per vertex. baseVertex (set per draw to mesh's
vertex offset) folds into @builtin(vertex_index) automatically;
firstInstance carries the instance slot for @builtin(instance_index).
No vertex-input layout — vertex pulling means no IA bindings.
- Render pipeline bound to depth-32-float (write-on, less compare),
back-face cull, CCW front face. Pre-multiplies a [-1,1]→[0,1] z-remap
matrix onto Qt's projection so WebGPU's clip-z convention is met.
- Two bind groups: group=0 per-frame (uniform with view-proj + key/fill
light + hemisphere ambient), group=1 per-model (three read-only
storage buffers: vertices, mesh quant, instances).
- Depth texture is created lazily and recreated on surface resize.
- Orbit camera state on WgpuViewportWindow with viewAll() that frames
the union of all loaded models' world AABBs after the first load.
Mouse navigation lands later.
- Draw loop: one drawIndexed per (mesh, instance) pair per model. This
is correct but CPU-heavy on dense scenes; stage 6 introduces the cull
+ compacted visible list that lets multiple instances of one mesh
collapse to a single call, and the eventual GPU-driven cull (post
sunset of the GL backend) goes further.
Verified on /tmp/quad_v13.ifcview (1 mesh, 1 instance) and on a real v13
sidecar baked from basic.ifc via the GL minimal viewer (3 meshes,
3 instances, 864 B verts). No wgpu validation errors fire across pipeline
creation, depth attachment, bind groups, or the draw loop on either.
Visual confirmation deferred until --screenshot lands (task #10) which
is being pulled forward next so we don't keep flying blind.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Dion Moult
|
9daa5fe195 |
wgpu backend: load .ifcview sidecars onto GPU buffers
Stage 2 of the wgpu port. WgpuViewportWindow gains a queueLoadSidecar API (called from the minimal driver before init) and an applyCachedModel that runs after init: reads via SidecarCache::readSidecar, allocates four wgpu buffers per model (vertex storage, index, mesh-quant storage, instance storage), uploads via wgpuQueueWriteBuffer, retains a CPU mirror of the MeshInfo/InstanceCpu arrays for the cull and picking paths that arrive in later stages. MeshGpu (the per-mesh quantization basis) is derived from MeshInfo on the fly; InstanceGpu (transform + ids) is derived from InstanceCpu and uses the cached float transform — composing from placement_transformation against federation-stage matrices lands when stage 5 wires those. SidecarCache.cpp is compiled into IfcViewerWgpu directly: it's pure C++ with no Qt/OCCT/IFC-parse deps, so dragging in the IfcViewer static lib for one source file would be wasteful. This duplication goes away once src/ifcviewer-core/ is extracted (task #12). Verified on a synthesised v13 sidecar (4 verts, 6 indices, 1 mesh, 1 instance) and a multi-sidecar load that assigns successive model_ids. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |