mirror of
https://github.com/IfcOpenShell/IfcOpenShell.git
synced 2026-08-30 08:33:10 +00:00
6f24133d35c79d6125232a52af1eb7f7b933cb74
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dion Moult
|
24616ed655 |
ifcviewer: stop the live budget from over-shrinking and from going stale
A 66-model session showed the pool reach a 1938 MB ceiling, the next poll lower the budget to 1756, and the shrink drop 402 MB (73+73+256) for a 182 MB excess, which the pool then spent seconds re-growing. Two causes. The release granularity is whole sub-buffers but the shrink ran "until capacity ≤ target", so the last 36 MB of excess cost a 256 MB sub-buffer. shrinkToCapacity now never undershoots — it releases only while doing so keeps capacity ≥ target, leaving a sub-buffer's worth of excess for the margin to absorb — and the pressure path uses a separate releaseAtLeast(bytes), whose contract is the opposite: free at least what the failed allocation needs, whatever the granularity. Resident geometry is also only evicted once the pool is over budget by half the margin (GpuBudget::shrinkTarget), so report jitter does not trigger a shrink-and-reload. The ceiling was a second old when the pool grew into it, and the upload staging that rides on growth had pushed device free memory to ~74 MB — below the driver's observed refusal point — before the next scheduled poll. pollDeviceMemory now re-derives the budget immediately after any sub-buffer is added, so the next growth decision sees the device as it is. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Dion Moult
|
ab99024307 |
ifcviewer: budget the geometry cache and make required allocations fallible
Loading enough models drove the chunk pool to the driver's refusal point, after which the first click aborted: the pick attachments are allocated lazily, wgpu-native reported their OOM as a validation error nobody observed, and the invalid views reached wgpuQueueSubmit, which panics across the FFI boundary. Two policy defects compounding: the cache was allowed to take the last byte, and nothing but the pool's own growth was treated as fallible. GPU memory is now two tiers. Required allocations (per-pixel attachments, a model's metadata buffers, readback staging) are eager, deterministic and fallible; the chunk pool is an elastic cache that grows only to a budget and yields whenever a required allocation fails. - GpuBudget (pure, unit-tested): desktop derives the budget from the driver's free-memory report minus a reserve for the attachments at 4K; web keeps the wasm-heap cap; either lowers it on pressure. The budget's source differs per platform, the mechanism does not. - GpuAllocScope: the OOM/Validation error-scope dance in one place, synchronous on wgpu-native, provisional on Dawn-web. BufferPool's inline copy now uses it. - BufferPool::shrinkToCapacity releases whole sub-buffers newest-first after the owner empties them; growth clamps to the budget instead of overshooting. - ViewportCore::allocateRequired runs any required creation under a scope and, on failure, lowers the budget, evicts and releases cache sub-buffers, waits for the device to reclaim them, and retries until it fits or the cache is at its floor. Pick attachments are created with the other attachments in configureSurface; render() skips a frame rather than submit invalid views; a model whose buffers cannot fit is not loaded instead of aborting. Verified on a 4 GB GeForce: the pool clamps itself at the derived budget (256+256+67 MB for a 579 MB budget) and, in a standalone check against the real device, a pool grown to the driver's refusal point observes a failed required allocation, releases 320 MB and succeeds on retry. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Dion Moult
|
ede689a8ff |
ifcviewer-web: fix two streaming stalls found in battle testing
driveStreamingLoads could deadlock: a chunk waiting on asynchronous pool growth parks in a frame-counted backoff cooldown, but once the render loop quiesced after the settle burst the frame index froze, so the cooldown never expired and streaming stalled part-loaded until the user moved the camera. Keep the loop alive while growth may still land (growth_pending() || can_grow()), exposed via a new BufferPool accessor. loadSidecarMetadataWeb put the model in the scene before reading the element-metadata block header, leaving a window where the locator was still zero. A getObjects() landing in that window could not distinguish "locator not read yet" from "sidecar has no element block" and latched the model as permanently empty. Read the 16-byte header first, then apply; carry the locator through applyCachedModel so it is set before any web element-metadata fetch can run. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Dion Moult
|
0b8c787ac0 |
ifcviewer: v16 zstd-compressed sidecars (~10x smaller over the wire)
The .ifcview data is hugely redundant (repeated double instance matrices, patterned indices) — measured 12x zstd whole-file. Server Content-Encoding can't be used (it breaks HTTP Range), so compress PER-CHUNK into the format. Format (v16): geometry becomes per-chunk zstd(vertices)+zstd(indices) frames — each independently Range-fetchable, so streaming is intact — and the critical + deferred metadata blocks are single zstd frames. SidecarChunk carries the compressed blob offsets/sizes; applyStreamedChunk (render/upload) is UNCHANGED — decompression slots into the fetch. Full readSidecar (test/tooling) reconstructs by decompress+scatter. zstd: desktop links libzstd (also compresses at bake); the web build (Emscripten has no zstd port) FetchContent's the pinned zstd source and compiles its decompress-only subset for wasm — no vendored blob, same version as desktop. New SidecarCompress wraps it (compress guarded off under Emscripten). Both stream paths — desktop StreamingThread worker + sync fallback (readChunkGeometryCompressed) and web beginWebChunkLoad — decompress; readSidecarMetadataOnly / the web bootstrap / loadDeferredMetadataWeb decompress the metadata blocks. streamingByteProgress reports COMPRESSED bytes. MEASURED: a 752 MB v15 federation → 75 MB v16 (10x; per-file 6.7-15.3x); PP-PLP 118→15 MB, loads 13/13 chunks on web, 0 errors. Three fixes found while testing big federations on a real server: - Web-streamed race: streaming_from_web was set in the deferred-header callback (a round-trip after the model+chunks exist), so driveStreamingLoads could take the sync fopen path meanwhile → "failed to read/decompress chunk 0". Now set immediately after applyCachedModel. - OOM abort on 18 models: the pool grew unbounded until an alloc failed, but on web that's an uncatchable bad_alloc abort. Cap total pool capacity (setMaxTotalCapacity, 3 GB) so it stops before the heap ceiling, and raise MAXIMUM_MEMORY 2→4 GB (wasm32 max) for headroom. - Web never evicted (grow-or-block only). At the hard budget, fall through to the LRU/priority evictor so a big federation stays navigable (highest-contribution chunks win) instead of freezing with holes. 113/113 desktop + 6/6 web smoke pass. No back-compat: regenerate sidecars (desktop bakes v16; scratch conv tool migrates v15→v16). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Dion Moult
|
9db42df81c |
ifcviewer-web: stop large-model network streaming thrash (grow before fetch)
Battle-testing real sidecars over HTTP Range exposed severe thrash: a
531 MB model re-fetched 2.25 GB (4×) and never converged — viewAll puts
the whole model in frustum, so every chunk wants to be resident, and the
web async path made it worse two ways:
- A web load only consumes pool space when it COMPLETES (async), so the
per-frame issuance over-committed the pool; completions then failed
applyStreamedChunk on a full pool, the chunk re-candidated with no
cooldown, and re-fetched every frame.
- Pool growth is itself async on web (provisional sub-buffers validated
off the JS event loop), so even fetched chunks failed to alloc until
the pool caught up, and re-fetched.
Fix: gate web chunk issuance on VALIDATED free space + in-flight
reservation, and grow the pool BEFORE fetching:
- streaming_web_inflight_bytes_ reserves each in-flight load's footprint
so we never have more bytes in flight than the pool can place.
- When a visible chunk doesn't fit validated free, don't fetch — call
pool_.requestGrowth() (BufferPool: drives the async provisional grow
without allocating) and short-back-off; the chunk is fetched once,
after space exists. When the pool is saturated (model > GPU memory),
long-cooldown so a never-fitting chunk isn't re-fetched. Gating before
the evictor also kills phase-2 visible↔visible swap thrash.
- On async load failure, cool down (short if the pool can still grow,
long if saturated) instead of re-candidating next frame.
Result (manual battle tool, host.mjs + real files): 531 MB now loads
23/23 chunks, 322 MB loads 14/14 — resident climbs monotonically with
ZERO thrash warnings and a stable resident set, vs the old re-fetch loop.
The whole model resides on the GPU and stays. (Remaining ~3× ramp
over-fetch — per-chunk re-loads during the async-growth ramp + read
amplification from chunk byte-locality — is a separate efficiency
follow-up, not thrash.) 6/6 web smoke + 107/107 unit pass; desktop
unaffected (the gate is web-only; requestGrowth is a no-op wrapper there).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Dion Moult
|
89beee6514 |
ifcviewer: detect web pool-grow OOM via provisional sub-buffers
On web we skip the desktop error-scope spin-wait (it blocks the JS event
loop and hangs the page). The old web addSubBuffer then judged success by
`buf != nullptr` — but Dawn-web returns a NON-NULL error buffer on OOM,
so the pool committed an invalid sub-buffer, alloc handed out slices in
it, and every chunk_bind_group built against it failed ("BindGroup is
invalid" spam + a wgpuQueueSubmit panic). Loading a model larger than the
browser's WebGPU budget triggered exactly this.
Add the grown sub-buffer as *provisional* (alloc and the capacity/free
tallies skip it) and validate it through a non-blocking AllowSpontaneous
PopErrorScope. resolveProvisionalGrowth() clears the flag when it's good,
or drops the sub-buffer and latches growth_disabled_ on a real OOM — at
which point the streaming evictor bounds the working set to what fits
instead of cascading. Only one provisional grow is in flight at a time
(growth_pending_). Desktop keeps its synchronous halve-retry path
unchanged. All 100 unit tests still pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
||
|
Dion Moult
|
1fe4570860 |
ifcviewer: extract ChunkPlanner + InstanceCompose; add Tier-1 test trio
The chunk planner (Morton sort + greedy pack) and instance composition (federation × placement matrix chain + world-AABB derive) were inline helpers in ViewportWindow.cpp. Pulled both out as free-function modules so the math + lookup logic can be exercised without a Qt window or a wgpu device. ViewportWindow now delegates; InstanceLookup is a using- alias to InstanceCompose::InstanceLookup. Also added an addSubBufferForTesting / clearSubPoolsForTesting seam to BufferPool so the sub-allocator invariants can be pinned with fake WGPUBuffer handles. The fakes are never dereferenced; the guard drops the sub-pools before destructor would call wgpuBufferRelease. Three new test binaries under src/ifcviewer/tests/, 33 cases / 173 assertions: BufferPool first-fit + alignment + coalescing + multi- sub-pool isolation; ChunkPlanner Morton split / interleave / stable sort / greedy-pack monotonicity and single-mesh-oversize; InstanceCompose identity / translation / order-of-multiplication / large-placement cancellation against federation false origin / column-major writeback / findInstance lookup paths. |
||
|
Dion Moult
|
8ab5c31e75 |
refactor: merge ifcviewer-wgpu into ifcviewer, drop Wgpu prefix
The GL backend is gone (task #53). The wgpu/non-wgpu folder split and the Wgpu* class prefix were both disambiguation artefacts from the overlap period — now pure dead weight. ## Folder + library merge * `src/ifcviewer-wgpu/` → folded into `src/ifcviewer/` (git mv tracks every file as a rename so blame/log history survives). * `src/ifcviewer-wgpu-minimal/` → `src/ifcviewer-minimal/` (the exe was already named `IfcViewerMinimal`; this just brings the folder + CMake target name into line). * `src/ifcviewer-wgpu/tests/test_wgpu_{selection,visibility}.cpp` → `src/ifcviewer/tests/test_{selection,visibility}.cpp`, folded into the existing `add_ifcviewer_unit_test(...)` helper. * The `IfcViewerWgpu` static library is dissolved — its sources become part of the unified `IfcViewer` static library, which now bundles scene/loader + renderer in one target. The pre-merge circular dependency (IfcViewer linking IfcViewerWgpu just to get the ViewportWindow.h include path that SceneLoader.h needs) goes away. * The wgpu-native FetchContent block, the Cocoa/QuartzCore link on Apple, the OBJCXX-enabled `.mm` source, and the wgpu-native runtime install all move into `src/ifcviewer/CMakeLists.txt` unchanged. ## Type renames (Wgpu prefix dropped from every Wgpu* identifier) WgpuAreaMeasurement → AreaMeasurement WgpuBufferPool → BufferPool WgpuLengthMeasurement → LengthMeasurement WgpuMetalSurface → MetalSurface WgpuModelGpuData → ModelGpuData WgpuOverlayFrame → OverlayFrame WgpuOverlayRenderer → OverlayRenderer WgpuSectionPlane → SectionPlane WgpuSelectionState → SelectionState WgpuStreamingLoader → StreamingLoader WgpuStreamingThread → StreamingThread WgpuViewportWindow → ViewportWindow WgpuVisibilityState → VisibilityState CMake target IfcViewerWgpuMinimal → IfcViewerMinimal (exe name was already this since wgpu shipped as default). Deliberately kept: `onWgpuLog` (wgpu-native log callback — names a binding to an external API, not one of *our* types), and the WGPU* enum/struct prefixes from wgpu-native's own headers. `WgpuMemProbe` lives in the separate `src/wgpu-mem-probe/` standalone diagnostic project and isn't touched. ## Include-path updates Every `#include "../ifcviewer-wgpu/Wgpu<X>.h"` → `"../ifcviewer/<X>.h"`, every in-directory `#include "Wgpu<X>.h"` → `"<X>.h"`. Includes from sibling subdirectories (modules/, etc.) are updated to point at `../../../ifcviewer/` instead of `../../../ifcviewer-wgpu/`. ## cmake/CMakeLists.txt simplification The redundant `add_subdirectory(ifcviewer-wgpu)` blocks (one inside the BUILD_BONSAIVIEWER fan-in, one in the BONSAIVIEWER-less standalone block) collapse into a single unconditional `add_subdirectory(../src/ifcviewer ifcviewer)`. The standalone block keeps only `wgpu-mem-probe` (the diagnostic tool, unrelated to the viewer lib). ## Verification * Full build green: `IfcViewer` static lib, `IfcViewerMinimal` exe, `BonsaiViewer` exe, all four pre-existing ifcviewer unit tests, and the two new-location tests (`test_selection`, `test_visibility`). * No stray `Wgpu<X>` identifier remains across `src/ifcviewer/`, `src/bonsaiviewer/`, `src/ifcviewer-minimal/` (verified by grep). * Renames tracked by git as `R` entries — `git log --follow` on ViewportWindow.cpp etc. continues to show history through the move. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> |