mirror of
https://github.com/IfcOpenShell/IfcOpenShell.git
synced 2026-08-31 08:56:34 +00:00
wgpu streaming: spatial chunk planning + coalesced multi-range reads
Chunks are now grouped by world-space centroid instead of mesh-id
range, so each chunk's AABB tightly bounds its geometry instead of
spanning the whole model. Distance-based eviction can finally
distinguish the near corner of a skyscraper from the far corner.
Algorithm:
1. Compute each mesh's centroid = mean of its instances' world AABB
centres.
2. Sort mesh indices lexicographically by (z, y, x) centroid. Stable
sort keeps mesh-id order as tiebreaker for instanced repeats.
3. Greedy-pack sorted meshes into chunks ≤ WGPU_CHUNK_VERTEX_BYTES_LIMIT.
4. Each Chunk stores its mesh_ids list; the per-mesh layout (chunk_local
base_vertex / ebo_first_u32) is computed by walking the list at plan
time.
Loader: chunk vertex/index bytes are no longer file-contiguous, so
streaming uses new multi-range read paths
(readSidecarVertexRanges / readSidecarIndexRanges). Each range list
is sorted by file offset and adjacent ranges coalesced with a 64 KB
gap tolerance — on the close-camera benchmark this brings the
per-chunk seek count back down to ~mesh-id-grouping levels, so the
spatial sort costs ~nothing on I/O while delivering tighter AABBs.
Non-streaming applyCachedModel mirrors the spatial plan but gathers
from in-memory data.vertices / data.indices via per-mesh
queueWriteBuffer calls at chunk-local offsets.
Chunk struct drops vertex_byte_offset and index_first_u32 (no longer
meaningful — each chunk is N scattered ranges). vertex_byte_size and
index_count stay as aggregates for pool sizing + eviction math.
Tuning: kept WGPU_CHUNK_VERTEX_BYTES_LIMIT at 128 MB. Tried 8 MB and
32 MB; both gave tighter AABBs but the scatter-gather I/O cost blew
up because the per-frame load count grows linearly as chunks shrink
(orbit shifts the working set faster across finer chunks). 128 MB +
coalescing is the empirical sweet spot pre-v14. Once sidecar v14
re-orders bytes on disk to match spatial chunks, we can drop the
limit to ~8 MB for sharp eviction without re-paying the seek cost.
Benchmarks (big federation, --streaming):
close camera: avg 36 fps median 53 (was 35/49) — parity
default camera: avg 33 fps median 47 (was 40/49) — small regression
likely from increased coalesce overhead on more-
scattered orbit traversals; will resolve with v14.
Pixel-identical to non-streaming on basic.ifc on both paths.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
@@ -46,10 +46,18 @@
|
||||
// 1–3 chunks), which is invisible compared to per-frame GPU work.
|
||||
//
|
||||
// At INSTANCED_VERTEX_STRIDE_BYTES = 12 B/vertex this caps a chunk at
|
||||
// 11.18 M vertices. A mesh whose vertex range is bigger than this can't fit
|
||||
// in any chunk and would need splitting — typical IFC meshes are nowhere
|
||||
// near (hundreds of verts), and applyCachedModel asserts loudly if it ever
|
||||
// happens.
|
||||
// ~11 M vertices. Tuned for the pre-v14 scatter-gather streaming model:
|
||||
// spatial chunk planning means each chunk's bytes are NOT contiguous in
|
||||
// the sidecar, so per-load I/O cost scales with mesh count per chunk
|
||||
// (one fseek+fread per file gap). Bigger chunks = more meshes per
|
||||
// chunk = more seeks per load, BUT also fewer chunks total = fewer
|
||||
// loads per frame as orbit shifts the visible set. The latter
|
||||
// dominates: 128 MB chunks → ~16 chunks per pool → 1-2 loads per
|
||||
// frame → ~20-30 ms stream cost. Smaller chunks (32 / 8 MB) bring
|
||||
// finer eviction granularity but explode the loads-per-frame count.
|
||||
// Sidecar v14 (on-disk spatial reorder) is the proper fix — once
|
||||
// chunks ARE file-contiguous, the per-mesh seek cost vanishes and we
|
||||
// can drop the chunk size back to ~8 MB for sharp eviction.
|
||||
static constexpr uint64_t WGPU_CHUNK_VERTEX_BYTES_LIMIT = 128ull * 1024 * 1024;
|
||||
|
||||
struct WgpuModelGpuData {
|
||||
@@ -116,22 +124,21 @@ struct WgpuModelGpuData {
|
||||
// Render and pick skip chunks where !is_resident.
|
||||
bool is_resident = true;
|
||||
|
||||
// Where the chunk's vertex bytes live in the sidecar (offsets
|
||||
// relative to vertex_section_offset on the model). Populated by
|
||||
// the streaming loader; zeroed for the non-streaming path.
|
||||
uint64_t vertex_byte_offset = 0; // 0 == start of vertex section
|
||||
// Aggregate vertex / index sizes across all meshes in this chunk
|
||||
// (sum of mesh.vertex_count * stride / mesh.index_count for each
|
||||
// mesh in mesh_ids). Used to size the pool allocation and to
|
||||
// compute the cull's per-chunk free-room check. Per-mesh layout
|
||||
// is recovered by walking mesh_ids and the model's MeshInfo[].
|
||||
uint64_t vertex_byte_size = 0;
|
||||
|
||||
// Same for indices. index_first_u32 is in u32 units relative to
|
||||
// the start of the index section (sidecar stores raw u32 indices,
|
||||
// no byte-level offset is needed beyond multiplying by 4).
|
||||
uint64_t index_first_u32 = 0;
|
||||
uint64_t index_count = 0;
|
||||
|
||||
// World-space AABB covering every instance whose mesh lives in
|
||||
// this chunk. Used by cull to reject whole chunks against the
|
||||
// frustum before iterating instances — and by the streaming
|
||||
// loader to prioritise which non-resident chunks to fetch first.
|
||||
// this chunk. With spatial chunk planning this AABB is tight
|
||||
// (chunks group meshes by world centroid, not mesh-id), so the
|
||||
// distance-based evictor can meaningfully tell chunks apart.
|
||||
// Used by cull to reject whole chunks against the frustum before
|
||||
// iterating instances — and by the streaming loader to
|
||||
// prioritise which non-resident chunks to fetch first.
|
||||
float aabb_min[3] = { std::numeric_limits<float>::infinity(),
|
||||
std::numeric_limits<float>::infinity(),
|
||||
std::numeric_limits<float>::infinity() };
|
||||
@@ -139,6 +146,17 @@ struct WgpuModelGpuData {
|
||||
-std::numeric_limits<float>::infinity(),
|
||||
-std::numeric_limits<float>::infinity() };
|
||||
|
||||
// Mesh IDs assigned to this chunk, in chunk-local layout order.
|
||||
// Spatial chunk planning sorts meshes by world centroid first,
|
||||
// so this list is not in mesh-id order in general — each mesh's
|
||||
// bytes live at scattered offsets in the sidecar file. The
|
||||
// loader walks this list to scatter-gather the chunk's vertex
|
||||
// + index bytes; mesh_chunk_local_base_vertex /
|
||||
// mesh_chunk_local_ebo_first_u32 are computed in this same
|
||||
// order at planning time so the cull's VisibleDrawGpu entries
|
||||
// point at the correct chunk-local offsets.
|
||||
std::vector<uint32_t> mesh_ids;
|
||||
|
||||
// LRU marker for streaming eviction. Updated to the window's
|
||||
// streaming_frame_idx_ every frame the chunk is rendered (i.e.
|
||||
// total_visible_draws > 0). The evictor picks the smallest value
|
||||
|
||||
Reference in New Issue
Block a user