mirror of
https://github.com/IfcOpenShell/IfcOpenShell.git
synced 2026-08-18 19:30:25 +00:00
Dion Moult
e1be2f208c
Makes large-model streaming over a network actually good — fixing read
amplification, then first-paint latency — building on the byte-range work.
v14 layout + TOC (SidecarLayout, pure + unit-tested)
The loader chunks meshes by spatial Morton order, but the sidecar stored
geometry in mesh-id order, so a chunk's meshes were scattered through the
file: streaming one chunk meant either hundreds of tiny range requests or
reading (and discarding) everything between them — a 113 MB model fetched
~340 MB, a 531 MB model 2.25 GB (4.2x). Fix: at bake, reorder meshes into
the loader's chunk order and rebuild vertex/index(LOD0+LOD1)/instance
sections so each chunk is one CONTIGUOUS byte range, and bake a chunk TOC
({first_mesh, mesh_count}). The loader builds chunks straight from the TOC
rather than re-deriving the plan — the float Morton quantisation isn't
bit-identical across toolchains (x86 baker vs wasm loader), so a re-derived
plan scatters the chunks. Format bumped to v14 (regenerate sidecars). The
reorder buckets instances by per-instance mesh_id (the baker never sets
MeshInfo.first_instance — trusting it scrambled every transform → geometry
at the origin). Multiset-verified on a 28,900-instance model: every
instance's placement + geometry preserved. Result: 531 MB fetches 531 MB
(1.0x) in 72 requests (was 2036).
Progressive streaming (concurrency cap + small chunks)
Even at 1x, geometry appeared only after ~the whole model arrived: the
browser multiplexes every in-flight Range request over one HTTP/2 conn, so
unbounded concurrency (9 in flight) split the bandwidth and nothing finished
until the end (measured: first paint after 113 of 118 MB / 35 s @ 24 Mbps).
Cap concurrent chunk loads (kMaxWebInflightChunks=2): the priority-sorted
top chunks finish and paint first, then the next → first paint 9 s. Chunk
size dropped 16->4 MB (cheap now that each chunk is one read; matches Cesium
3D Tiles / xeokit / SVF2) for smoother progression. First-paint is now
metadata-bound (~10 MB tail) — the next lever.
111/111 unit (new test_sidecar_layout: geometry preserved, contiguous layout,
Morton-identity) + 6/6 web smoke pass; desktop bake (SceneLoader) reorders
before writeSidecar; embedded web sample regenerated to v14.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
97 lines
4.9 KiB
C++
97 lines
4.9 KiB
C++
/********************************************************************************
|
||
* *
|
||
* This file is part of IfcOpenShell. *
|
||
* *
|
||
* IfcOpenShell is free software: you can redistribute it and/or modify *
|
||
* it under the terms of the Lesser GNU General Public License as published by *
|
||
* the Free Software Foundation, either version 3.0 of the License, or *
|
||
* (at your option) any later version. *
|
||
* *
|
||
* IfcOpenShell is distributed in the hope that it will be useful, *
|
||
* but WITHOUT ANY WARRANTY; without even the implied warranty of *
|
||
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the *
|
||
* Lesser GNU General Public License for more details. *
|
||
* *
|
||
* You should have received a copy of the Lesser GNU General Public License *
|
||
* along with this program. If not, see <http://www.gnu.org/licenses/>. *
|
||
* *
|
||
********************************************************************************/
|
||
|
||
#ifndef CHUNKPLANNER_H
|
||
#define CHUNKPLANNER_H
|
||
|
||
// Chunk-planning helpers for the wgpu viewport. Replaces a previous
|
||
// lexicographic (z, y, x) sort with a 3D Morton (Z-order) sort over mesh
|
||
// centroids, then greedy-packs the resulting order into chunks bounded by
|
||
// a vertex-bytes ceiling. The two passes are split so each is unit-
|
||
// testable in isolation (no Qt / no wgpu).
|
||
|
||
#include <cstddef>
|
||
#include <cstdint>
|
||
#include <vector>
|
||
|
||
// Vertex-bytes ceiling for one streaming chunk. The greedy packer starts a new
|
||
// chunk before a mesh would push the running vertex bytes over this. Lives here
|
||
// (pure, no wgpu) so the bake-time layout pass and the GPU loader share one
|
||
// value.
|
||
//
|
||
// 4 MB (down from 16 MB): with v14 each chunk is one contiguous range read, so
|
||
// small chunks are cheap, and they paint progressively far sooner over a
|
||
// network — first-paint payload ≈ metadata + (concurrency cap × this). In line
|
||
// with what streaming viewers (Cesium 3D Tiles, xeokit, SVF2) use.
|
||
static constexpr std::uint64_t WGPU_CHUNK_VERTEX_BYTES_LIMIT = 4ull * 1024 * 1024;
|
||
|
||
namespace ChunkPlanner {
|
||
|
||
// Interleave the low 21 bits of v with two zero bits between each,
|
||
// returning bits at positions 0, 3, 6, ..., 60 — one axis of a
|
||
// standard 21-bit-per-axis 3D Morton code. ORing three of these
|
||
// shifted by 0, 1, 2 gives a 63-bit (x, y, z)-interleaved code; the
|
||
// resulting integer ordering puts spatially-close points close in
|
||
// the sorted sequence (the classic Z-order curve).
|
||
uint64_t mortonSplit21(uint32_t v);
|
||
|
||
uint64_t mortonCode3D(uint32_t x, uint32_t y, uint32_t z);
|
||
|
||
// Return a mesh-id permutation sorted by 3D Morton (Z-order) code over
|
||
// the meshes' centroids. Replaces a lexicographic (z, y, x) sort,
|
||
// which was effectively a 1D Z-slab traversal — chunks ended up
|
||
// spanning the whole XY extent of the model, ~50m × 50m × 0.5m for a
|
||
// typical building. Morton clusters spatially in all 3 axes, so each
|
||
// chunk's AABB becomes a tight 3D voxel — small enough that
|
||
// per-chunk frustum / contribution / HiZ rejection becomes meaningful
|
||
// (a 1km-wide AABB never gets occluded; a 10m voxel often does).
|
||
//
|
||
// Meshes with no instances get a Morton code of 0 and sink to the
|
||
// front; they contribute no geometry / AABBs so where they land in
|
||
// the chunk plan doesn't matter.
|
||
std::vector<uint32_t> sortMeshIdsByMorton(
|
||
std::size_t n_meshes,
|
||
const std::vector<float>& mesh_cx,
|
||
const std::vector<float>& mesh_cy,
|
||
const std::vector<float>& mesh_cz,
|
||
const std::vector<uint32_t>& mesh_inst_count);
|
||
|
||
// Greedy-pack a pre-sorted mesh-id sequence into chunks bounded by
|
||
// `chunk_vertex_bytes_limit`. Each mesh is placed in the current
|
||
// chunk; if adding it would push the running byte count over the
|
||
// limit (and the chunk is non-empty), a new chunk is started.
|
||
//
|
||
// A mesh whose own vertex bytes already exceed the limit lands alone
|
||
// in its own (over-sized) chunk — the planner never splits a mesh
|
||
// across chunks, because the mega-draw bookkeeping is per-mesh
|
||
// chunk-local-offset.
|
||
//
|
||
// `sorted_mesh_ids` is the order produced by sortMeshIdsByMorton.
|
||
// `mesh_vertex_count[mesh_id]` gives the vertex count per mesh.
|
||
// `vertex_stride_bytes` is the per-vertex byte size on the GPU.
|
||
std::vector<std::vector<uint32_t>> greedyPackChunks(
|
||
const std::vector<uint32_t>& sorted_mesh_ids,
|
||
const std::vector<uint32_t>& mesh_vertex_count,
|
||
uint64_t vertex_stride_bytes,
|
||
uint64_t chunk_vertex_bytes_limit);
|
||
|
||
} // namespace ChunkPlanner
|
||
|
||
#endif // CHUNKPLANNER_H
|