Backface culling with reflection-aware two-pass MDI

Enables GL_CULL_FACE by default (user-toggleable in Settings) so
closed solids skip shading their back halves.  The catch is that
IFC placements can contain reflections (mat4 with det<0 — mirrored
families, symmetric instances).  Naively culling would make every
mirrored instance vanish because the rasterizer sees its screen-space
winding as backwards.

Fix: detect reflections at upload time via determinant sign, bucket
visible instances into forward (det>=0) and reverse (det<0) per mesh
during culling, and issue two glMultiDrawElementsIndirect calls per
model with glFrontFace toggled CCW/CW between them.  The indirect
buffer is still one buffer — just split into a forward slice followed
by a reverse slice, with m.indirect_forward_count recording the split.

Vertex shader flips the normal when the transform has negative
determinant, keeping lighting correct on mirrored instances.  The
fragment shader keeps the gl_FrontFacing fallback as a safety net
when culling is disabled (e.g. for files with open shells).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Dion Moult
2026-04-12 22:07:19 +10:00
parent 0e2a62d3b7
commit 3110c98429
6 changed files with 172 additions and 41 deletions
+10 -3
View File
@@ -71,7 +71,12 @@ struct ModelGpuData {
uint32_t total_triangles = 0;
std::vector<MeshInfo> meshes;
std::vector<InstanceCpu> instances; // unsorted until finalize
std::vector<InstanceCpu> instances; // unsorted
// 1:1 with instances[] — true when the instance transform has
// det < 0 (a reflection). Reflected instances need their
// triangle winding treated as reversed so GL_CULL_FACE culls
// the correct side.
std::vector<uint8_t> instance_reflected;
uint32_t ssbo_instance_count = 0;
// Per-instance world AABB + BVH (built at finalize). The BVH is the
@@ -88,7 +93,8 @@ struct ModelGpuData {
// non-empty mesh. Re-uploaded each frame.
GLuint indirect_buffer = 0;
size_t indirect_capacity = 0; // bytes
uint32_t indirect_command_count = 0; // valid commands this frame
uint32_t indirect_command_count = 0; // total valid commands this frame
uint32_t indirect_forward_count = 0; // first N are CCW-winding draws
bool finalized = false;
bool hidden = false;
@@ -211,7 +217,8 @@ private:
// per-frame allocation. indirect_scratch_ is the matching array of
// DrawElementsIndirectCommand records — forward-declared as bytes so
// the header doesn't need the struct definition.
std::vector<std::vector<uint32_t>> visible_by_mesh_;
std::vector<std::vector<uint32_t>> visible_by_mesh_fwd_;
std::vector<std::vector<uint32_t>> visible_by_mesh_rev_;
std::vector<uint32_t> visible_flat_;
std::vector<DrawElementsIndirectCommand> indirect_scratch_;