Each instance was four separate allocations: the instance_data record, the
attribute array object it pointed at, that array's index bytes, and its
slot storage. A 58 MB model made 10.4 million mallocs to load 918k
instances, and massif attributed 99 MB of its 450 MB peak to malloc
bookkeeping alone.
variant_array now allocates the size byte, the per-slot type indices and
the slots as one block, and instance_data holds the array in a
std::optional instead of behind a pointer (an empty optional keeps the
meaning the null pointer had: attribute storage constructed on the fly
from the RocksDB backend). No ownership or lifetime changes; the same
object owns the same data.
Parse, C++ file constructor, on top of the previous commits:
TXG 58 MB 0.96 -> 0.94 s steady 310 -> 262 MB peak 383 -> 335 MB
210_King 148 MB 2.74 -> 2.63 s steady 758 -> 630 MB peak 948 -> 820 MB
OKgate22 232 MB 3.70 -> 3.61 s steady 1165 -> 971 MB peak 1447 -> 1252 MB
mallocs while loading TXG: 10.39M -> 7.56M.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
byguid_ was a std::map<std::string, ...>: a red-black node plus a
heap-allocated 22-character string per rooted instance, and a lookup that
walks ~18 levels of string comparisons on a 200k-entry file.
guid_map keeps keys of up to 23 characters inline in an unordered_map node
(every valid GlobalId is 22), and routes anything longer to an ordered map
so invalid files still work. Same std::string-keyed interface as before.
Parse, C++ file constructor, on top of the previous commits:
TXG 58 MB 1.08 s -> 1.03 s 341 -> 335 MB
210_King 148 MB 2.80 s -> 2.72 s 836 -> 830 MB
OKgate22 232 MB 4.25 s -> 3.94 s 1271 -> 1253 MB
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
full_buffer_impl::size(), get(), get_u32() and get_u64() (and their mmap_impl
twins) were defined out of line, so every character the lexer read crossed a
call boundary with its own bounds check. Callgrind put the three at 5.6% of
parse self time; inlining them lets the compiler hoist the checks out of the
scanning loops, which is worth more than their own cost.
Also reserve the streamer's inverse vector from the file size (about one
record per 32 bytes of SPF on real models) and shrink it once the bulk load
is sorted, so the doubling copies and the capacity slack go away.
Parse time, C++ file constructor, 12-core Linux box:
TXG 58 MB 1.35 s -> 1.15 s
210_King 148 MB 3.70 s -> 3.09 s
OKgate22 232 MB 6.18 s -> 5.30 s
Python ifcopenshell.open(): 1.40 -> 1.22, 3.70 -> 3.24, 6.27 -> 5.52 s.
Memory unchanged. The removed exported symbols mean the Python wrapper must
be rebuilt against this library.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
* Rewrite to_string() to use switch{} and handle Token_NONE and identifier without as_string()
* Add regression tests for to_string() on tokens without a string form
Cover both halves of the recursion that made a whitespace-only file
segfault: token::to_string() on the EOF marker and on an instance name,
and a parse of input that lexes to zero tokens, which is how the header
parser reaches token::as_string() on the EOF marker.
Generated with the assistance of an AI coding tool.
---------
Co-authored-by: Bruno Postle <bruno@postle.net>
process_deletion_inverse() called inverse_index::remove_source(), which
walked every record in the file's inverse index to find the ones whose
source is the deleted instance: O(R) per deletion, the dominant cost of
file.remove() on large files now that the lookup side no longer re-sorts.
The records a deleted instance contributed are exactly the entity
references in its own attributes, so walk those with the same visitor
build_inverses_() uses for registration and remove each record with a
targeted binary search instead. remove_source() has no callers left and
is deleted.
Also use the ordered view of batch_deletion_ids_ (a boost multi_index
that already had one) for the is-this-referencer-also-being-deleted
check in process_deletion_(), which was a linear std::find over the
sequenced view: O(b) per referencing instance made batch deletion of b
instances quadratic.
file.remove on 300 IfcPropertySet of a 155 MB IFC4 model (201k IfcRoot)
drops from 3.15 ms to 0.17 ms per call, batched removal of 2000 from
3.34 ms to 0.17 ms per call, root.remove_product on 100 walls from
332 ms to 131 ms per call.
Claude-Session: https://claude.ai/code/session_01HNrXDmR88wKPCYwGE21SyH
(cherry picked from commit 938442303f)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Modifications to build C++ with Visual Studio 2026 and the v145 toolset.
* Fixes linker settings for rocksdb for Debug and Release builds
* module is a C++ 20 keyword. Explicitly stating namespace allows cpp20 projects to build against the library
* Fixes crash when initializing an object with the initialize function when some of the attributes are empty, {}, or omitted, std::nullopt
* cleanup for vs2026 v145 toolset per @aothms review
* Fixes bug, IfcCurveSegment.setStartLength was setSegmentLength in alignment_helper.cpp
* Bumps boost to 1.92
Rename header-scope aliases, enums, and helper types while retaining descriptive names where dropping the suffix would create a collision.
Generated with the assistance of an AI coding tool.
Create file-owned headers after storage is selected but before streaming starts. Let owner-backed streamers use that header directly, and keep owned_header_ exclusively for ownerless streamers.
Generated with the assistance of an AI coding tool.
Both surfaced on the first Windows/WASM CI run of this branch:
- XmlSerializer.cpp: the IfcPropertySetDefinitionSet block used a C-style
cast to convert the set to std::vector<IfcPropertySetDefinition>. GCC
invokes the non-explicit conversion operator; MSVC rejects the cast to a
template type (C2440/C3536/C2661). Use copy-initialisation instead, which
invokes the same implicit conversion portably. (This block was dead until
the SCHEMAS_->SCHEMA_HAS_ typo fix enabled it, so it had never hit MSVC.)
- parse.cpp: the floating-point parse path falls back to strtod_l because
libc++ =deletes the float from_chars overload. That fallback was guarded
for __APPLE__ only; Emscripten uses the same libc++, so WASM hit the
deleted from_chars. Extend the guard to __EMSCRIPTEN__ (its musl provides
strtod_l/newlocale, treating all locales as C).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both fixes were made on v0.8.0's IfcParse.cpp AFTER the datamodel branch
had already renamed it to parse.cpp. When main was later merged into the
branch, the modify/delete was resolved toward the deletion (merge
8c9c3cde2), so the changes never reached the live parse.cpp. They are
invisible to `git log v0.8.0...HEAD` because the originating commits sit
in the merged-in shared ancestry; only a content sweep of the renamed
files surfaces them.
- format_double now uses the shortest decimal representation that
round-trips exactly (Mac-safe manual implementation, no std::to_chars),
instead of setprecision(digits10) which padded clean REALs with noise
digits (0.0174532925199433 -> 0.017453292519943299) and rewrote every
REAL on re-save. Recovers ee2b357d7 + fa597536e + 821cf7b67, #7696.
- The [SYN004] non-entity-type parse-error branch now resets current_id
to 0 before advancing, matching its sibling error branches, so a
malformed non-entity instance no longer erroneously terminates parsing.
Recovers 7c9df9f98.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two related bugs in read_from_stream's reference-resolution
loop, both reachable from malformed input:
- has_attribute_value<IfcBaseClass*> only checks the stored
slot's type, not that it's non-null (e.g. an explicit $
value), so the following get_attribute_value() call could
return null and inst->declaration() crashed on it.
- byid_[ref] default-inserts (and returns) a null pointer
when the owning instance id isn't present, which was then
dereferenced unconditionally via ->data().
Added regression tests using the two minimized crash inputs
that found these.
Generated with the assistance of an AI coding tool.
(cherry picked from commit 56121ca061)
Follow-up to the scalar-only fix in #8754, per aothms's direct request on
that PR ("Please do make all int types consistent") and his own original
2023 design intent on issue #3058 ("make all integers (incl. schema
namespaces) an int64_t"). Widens the remaining inconsistent spots now that
compatibility isn't a constraint on this v0.9 branch:
- Integer aggregates (IfcTriangulatedFaceSet.CoordIndex and similar
List<int> attributes), including the SWIG to_vec_int/to_vec_vec_int
helpers, which previously silently truncated via static_cast<int> on the
Python-set path - the same bug class as the original scalar issue.
- The schema code generator (express/mapping.py's integer type mapping),
and all 12 generated schema header/source pairs regenerated to match, so
every schema-typed getter/setter (e.g. IfcOwnerHistory::CreationDate) is
int64_t end to end, not just the dynamic attribute-value path.
Instance/reference identifiers (STEP #123 ids) are deliberately left at
32-bit: they're a file-local index into internal maps, not an EXPRESS
domain value an application chooses, and no realistic STEP file has
billions of entities. The lexer's Token_IDENTIFIER parsing still funnels
through a 32-bit int for this reason - flagged as a known, low-risk gap
rather than fixed, since fixing it would mean touching indexing/hashing
code for no realistic benefit.
Verified: original PR's round-trip tests extended with aggregate cases
(IfcTriangulatedFaceSet.CoordIndex, InnerCoordIndices) at 64-bit boundary
values, in memory and through STEP text, IFC2X3 and IFC4. A standalone C++
program exercising the generated schema API directly (Ifc4::IfcOwnerHistory
::setCreationDate/CreationDate, IfcTriangulatedFaceSet::setCoordIndex/
CoordIndex) confirms int64_t end to end, bypassing SWIG. Full build
(BUILD_IFCGEOM, WITH_OPENCASCADE, BUILD_IFCPYTHON, IFC2X3+IFC4) clean.
test/util/test_attribute.py and test_file.py pass unchanged.
This contribution was produced with the assistance of an AI coding tool.
Setting an IfcInteger/IfcTimeStamp typed attribute (e.g. IfcOwnerHistory.CreationDate)
outside the signed 32-bit range corrupted the value instead of raising, since the
Python wrapper's set_attribute_value_py() truncated it with a plain static_cast<int>
before handing it to the C++ storage. Unix timestamps before 1901-12-13 or after
2038-01-19 silently wrapped around (e.g. 3000000000 became -1294967296) rather than
being rejected or stored correctly. Fixes#3058, equivalent to PR #8683 but ported to
this branch's rewritten ifcparse (snake_case files, variant_array/instance_data
storage, SWIG PyObject-based attribute setter) instead of the old IfcEntityInstanceData
sources, which no longer exist here.
The scalar slot of the attribute variant (Argument_INT) becomes int64_t. Integer
aggregates (Argument_AGGREGATE_OF_INT, e.g. CoordIndex) and instance/reference
identifiers stay 32-bit, since neither is the value that overflows here; this narrow
scope is kept on its own technical merits (aggregates and identifiers were never the
source of the bug, and widening them would be a much larger, riskier change for no
benefit) even though aothms said compatibility isn't a concern on this v0.9-track
branch. express::Base::set_attribute_value promotes the schema-generated int to
int64_t at a single choke point, so the generated setters keep compiling unchanged.
The STEP lexer, writer, and SWIG wrapper (set_attribute_value_py, pythonize) are all
widened together, since widening only the Python-facing setter would have silently
wrapped the value on file write instead of raising.
Verified in a build (IFC2X3 and IFC4, BUILD_IFCGEOM off, no kernels): pre-1901,
post-2038, both 32-bit boundaries, and a 9e12 value all round trip exactly both in
memory and through STEP text serialization (write then reopen). A value outside the
64-bit range now raises a clean exception instead of corrupting data. Ordinary
in-range integers and integer aggregates (e.g. IfcTriangulatedFaceSet.CoordIndex) are
unaffected. The existing util/test_attribute.py and test_file.py suites pass
unchanged; test_entity_instance.py has 5 pre-existing failures unrelated to this
change (confirmed identical on an unfixed build of this branch, caused by a missing
get_info_2 binding and _patch_swig_comparisons never being implemented here).
Generated with the assistance of an AI coding tool.
parse_num_ used std::from_chars for both integers and doubles, but the
floating-point from_chars overload is =deleted in Apple clang's libc++, so
the macOS build failed to compile (parse.cpp:136, instantiated for double).
Split parse_num_ with `if constexpr`: integers keep std::from_chars
everywhere; on macOS, doubles parse via strtod_l with a cached "C" locale
(locale-independent, restoring the pre-charconv Apple path). libstdc++ and
the MSVC STL have working float from_chars and are left unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dion Moult