The parser could not resolve a #name when it read it, because the
instance may be defined further down the file, so it left the slot empty
and appended (owner, attribute, name) to a side table that a second pass
walked. The table held one entry per reference for the whole read: 64 MB
on a 58 MB model, the high-water mark of opening.
Now the reference stays where the tokenizer put it: the attribute slot
holds the instance_reference, or the reference_or_simple_type aggregate
for a list (mixed with inline typed values or not), until every instance
has been read, and resolve_instance_references() walks each instance's
slots and swaps names for instances. Ordering is what the tokenizer
produced; nothing is re-derived. A missing name becomes null in a scalar
and is dropped from an aggregate, as before; the error keeps its offset.
The three transient alternatives are appended to the attribute pack and
to argument_type in lock step and are never visible once a file is
loaded. Simple type instances read inline (IfcPropertySetDefinitionSet)
have their own slots, so their references need no diversion.
The table remains for the header entities and for streaming consumers of
instance_streamer::references(), which leave resolve_references_in_place
off.
TXG 58 MB / 210_King 147 MB / OKgate22 231 MB, single thread: time
unchanged (1.05 / 2.69 / 4.99 s), memory after the parse 287 -> 274,
698 -> 654, 1086 -> 1036 MB, peak 400 -> 365, 965 -> 871, 1471 -> 1347 MB.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
Each instance was four separate allocations: the instance_data record, the
attribute array object it pointed at, that array's index bytes, and its
slot storage. A 58 MB model made 10.4 million mallocs to load 918k
instances, and massif attributed 99 MB of its 450 MB peak to malloc
bookkeeping alone.
variant_array now allocates the size byte, the per-slot type indices and
the slots as one block, and instance_data holds the array in a
std::optional instead of behind a pointer (an empty optional keeps the
meaning the null pointer had: attribute storage constructed on the fly
from the RocksDB backend). No ownership or lifetime changes; the same
object owns the same data.
Parse, C++ file constructor, on top of the previous commits:
TXG 58 MB 0.96 -> 0.94 s steady 310 -> 262 MB peak 383 -> 335 MB
210_King 148 MB 2.74 -> 2.63 s steady 758 -> 630 MB peak 948 -> 820 MB
OKgate22 232 MB 3.70 -> 3.61 s steady 1165 -> 971 MB peak 1447 -> 1252 MB
mallocs while loading TXG: 10.39M -> 7.56M.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
Rename header-scope aliases, enums, and helper types while retaining descriptive names where dropping the suffix would create a collision.
Generated with the assistance of an AI coding tool.
Follow-up to the scalar-only fix in #8754, per aothms's direct request on
that PR ("Please do make all int types consistent") and his own original
2023 design intent on issue #3058 ("make all integers (incl. schema
namespaces) an int64_t"). Widens the remaining inconsistent spots now that
compatibility isn't a constraint on this v0.9 branch:
- Integer aggregates (IfcTriangulatedFaceSet.CoordIndex and similar
List<int> attributes), including the SWIG to_vec_int/to_vec_vec_int
helpers, which previously silently truncated via static_cast<int> on the
Python-set path - the same bug class as the original scalar issue.
- The schema code generator (express/mapping.py's integer type mapping),
and all 12 generated schema header/source pairs regenerated to match, so
every schema-typed getter/setter (e.g. IfcOwnerHistory::CreationDate) is
int64_t end to end, not just the dynamic attribute-value path.
Instance/reference identifiers (STEP #123 ids) are deliberately left at
32-bit: they're a file-local index into internal maps, not an EXPRESS
domain value an application chooses, and no realistic STEP file has
billions of entities. The lexer's Token_IDENTIFIER parsing still funnels
through a 32-bit int for this reason - flagged as a known, low-risk gap
rather than fixed, since fixing it would mean touching indexing/hashing
code for no realistic benefit.
Verified: original PR's round-trip tests extended with aggregate cases
(IfcTriangulatedFaceSet.CoordIndex, InnerCoordIndices) at 64-bit boundary
values, in memory and through STEP text, IFC2X3 and IFC4. A standalone C++
program exercising the generated schema API directly (Ifc4::IfcOwnerHistory
::setCreationDate/CreationDate, IfcTriangulatedFaceSet::setCoordIndex/
CoordIndex) confirms int64_t end to end, bypassing SWIG. Full build
(BUILD_IFCGEOM, WITH_OPENCASCADE, BUILD_IFCPYTHON, IFC2X3+IFC4) clean.
test/util/test_attribute.py and test_file.py pass unchanged.
This contribution was produced with the assistance of an AI coding tool.
Setting an IfcInteger/IfcTimeStamp typed attribute (e.g. IfcOwnerHistory.CreationDate)
outside the signed 32-bit range corrupted the value instead of raising, since the
Python wrapper's set_attribute_value_py() truncated it with a plain static_cast<int>
before handing it to the C++ storage. Unix timestamps before 1901-12-13 or after
2038-01-19 silently wrapped around (e.g. 3000000000 became -1294967296) rather than
being rejected or stored correctly. Fixes#3058, equivalent to PR #8683 but ported to
this branch's rewritten ifcparse (snake_case files, variant_array/instance_data
storage, SWIG PyObject-based attribute setter) instead of the old IfcEntityInstanceData
sources, which no longer exist here.
The scalar slot of the attribute variant (Argument_INT) becomes int64_t. Integer
aggregates (Argument_AGGREGATE_OF_INT, e.g. CoordIndex) and instance/reference
identifiers stay 32-bit, since neither is the value that overflows here; this narrow
scope is kept on its own technical merits (aggregates and identifiers were never the
source of the bug, and widening them would be a much larger, riskier change for no
benefit) even though aothms said compatibility isn't a concern on this v0.9-track
branch. express::Base::set_attribute_value promotes the schema-generated int to
int64_t at a single choke point, so the generated setters keep compiling unchanged.
The STEP lexer, writer, and SWIG wrapper (set_attribute_value_py, pythonize) are all
widened together, since widening only the Python-facing setter would have silently
wrapped the value on file write instead of raising.
Verified in a build (IFC2X3 and IFC4, BUILD_IFCGEOM off, no kernels): pre-1901,
post-2038, both 32-bit boundaries, and a 9e12 value all round trip exactly both in
memory and through STEP text serialization (write then reopen). A value outside the
64-bit range now raises a clean exception instead of corrupting data. Ordinary
in-range integers and integer aggregates (e.g. IfcTriangulatedFaceSet.CoordIndex) are
unaffected. The existing util/test_attribute.py and test_file.py suites pass
unchanged; test_entity_instance.py has 5 pre-existing failures unrelated to this
change (confirmed identical on an unfixed build of this branch, caused by a missing
get_info_2 binding and _patch_swig_comparisons never being implemented here).
Generated with the assistance of an AI coding tool.
Fixes builds with newer GCC/libstdc++ that no longer provide <cstdint>,
<cstring>, <cfloat>, <memory>, <algorithm> etc. transitively. Also
disambiguates visit<> calls in taxonomy.h with the full namespace and
casts the character value in IfcCharacterDecoder to uint32_t to silence
ambiguous overload warnings.
Dion Moult