Glassvision/IfcOpenShell
1
0
Fork 0
mirror of https://github.com/IfcOpenShell/IfcOpenShell.git synced 2026-09-21 14:23:53 +00:00
Code Issues Packages Projects Releases Wiki Activity

ifcparse: opt in to running the full parse through the paged reader

Browse Source 
file::paged_reading(true), set before initialize(), runs the full parse
(serial or parallel) through the paged reader with 64 KB pages and a
4 MB cache instead of reading the whole file into memory; the whole
file is then never held. Every stage already reads through the reader,
so nothing else changes. The equality test now runs the same file paged,
serially and with five workers each holding its own page cache.

TXG 58 MB / 210_King 147 MB / OKgate22 231 MB: one thread 1.09 / 2.89 /
5.25 s against 1.07 / 2.75 / 5.12 s in memory, twelve threads 0.48 /
1.27 / 1.99 s against 0.44 / 1.24 / 2.01 s; peak memory 311 / 727 /
1119 MB against 365 / 871 / 1347 MB, that is, down by the size of the
file. Whether this should become the default is a decision the numbers
on the PR are meant to inform.

This commit was written by an AI coding tool and has not been verified by
a human.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
This commit is contained in:
Dion Moult
2026-09-14 07:32:12 +10:00
parent b1cba757b8
commit 43686bde85
4 changed files with 41 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/ifcparse/tests/test_ifcopenshell_parse.cpp
+19 -1
View File
@@ -508,7 +508,7 @@ TEST_CASE("Lazy loading passes over a stray keyword like the full parser and fal
CHECK(lazy.instance_by_id(8));
}
TEST_CASE("Parallel parsing yields the same instances, attributes, inverses and GlobalIds as serial parsing, comments in DATA included", "[ifcparse]") {
TEST_CASE("Parallel and paged parsing yield the same instances, attributes, inverses and GlobalIds as serial in-memory parsing, comments in DATA included", "[ifcparse]") {
// The fixture is small, so the threshold would keep it serial; write a
// file big enough to be chunked by repeating its DATA section under new
// names, with a comment and a string holding '/*' between the copies.
@@ -560,6 +560,16 @@ TEST_CASE("Parallel parsing yields the same instances, attributes, inverses and
ifcopenshell::file parallel(ifcopenshell::uninitialized_tag{});
parallel.parse_threads(5);
REQUIRE(parallel.initialize(path.string()));
// The same file through the paged reader, serially and with 5 workers,
// each with its own page cache.
ifcopenshell::file paged(ifcopenshell::uninitialized_tag{});
paged.paged_reading(true);
paged.parse_threads(1);
REQUIRE(paged.initialize(path.string()));
ifcopenshell::file paged_parallel(ifcopenshell::uninitialized_tag{});
paged_parallel.paged_reading(true);
paged_parallel.parse_threads(5);
REQUIRE(paged_parallel.initialize(path.string()));
std::filesystem::remove(path);
size_t count = 0;
@@ -573,6 +583,14 @@ TEST_CASE("Parallel parsing yields the same instances, attributes, inverses and
a.to_string(sa);
b.to_string(sb);
REQUIRE(sb.str() == sa.str());
for (ifcopenshell::file* other : {&paged, &paged_parallel}) {
const express::base c = other->instance_by_id((int)a.id());
REQUIRE(c);
std::ostringstream sc;
c.to_string(sc);
REQUIRE(sc.str() == sa.str());
REQUIRE(other->instances_by_reference((int)a.id()).size() == serial.instances_by_reference((int)a.id()).size());
}
++count;
}
size_t parallel_count = 0;