file.remove() on a RocksDB-backed file segfaulted. Reducing it turned up
five gaps that each made editing such a file crash or silently do
nothing:
- process_deletion_inverse() decoded the v| inverse-record values as
size_t while the serializer, register_inverse(), unregister_inverse()
and instances_by_reference() use uint32_t, so std::find failed and
vals.erase(end()) was undefined behaviour. It also took the DeleteRange
end from an iterator that is invalid when the instance has no inverse
records. Decode as uint32_t, remove every occurrence guarded on
"found", and derive the range end from the prefix itself.
- attribute_value::size() ignored storage_model_, so every aggregate
assignment on a RocksDB instance threw "Invalid variant index" from
set_attribute_value(). Branch on the storage model like the sibling
accessors and count the deserialized aggregate.
- rocks_db_file_storage::create() was a stub returning an empty handle,
which anything creating an instance then dereferenced. Implement it
after in_memory_file_storage::create().
- max_id_ is only initialised by the in-memory parse, so a RocksDB file
would have handed out ids that overwrite existing instances.
Implement the recalculate_id_counter() stub per backend and run it
once before the first fresh_id() on RocksDB.
- byid_.erase() was a no-op: set_to_map_transformer::erase() and
rocksdb_set_view::erase() were stubs. The deleted instance's attribute
keys and cached handle survived, entity_names() still listed it and
reopening the database threw. Erase deletes every key under the
instance's prefix; the transformer forwards to it and takes an
on-erase hook the storage uses to drop the cached handle.
root.remove_product on the first 200 products of a 61 MB model now
leaves the same surviving ids and inverse counts whether the file was
opened from SPF or converted to RocksDB.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HNrXDmR88wKPCYwGE21SyH
Expose the reader's cached span and pass complete names directly to from_chars in the shared tokenizer. Retain the existing path for whitespace, page boundaries and invalid spellings, including overflow. Both consumers use the same shortcut.
Add tests for one-byte pages, pushed readers, escaped strings, identifier values and invalid-token errors. Targeted MSVC tests pass.
Generated with the assistance of an AI coding tool.
Keep the existing delimiter logic and compile out repeated SWAR probes for consumers that do not retain text. On Clinic MEP the index scan rises from about 200 to 285 MB/s in memory before the identifier optimization.
Generated with the assistance of an AI coding tool.
Only acquire and release a string-pool entry when the consumer retains text or reads an identifier. Discarded literals no longer assign their first character to a pooled string.
Generated with the assistance of an AI coding tool.
Export the explicit full-buffer instantiation instead of the generic class. MSVC otherwise emits a default-constructor closure for the paged reader, instantiating an unsupported constructor.
Generated with the assistance of an AI coding tool.
After the parallel parse (and the parallel lazy index) the main thread
did, per instance, the by-type list push, the GlobalId extraction and
the name-table insert, then concatenated every worker's inverse records
and sorted the lot: on a 12-core box that serial merge was most of the
open. Now each worker sorts its own records, builds its own by-type
lists and reads its own GlobalIds; the main thread appends the by-type
lists, inserts names and GlobalIds, and merges the sorted runs pairwise
(inverse_index::merge_sorted, O(n log k)) instead of sorting again.
Instance order, GlobalId precedence and record order are unchanged.
The workers sort their runs in place (inverse_index::sort_in_place, no
radix buffer, no shrink): merge_sorted() copies the runs anyway, and a
buffer allocated on a worker's arena stays there after the open, which
showed as 3–10% more resident memory after a 12-thread open until the
runs sorted in place; peak memory at 12 threads is one transient copy
of the records higher (the merge), single-threaded it is unchanged.
Experiment on top of the series; measured in isolation against the
previous commit, five models, best of three: single-threaded within ±2%
(by construction); 12 threads, strict / lazy: TXG −16% / −25%,
210_King −23% / −26%, OKgate22 −25% / −20%, a 107 MB model −25% / −23%,
a 523 MB model of few large instances −9% / −7%. Together with the
radix sort: strict 12 threads −20% to −29%, lazy 12 threads −31% to
−35% on the ordinary models.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
The inverse index is sorted once after a parse (and after a parallel
merge). std::sort on 1.7–5 million 12-byte records was the largest
serial phase left after the parse itself. sort_records() now does a
stable LSD radix sort on referenced_id, 11 bits per pass and as many
passes as the largest id needs, then applies record_less within each run
of equal ids, so the order is exactly what std::sort produced. Inputs
under 4096 records still use std::sort.
Experiment on top of the series; measured in isolation against the
previous commit, five models, best of three (strict parse 1 / 12 threads,
lazy open 1 / 12 threads): TXG 58 MB −7% / −16% / −10% / −19%,
210_King 147 MB −8% / −17% / −15% / −22%, OKgate22 231 MB −6% / −12% /
−11% / −17%, a 107 MB model −8% / −18% / −12% / −19%, a 523 MB model of
few large instances −3% / −4% / 0% / −5%.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
The body of next() becomes spf_lexer::scan(Consumer&), the same code
wrapped in a loop that hands each token to the consumer's callbacks
(operator_, identifier, string, keyword, enumeration, binary, boolean,
integer, real, literal) instead of building a token object; each
callback returns whether to go on. The consumer's constexpr flags say
what is decoded: decode_strings, decode_values, keep_keywords. It lives
in spf_scan.h, with the SWAR helpers and number parsing it needs, so a
consumer inlines into the loop. next<Policy>() is kept as the consumer
that stops after one token: the attribute reader, header parser and
streamer pull tokens recursively and stay as they are.
The lazy index is now attribute_consumer: depth and attribute index from
the operators, every name straight into the inverse index, the bounds
of the first attribute if it is a string, done at the closing semicolon.
The attribute_tokens policy it replaces is gone.
Tokenizing 50 MB files with nothing decoded, in memory: index policy
through next() 214–247 MB/s, scan() with the inlined consumer 314–403
MB/s (TXG 247 → 345); through 64 KB pages 245–284 MB/s. Lazy open on one
thread TXG / 210_King / OKgate22 0.55 / 1.59 / 2.11 s → 0.52 / 1.57 /
2.05 s. The full tokenizer through the adapter is unchanged (TXG 202–210
MB/s against 195–219 before), as is the strict parse.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
On a file large enough the DATA section is split at the same boundaries
the parallel parse uses (chunk_bounds(), now shared) and each chunk is
indexed by its own worker with its own paged reader, lexer, shells,
offsets, GlobalIds and inverse records; the results are merged in file
order, so instance order, GlobalId precedence and inverse records are
identical to the serial index. The serial index is the same code run on
one chunk. The name table is reserved before the merge, which also
helps the serial case. The default thread count is the one the full
parse uses.
TXG 58 MB / 210_King 147 MB / OKgate22 231 MB, lazy open: 12 threads
0.37 / 1.01 / 1.34 s against 0.55 / 1.59 / 2.11 s on one thread and
0.44 / 1.17 / 1.91 s for the default (parallel full) open; memory after
the open +15 to +50 MB at 12 threads for the workers' page caches. The
equality test now opens the 12 MB replicated fixture lazily with five
workers and compares it instance by instance with the serial parse.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
Inside an attribute list the index looks only at operators and names,
so a third tokenizer policy, attribute_tokens, returns a keyword (an
inline typed value such as IFCLABEL), an enumeration or a binary as
Token_LITERAL without copying its text; only a name's digits are kept.
The instance headers still go through index_tokens, which keeps the
keyword. Same next(), one more compile-time branch.
Found by callgrind on the lazy open (see the PR): string-pool access and
keyword text copying for tokens the index never read. Whole-file
tokenizing of TXG (58 MB) with the index policy 265 MB/s -> 293 MB/s;
lazy open TXG / 210_King / OKgate22 0.58 / 1.73 / 2.86 s -> 0.57 / 1.65 /
2.78 s. Small: the per-token call is the larger cost, which the next
commit addresses by splitting the work over threads.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
-fastproxy binds each wrapped method directly to its compiled function
instead of going through a generated Python def, and -fastdispatch
shortens overload dispatch. Reading is_a(), id() and an attribute of
every one of TXG's 918k instances takes 2.15 s instead of 2.36 s (-8%);
the tight loop over 133 walls reading GlobalId, Name and is_a() twenty
times over 9 ms instead of 11 ms.
validate_stub compared the stub against the wrapper's def signatures by
parsing both files; a fast-proxy wrapper has assignments instead of
defs. It now rebuilds the signature from the compiled function's
autodoc docstring (the C++ prototype) with the rule SWIG itself uses:
one prototype whose defaults are Python literals becomes named
parameters, several prototypes or a non-literal default such as an enum
become *args. Checked against every def of a wrapper built without
-fastproxy: 773 of 773 signatures rebuild identically. The autodoc
feature moves above the SWIG library includes so the iterator and
container classes carry prototypes too.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
file::paged_reading(true), set before initialize(), runs the full parse
(serial or parallel) through the paged reader with 64 KB pages and a
4 MB cache instead of reading the whole file into memory; the whole
file is then never held. Every stage already reads through the reader,
so nothing else changes. The equality test now runs the same file paged,
serially and with five workers each holding its own page cache.
TXG 58 MB / 210_King 147 MB / OKgate22 231 MB: one thread 1.09 / 2.89 /
5.25 s against 1.07 / 2.75 / 5.12 s in memory, twelve threads 0.48 /
1.27 / 1.99 s against 0.44 / 1.24 / 2.01 s; peak memory 311 / 727 /
1119 MB against 365 / 871 / 1347 MB, that is, down by the size of the
file. Whether this should become the default is a decision the numbers
on the PR are meant to inform.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
The DATA section is split into one chunk per thread and each worker runs
the same per-instance reader as the serial parse over its own reader,
storage, inverse records and simple-type list; the results are merged in
file order, so instance order, GlobalId precedence and inverse records
are identical to the serial parse. Reference resolution then splits over
the same threads: each instance's slots are its own and the name table
is complete and read-only by then. The default is one thread per core,
capped at 16; IFCOPENSHELL_PARSE_THREADS or file::parse_threads()
overrides it, and 1 parses as before.
The instance headers are read by one loop, for_each_instance_header(),
shared with the lazy index: it looks declarations up once per keyword,
passes over a bypassed instance's attribute list and slides past a stray
keyword the way the serial reader does (the lazy index therefore no
longer falls back on one).
Finding the split points is the one place that looks at raw bytes rather
than tokens, because tokenizing the file serially first would leave
nothing to parallelise. It applies three rules: a string starts and ends
at a quote and cannot span a line, and a comment runs from /* to */; a
split is a '#' that starts a line outside both. Getting a string's end
wrong can only lose a candidate, never accept a wrong one, since no
string contains a newline. The equality test puts a comment holding a
fake instance and a string holding "/*" between the chunks.
file_reader gains for_each_span(), which hands a byte range out span by
span (one span for a buffer, one per page for the paged reader), and
reopen(), a reader over the same file for another thread.
TXG 58 MB / 210_King 147 MB / OKgate22 231 MB, 12 threads: 0.44 / 1.24 /
2.01 s against 1.07 / 2.75 / 5.12 s on one thread; memory after the parse
within 1–4%, peak +5–4%.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
A lazy open reads the DATA section once with the tokenizer's index
policy and builds what indexes the file: a shell per instance (name and
declaration, no attribute array), the complete inverse index with
attribute indices, the GlobalId map and the by-type lists. No attribute
value is decoded. The first time an instance's attributes are touched,
ensure_loaded() seeks the retained paged reader to the instance and runs
the same load_attributes() the full parse runs, with inverse registration
off, then resolves that instance's references from its own slots. A
modified instance is materialised first, so writing works.
There is no scanner of its own: the index pass consumes next<index_tokens>()
and counts parentheses and commas on the operator tokens; a keyword where
an instance should start, or a token the tokenizer rejects, stops the
index and the file is parsed in full. The offset of each instance's
attribute list is kept in one sorted vector that exists only in lazy
mode, so a full parse pays nothing for it. Materialising from several
threads at once is not safe.
TXG 58 MB / 210_King 147 MB / OKgate22 231 MB, single thread: lazy open
0.61 / 1.73 / 2.86 s against the full parse's 1.05 / 2.69 / 4.99 s, at
141 / 374 / 534 MB against 274 / 654 / 1036 MB; reading one attribute of
every instance afterwards costs a further 0.56 / 1.44 / 4.86 s.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
The parser could not resolve a #name when it read it, because the
instance may be defined further down the file, so it left the slot empty
and appended (owner, attribute, name) to a side table that a second pass
walked. The table held one entry per reference for the whole read: 64 MB
on a 58 MB model, the high-water mark of opening.
Now the reference stays where the tokenizer put it: the attribute slot
holds the instance_reference, or the reference_or_simple_type aggregate
for a list (mixed with inline typed values or not), until every instance
has been read, and resolve_instance_references() walks each instance's
slots and swaps names for instances. Ordering is what the tokenizer
produced; nothing is re-derived. A missing name becomes null in a scalar
and is dropped from an aggregate, as before; the error keeps its offset.
The three transient alternatives are appended to the attribute pack and
to argument_type in lock step and are never visible once a file is
loaded. Simple type instances read inline (IfcPropertySetDefinitionSet)
have their own slots, so their references need no diversion.
The table remains for the header entities and for streaming consumers of
instance_streamer::references(), which leave resolve_references_in_place
off.
TXG 58 MB / 210_King 147 MB / OKgate22 231 MB, single thread: time
unchanged (1.05 / 2.69 / 4.99 s), memory after the parse 287 -> 274,
698 -> 654, 1086 -> 1036 MB, peak 400 -> 365, 965 -> 871, 1471 -> 1347 MB.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
spf_lexer::next() becomes next<Policy>(). full_tokens, the default, is
what the parser has always had. index_tokens is what the lazy index
needs: a string is ended but not decoded, and a number, enumeration or
binary comes back as Token_LITERAL with only its position; names,
keywords and operators are read as before. Each policy compiles to its
own loop from the one implementation, so there is no second tokenizer.
character_decoder gains skip(): the same state machine as the
conversion with the collection compiled out, so an escape such as \S\'
(an apostrophe as the page character) ends the string at the same byte
under both policies. A byte-level scan would have ended it early.
Also fixes a comment that follows a token without whitespace, ",/* x */",
which skip_comment() never saw because the slash had been consumed.
TXG (58 MB), single thread: tokenizing the whole file 194 MB/s with
full_tokens, 249 MB/s with index_tokens; through 64 KB pages 196 and
205 MB/s. The parse itself is unchanged.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
callgrind on the tokenizer showed SWAR::has_special_char and eq_mask
compiled as calls, one per eight bytes; paged_file_impl::size() out of line
behind every eof() and remaining(); and the cursor's page-cache check not
inlined into peek() because it shared a function with the page fetch. The
SWAR helpers are forced inline, size() is defined in the class, and
cached_() is split into an inline check and an out-of-line refresh.
TXG (58 MB), single thread: tokenizer 196 -> 204 MB/s in memory and
136 -> 193 MB/s through 64 KB pages; strict parse through pages
1.15 -> 0.97 s against 0.93 s in memory; lazy index pass over pages
218 -> 311 MB/s; lazy open 0.52 -> 0.43 s.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
Every peek(), get() and SWAR word read on a paged reader went through the
page cache's hash map and LRU list, so tokenizing through pages ran at a
fifth of the speed of the in-memory buffer. The reader now remembers the
page its cursor was last on and serves reads that fall inside it from the
pointer, revalidated against an eviction counter on the implementation
so a page that left the cache is never read through a stale pointer.
Reads that straddle a page boundary take the existing paths.
The whole tokenizer over each file, 64 KB pages, 64 cached (4 MB):
in-memory buffer paged before paged after
TXG 58 MB 200 MB/s 42 MB/s 142 MB/s
210_King 148 MB 181 MB/s 34 MB/s 118 MB/s
OKgate22 232 MB 197 MB/s 38 MB/s 128 MB/s
This is the step that makes reading in pages a candidate for the default
path rather than a fallback; the remaining gap is the page fetch itself.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
Each instance was four separate allocations: the instance_data record, the
attribute array object it pointed at, that array's index bytes, and its
slot storage. A 58 MB model made 10.4 million mallocs to load 918k
instances, and massif attributed 99 MB of its 450 MB peak to malloc
bookkeeping alone.
variant_array now allocates the size byte, the per-slot type indices and
the slots as one block, and instance_data holds the array in a
std::optional instead of behind a pointer (an empty optional keeps the
meaning the null pointer had: attribute storage constructed on the fly
from the RocksDB backend). No ownership or lifetime changes; the same
object owns the same data.
Parse, C++ file constructor, on top of the previous commits:
TXG 58 MB 0.96 -> 0.94 s steady 310 -> 262 MB peak 383 -> 335 MB
210_King 148 MB 2.74 -> 2.63 s steady 758 -> 630 MB peak 948 -> 820 MB
OKgate22 232 MB 3.70 -> 3.61 s steady 1165 -> 971 MB peak 1447 -> 1252 MB
mallocs while loading TXG: 10.39M -> 7.56M.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
byguid_ was a std::map<std::string, ...>: a red-black node plus a
heap-allocated 22-character string per rooted instance, and a lookup that
walks ~18 levels of string comparisons on a 200k-entry file.
guid_map keeps keys of up to 23 characters inline in an unordered_map node
(every valid GlobalId is 22), and routes anything longer to an ordered map
so invalid files still work. Same std::string-keyed interface as before.
Parse, C++ file constructor, on top of the previous commits:
TXG 58 MB 1.08 s -> 1.03 s 341 -> 335 MB
210_King 148 MB 2.80 s -> 2.72 s 836 -> 830 MB
OKgate22 232 MB 4.25 s -> 3.94 s 1271 -> 1253 MB
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
full_buffer_impl::size(), get(), get_u32() and get_u64() (and their mmap_impl
twins) were defined out of line, so every character the lexer read crossed a
call boundary with its own bounds check. Callgrind put the three at 5.6% of
parse self time; inlining them lets the compiler hoist the checks out of the
scanning loops, which is worth more than their own cost.
Also reserve the streamer's inverse vector from the file size (about one
record per 32 bytes of SPF on real models) and shrink it once the bulk load
is sorted, so the doubling copies and the capacity slack go away.
Parse time, C++ file constructor, 12-core Linux box:
TXG 58 MB 1.35 s -> 1.15 s
210_King 148 MB 3.70 s -> 3.09 s
OKgate22 232 MB 6.18 s -> 5.30 s
Python ifcopenshell.open(): 1.40 -> 1.22, 3.70 -> 3.24, 6.27 -> 5.52 s.
Memory unchanged. The removed exported symbols mean the Python wrapper must
be rebuilt against this library.
This commit was written by an AI coding tool and has not been verified by
a human.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013wcN7XquTfUi4vsKQ4KchL
Also drop-in replacement mostly, except extra cmake args need to be provided now after `--` - `python run-cmake.py vs2022-x64 -- -DGLTF_SUPPORT=ON`.
Internally, script relies on env variables much much less.
Mainly drop-in replacement for `build-ifcopenshell.bat` with `--help`, kw args and args validation, but there's a small caveat.
Previously it was possible to pass args to the underlying build tool as simple positionals - e.g. `build-ifcopenshell vs2022-x64 Release /p:Foo=bar`.
This behaviour is disabled now, because it doesn't allow validating provided args - it's impossible to tell whether `--config Release` is meant to be passed to msbuild or was meant as `--build-cfg Release` for `build-ifcopenshell`.
But it's still possible to pass args to msbuild by using `--` - `python build-ifcopenshell.py vs2022-x64 Release -- /p:Foo=bar`
`install-ifcopenshell.py` is now just a small wrapper passing `--target INSTALL` arg.
* Rewrite to_string() to use switch{} and handle Token_NONE and identifier without as_string()
* Add regression tests for to_string() on tokens without a string form
Cover both halves of the recursion that made a whitespace-only file
segfault: token::to_string() on the EOF marker and on an instance name,
and a parse of input that lexes to zero tokens, which is how the header
parser reaches token::as_string() on the EOF marker.
Generated with the assistance of an AI coding tool.
---------
Co-authored-by: Bruno Postle <bruno@postle.net>
process_deletion_inverse() called inverse_index::remove_source(), which
walked every record in the file's inverse index to find the ones whose
source is the deleted instance: O(R) per deletion, the dominant cost of
file.remove() on large files now that the lookup side no longer re-sorts.
The records a deleted instance contributed are exactly the entity
references in its own attributes, so walk those with the same visitor
build_inverses_() uses for registration and remove each record with a
targeted binary search instead. remove_source() has no callers left and
is deleted.
Also use the ordered view of batch_deletion_ids_ (a boost multi_index
that already had one) for the is-this-referencer-also-being-deleted
check in process_deletion_(), which was a linear std::find over the
sequenced view: O(b) per referencing instance made batch deletion of b
instances quadratic.
file.remove on 300 IfcPropertySet of a 155 MB IFC4 model (201k IfcRoot)
drops from 3.15 ms to 0.17 ms per call, batched removal of 2000 from
3.34 ms to 0.17 ms per call, root.remove_product on 100 walls from
332 ms to 131 ms per call.
Claude-Session: https://claude.ai/code/session_01HNrXDmR88wKPCYwGE21SyH
(cherry picked from commit 938442303f)
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Requiring quotes around a decimal was a surprise with no way to predict it:
Foobar.Baz>1 works, Foobar.Baz>1.5 is a syntax error, and > >= < <= are
almost always used on numbers. Writing Width>"0.2" also reads like a text
comparison even though it is not.
The "." is excluded from unquoted_string because it separates a pset from a
property, but that separator has already been consumed by the time a value is
read, so a number is unambiguous in value position. Add a decimal_string
alternative to the value rule only. Signed and leading dot forms are accepted,
1. is not, and pset, prop and query keys are unchanged - they still need
quoting for a ".".
The transformer returns the same string a quoted value would, so =1.5 and
="1.5" are the same query and compare() does the existing type coercion.
Nothing that parses today changes meaning; only inputs that used to be a
syntax error now work.
Update the quoting docs accordingly, and assert the Bay property that the
filter test set up but never checked.
Generated with the assistance of an AI coding tool.
An unquoted pset, prop, key or value may not contain any of , . = > < * !
or whitespace. This was never stated, and the only motivation the docs gave
for quoting was the "Level 3" example having a space, so there was nothing
to suggest that Pset_WallCommon.ThermalTransmittance=1.5 is a syntax error
while FireRating=2HR is fine. Decimal numbers are the common way to hit it.
State the rule where the reader meets values, in a new subsection under each
of the two value tables. The two grammars exclude different characters, so
the element value section states its own set and only the delta. Note that
all of these characters are a syntax error except the comma, which is read
as the filter separator instead, so Name=Foo,IfcWall silently means "named
Foo and an IfcWall" rather than matching the literal value.
Also fix the Query filter example, which had the same problem in the file
itself: query:types.count=0 does not error, it is silently parsed as a
property filter for a count property in a property set named query:types,
and matches nothing. Quote the keys, as the other query examples already do.
Generated with the assistance of an AI coding tool.
Moving to Python to make Windows build scripts more maintainable.
It's intended to be a drop-in replacement, so it should be possible to just switch `.\build-deps.cmd` to `python build-deps.py`, keeping exactly the same arguments and behaviour will be the same.
`build-deps.cmd` is deprecated, but not yet removed, but will be shortly after more testing.
Other batch files will be migrated to Python shortly after too.
Apparently `ty` is being too strict here and warning about `Any` possibly being `PathLike` which is not supported on older Pythons.
```
error[deprecated]: The overload of `which` is deprecated
--> src\bonsai\bonsai\bim\module\drawing\operator.py:2267:34
|
2267 | command[0] = shutil.which(command[0]) or command[0]
| ^^^^^^^^^^^^ On Windows before Python 3.12, using a PathLike as `cmd` would always fail or return `None`.
error[deprecated]: The overload of `which` is deprecated
--> src\bonsai\bonsai\tool\drawing.py:1324:30
|
1324 | command[0] = shutil.which(command[0]) or command[0]
| ^^^^^^^^^^^^ On Windows before Python 3.12, using a PathLike as `cmd` would always fail or return `None`.
```
Dion Moult