util.doc: stop truncating entity descriptions at the first paragraph (#4624)

get_entity_doc returned incomplete descriptions for many entities and
types, e.g. IFC4 IfcAlarmType ended mid-sentence at "The set of shared
information may include:". The DocExtractor took only the FIRST <p> of the
markdown-derived HTML (BeautifulSoup .find("p").text), silently dropping
any bulleted list embedded in the definition and every paragraph after it.

Add DocExtractor.extract_full_description, which walks all top-level
<p>/<ul>/<ol> elements in document order (list items rendered as "- item"),
stops before any <blockquote> (HISTORY/NOTE remarks), and strips inline
kramdown attribute markers. The four entity/type extraction sites now use
it; the property-set extraction sites are left untouched to keep this
change scoped.

The shipped schema JSON data is refreshed from the buildingSMART IFC doc
sources (IFC4.0.2.1 and Ifc2.3.0.1): 1491 entity/type descriptions are
completed. Only top-level description fields change; attribute
descriptions and all other fields are byte-identical to the previous data,
and 11 regenerated descriptions containing extraction artifacts were kept
at their previous text.

Verified: IfcAlarmType now includes the full "may include" list and the
closing paragraph; IfcBeamType, IfcWindow and IFC2X3 IfcWallStandardCase
spot-checked complete with no HISTORY leakage.

Generated with the assistance of an AI coding tool.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Petru Conduraru
2026-07-12 14:32:40 +03:00
parent 0b7e25a3ef
commit 5a7ab0ca68
5 changed files with 1522 additions and 1495 deletions
File diff suppressed because one or more lines are too long