Generated by: julia --project=. src/basis/lagrange_generator.jl
Changes:
- All 15 element types now use Lagrange{T,P} parametric type
- Functions: get_reference_element_coordinates(), eval_basis!(), eval_dbasis!()
- Reference coordinates now return tuples (zero-allocation)
- Removed old Seg2Basis, Tri3Basis, Quad4Basis, etc. struct definitions
- All methods work with both Type{Lagrange{T,P}} and Lagrange{T,P} instances
Validated:
- Triangle: Kronecker delta property holds (N_i(x_j) = δ_ij)
- Quadrilateral, Tetrahedron, Hexahedron: First node evaluates to (1,0,0,...)
- Derivatives: Correct gradients at reference coordinates
- Comment out Base.getproperty(element::Element, :fields) redirection
- Old code redirected element.fields → element.dfields (Dict-based fields)
- New Element has fields::F directly (type-stable NamedTuple or struct)
- No redirection needed with new architecture
- Rationale: New Element{N,NIP,F,B} has fields as direct struct member
Design for handling both nodal and element fields in nodal assembly:
Architecture:
- Nodes have geometry (immutable)
- Elements have connectivity + fields (immutable struct)
- Nodal fields: displacement, temperature, contact pressure
- Element fields: integration point data (σ, ε_plastic, α, C)
Update pattern:
- Create new field containers (NamedTuples)
- Create new elements with updated fields
- Shallow copy element vector, replace elements
- All immutable (GPU-compatible, thread-safe)
GPU kernel:
- Loops over nodes (nodal assembly)
- Accesses nodal_fields for global quantities
- Accesses element.fields for integration point data
- Gathers from connected elements (node_to_elements)
- No atomic operations (each node owns DOFs)
Material state update:
- Process elements in parallel (Threads.@threads)
- Extract nodal displacements from solution
- Compute strains at integration points
- Run material model (plasticity, damage, etc.)
- Create new elements with updated state
- Return new problem with updated fields
Newton iteration:
- Residual uses element.fields.C (current tangent)
- GMRES with matrix-free matvec (nodal assembly)
- Material update after each iteration
- All data structures immutable throughout
Benchmarks show creating new containers ~1000× faster than deepcopy
Explains why JuliaFEM uses nodal assembly instead of element assembly:
Five major advantages:
1. No atomic operations on GPU (each node writes to own DOFs)
2. Contact mechanics is natural (forces at nodes, not elements)
3. Clean domain decomposition (explicit node ownership for MPI)
4. Better cache locality (sequential node processing)
5. Adaptive refinement easier (local node operations)
Key data structure:
- NodeSet contains nodes + elements + node_to_elements connectivity
- Inverse connectivity enables gathering from connected elements
- Fields accessed via node_set.fields (type-stable)
Algorithm:
- Loop over nodes (not elements)
- Each node gathers contributions from connected elements
- Direct write to owned DOFs (no race conditions)
- Perfect for matrix-free Krylov methods
Validated with demo:
- CPU/GPU results match exactly (0.0 relative error)
- Average 3.24 elements per node (efficient gathering)
- Natural integration with contact mechanics
Compares to traditional element assembly:
- Element: scatter to nodes (atomic ops, cache misses)
- Nodal: gather from elements (no atomics, better cache)
Complete algorithm for GPU-resident FEM solver with nodal assembly:
- Data partitioning by node ownership (domain decomposition)
- GPU-resident data structures (nodes, elements, connectivity, state)
- Three GPU kernels: residual, matvec, state update
- MPI communication patterns for interface nodes
- Full Newton-GMRES loop on GPU (data stays resident)
Architecture:
- Each GPU owns subset of nodes (exclusive ownership)
- Ghost elements copied for gathering during assembly
- node_to_elements connectivity enables nodal assembly
- No atomic operations (each GPU writes to owned DOFs only)
Key features:
- Data moves to GPU once at start, back once at end
- GMRES iterations entirely on GPU (Arnoldi steps)
- Material state updates on GPU (integration points)
- MPI exchanges only for interface DOFs between iterations
- O(N) memory per GPU (matrix-free)
Handles nonlinearity:
- Element state contains σ, ε_plastic, α, C (tangent)
- Residual kernel uses current stress/tangent
- State update kernel after convergence
- Natural for contact mechanics (nodal forces)
Status: Design document for future GPU implementation
Explains the correct pattern for matrix-vector products in Krylov methods:
- Fields accessed through element_set (not passed separately)
- GPU kernel computes y=K*x (not K itself)
- O(N) memory (vs O(N²) for stored matrix)
- Type-stable field access (compile-time types)
Key insights:
- GMRES needs matvec operation, not the matrix
- ElementSet contains elements + fields together
- Zero allocations with immutable connectivity/fields
- Natural pattern for contact mechanics (nodal updates)
- Material state separate from field parameters
Compares old vs new approach:
- Old: Dict{String,Any} in element (type-unstable)
- New: NamedTuple in ElementSet (type-stable)
- Old: O(N²) matrix storage
- New: O(N) matrix-free operator
Validated with gpu_elementset_matvec_demo.jl:
- GPU/CPU results match exactly
- Fields accessed naturally through element_set
- Returns y vector (what Krylov methods need)
Analyzes field storage patterns and recommends ElementSet approach:
- Element has NO field type parameter (simpler type)
- ElementSet groups elements + shared fields
- Fields can be NamedTuple, struct, any type-stable container
- Embraces immutability (GPU-compatible, thread-safe)
- Separates mutable state from immutable parameters
Design rationale:
- Benchmarks show NamedTuple gives 9-92× speedup vs Dict
- Immutability enables GPU execution without copying
- Creating new containers ~1000× faster than deepcopy
- Matches physical reality (material properties per set)
Compares three options:
1. Fields as type parameter (type proliferation)
2. ElementSet pattern (RECOMMENDED)
3. Hybrid approach (too complex)
Addresses common concerns:
- Time-dependent fields (use interpolation)
- Material state (separate mutable arrays)
- Custom field types (any type-stable container works)
Status: Ready for implementation
New 518-line demo showing ElementSet + immutable fields architecture:
- ElementSet struct groups elements with type-stable fields
- Elements contain only connectivity (NTuple, zero-cost)
- Fields live in ElementSet as NamedTuple (immutable, type-stable)
- Mock GPU execution showing kernel compatibility
Three execution modes demonstrated:
1. CPU assembly with zero allocations
2. Mock GPU assembly (simulates CUDA pattern)
3. Time stepping with field container recreation
Key validations:
- Zero allocations in assembly loop (verified with @benchmark)
- GPU kernel accesses element.connectivity directly
- Creating new NamedTuple ~1000× faster than deepcopy (10ns vs 10μs)
- CPU/GPU results match within 1e-10 relative error
Features:
- Mock GPU module (simulates CUDA.jl without dependency)
- AssemblyCache for pre-allocated buffers
- Time stepping simulation (5 steps with displacement updates)
- Memory usage comparison (immutable vs mutable patterns)
- Complete validation suite with performance metrics
Run with: julia --project=. demos/gpu_elementset_demo.jl
Remove duplicate "JuliaFEM Documentation" header (line 11) and add blank lines
for consistent list formatting in three-manual organization document.
Changes:
- Line 11: Removed duplicate H1 title (already in frontmatter)
- Lines 22, 41, 61, 110: Added blank lines before list items for markdown clarity
- Maintains three-audience structure: Users, Contributors, Researchers
- Preserves content organization and cross-reference section
This is a formatting-only change to improve readability. No content modified.
Create 299-line focused design rationale explaining why type stability is essential
for CPU/GPU/MPI performance, without mandating specific storage patterns.
Executive summary (lines 16-26):
- v0.5.1 Dict{String,Any}: 9-92× performance degradation
- Type-unstable code CANNOT run on GPUs
- Significant MPI communication overhead
- Document does NOT prescribe storage location
- Demonstrates why type stability at access points is essential
- Key: Storage pattern matters less than type inference
Problem analysis (lines 28-67):
- Type instability definition: Runtime dispatch when type unknown at compile time
- Why it matters: 10-100× slower CPU, GPU compilation fails, MPI serialization
- Measured impact table: 9-92× speedup, 0 allocations with type stability
- Critical: Zero allocations required for GPU kernels
Design requirements (lines 69-136):
1. Type stability at access points (compiler must infer types)
- Fields could be element-local, global arrays, or arguments
- Access pattern must be type-stable regardless
2. Zero allocations in hot paths (GPU/MPI requirement)
- Assembly loop must allocate nothing
3. Contiguous memory layout (GPU/MPI optimization)
- CUDA transfers contiguous arrays directly
4. Immutable where possible (safe parallelism)
- Thread-safe reads without locks
Demonstrated solutions (lines 138-201) - EXAMPLES, not mandates:
1. NamedTuple container: Simple, type-stable, immutable
2. Struct with typed fields: Explicit, self-documenting
3. Passed as arguments: Maximum type stability, explicit dependencies
- All three achieve type stability
- Choice depends on use case, not performance
GPU and MPI rationale (lines 203-237):
- GPU execution: CUDA requires all code type-stable
- Mock demonstration in benchmarks/gpu_mpi_mock.jl
- MPI communication: Typed arrays use fast memcpy vs slow serialization
- Type stability enables identical code for CPU/GPU
Recommendations (lines 239-256):
- Use type-stable access patterns (REQUIRED)
- Prefer immutable data structures (threading/GPU)
- Pre-allocate caches (zero allocations)
- Use contiguous arrays (GPU/MPI transfer)
- Profile with @btime (verify zero allocations)
- Does NOT mandate: Storage location, container type, dynamic vs static
Validation (lines 258-275):
- benchmarks/field_storage_comparison.jl: 9-92× CPU speedup
- benchmarks/gpu_mpi_mock.jl: GPU/MPI patterns
- benchmarks/VALIDATION_RESULTS.md: Summary table
Conclusion (lines 277-299):
- Type stability is fundamental requirement, not implementation detail
- Enables: High CPU performance, GPU execution, efficient MPI, safe threading
- v1.0 must ensure type stability at access points
- Storage pattern is secondary concern (memory, cache, API)
- Next steps: Review, benchmark, choose pattern, implement, validate CUDA
Key difference from v1: Shorter (299 vs 1114 lines), focused on WHY not HOW,
explicitly states storage pattern is flexible, emphasizes GPU/MPI requirements.
Platform: Julia 1.12.1, November 9, 2025
Series: The JuliaFEM Book, Chapter 5
Status: Design rationale with validated measurements
Create 415-line blog post combining technical demonstration with philosophical
vision for JuliaFEM v1.0 nodal assembly architecture.
Content structure:
- Lines 1-32: Why Krylov+nodal is brilliant for contact mechanics
* Contact is inherently nodal (constraints at nodes, not elements)
* Krylov only needs matvec (never forms global matrix)
* Nodal assembly provides natural row-by-row interface
* O(N) memory vs O(N²) for traditional element assembly
- Lines 34-73: Controversial hypothesis about nodal material modeling
* Claims material state should be at nodes, not integration points
* Argues integration points constrain physics to numerical method
* Variational consistency, physical meaning, scalability arguments
* "I will show them they're wrong" - experimental vision
- Lines 75-95: GMRES advantage for unsymmetric systems
* Material nonlinearity, contact, large deformation all unsymmetric
* GMRES solves positive definite unsymmetric systems
* O(N·iter) time, O(N) memory vs O(N³)/O(N²) for direct solvers
- Lines 97-415: Working GMRES demonstration on 10×10 unsymmetric system
* Problem setup: Positive definite but unsymmetric matrix (lines 105-145)
* Nodal assembly pattern: get_row() interface (lines 147-193)
* Simplified GMRES implementation (lines 195-282)
* Execution and verification (lines 284-318)
* Results: Converged in 10 iterations, 6.28×10⁻¹⁶ relative error
* Key insights section explaining significance (lines 320-365)
* Development roadmap: Immediate → Near-term → Long-term → Vision (lines 367-401)
* Conclusion: Philosophical statement about nodal correctness (lines 403-415)
Technical validation:
- Matrix: 10×10, eigenvalues [9.94, 38.06], condition number 3.83
- Nodal matvec: 1.59×10⁻¹⁴ error vs direct computation
- GMRES: 10 iterations to convergence
- Solution accuracy: 1.23×10⁻¹⁴ absolute error, 6.28×10⁻¹⁶ relative
Dependencies: LinearAlgebra, Random, Printf
Format: Literate.jl (# # for section headers, # for narrative)
Target: Blog post for JuliaFEM v1.0 development documentation
Tone: Opinionated, controversial, technically rigorous
New 203-line GPU-only demonstration (simplified without MPI complexity):
Setup and validation (lines 1-52):
- Checks for CUDA availability, exits if not found
- Reports GPU model and memory
- Sets up problem: 10000 nodes, 1000 elements
- Type-stable data: nodes (Float64), connectivity (Int), E, ν
GPU kernel (lines 54-120):
- assemble_element_kernel! for CUDA
- Type-stable: Float64, Int32, CuDeviceMatrix/Vector
- No allocations in kernel
- Computes simplified assembly: K_local = E * (1-ν²)
Execution (lines 122-170):
- Transfers data to GPU (nodes, connectivity)
- Reports bytes transferred
- Launches kernel with thread blocks
- Transfers results back from GPU
Verification (lines 172-203):
- Compares computed vs expected values
- Reports success/failure
- Key achievements summary:
* Type-stable kernel compiled
* Fast GPU memory transfer (typed arrays)
* Zero allocations in kernel
- Why it matters: Dict-based storage CANNOT compile for GPU
- Conclusion: type stability required for modern HPC
Purpose: Simpler test than full MPI version, focuses on GPU capability
Run: julia demos/gpu_only_demo.jl (requires CUDA GPU)
New 327-line mock demonstration (prototype before real hardware version):
MockCUDA module (lines 20-64):
- Mock CuArray type wrapping CPU arrays
- Mock cu() transfer (simulates CPU→GPU)
- Mock @cuda macro (simulates kernel launch)
- Mock thread/block indexing functions
- Demonstrates API without requiring CUDA.jl dependency
GPU kernel example (lines 66-180):
- Type-stable element assembly kernel
- Shows concrete types required (Float64, Matrix{Float64})
- Demonstrates zero-allocation pattern
- Mock execution showing what real CUDA would do
MPI communication examples (lines 182-280):
- Mock MPI module with Send/Recv
- Type-stable data transfer patterns
- Demonstrates fast vs slow paths
Summary (lines 282-327):
- Why type stability matters for GPU/MPI
- GPU: type-unstable code FAILS to compile
- MPI: typed arrays 100× faster than serialization
- Zero allocations required in GPU kernels
- Pattern: typed structures → pre-allocated buffers → type-stable code
- Critical insight: type stability is REQUIREMENT not optimization
Purpose: Educational prototype demonstrating concepts before real hardware.
Superseded by: gpu_mpi_demo.jl (real CUDA and MPI)
New 400-line distributed FEM solver demonstration with 6 parts:
Part 1: Generate test problem (lines 61-100)
- 10×10 SPD system, condition number ~3.45
- Distributed nodal assembly: each rank owns nodes
- Exact solution x=[1,2,...,10], RHS b=A*x
Part 2: Nodal assembly pattern (lines 101-140)
- get_row(i) and get_rhs(i) abstractions
- Row-by-row matrix construction
- Each rank assembles its local rows
Part 3: GPU transfer (lines 141-167)
- Transfer local data to GPU if CUDA available
- Falls back to CPU arrays if no GPU
- Reports bytes transferred per rank
Part 4: Distributed matrix-vector product (lines 168-203)
- matvec_distributed! function
- Each rank computes y_local = A_local * x_global
- GPU acceleration if available, CPU fallback
Part 5: Conjugate Gradient solver (lines 204-318)
- cg_distributed() with MPI collectives
- Allreduce for global dot products
- Allgatherv for vector assembly
- Reports convergence progress per iteration
Part 6: Verification (lines 319-345)
- Compare computed vs exact solution
- Report relative error
- Pass/fail verification (threshold 1e-6)
Summary (lines 346-400):
- Reports what was demonstrated on real hardware
- Nodal assembly, distributed computing, multi-GPU, Krylov CG
- Key insight: type-stable + nodal → scalable
- Relevance to JuliaFEM contact mechanics
Results: Converges in 9 iterations, 7.73×10⁻¹⁴ relative error
Hardware: 2 MPI ranks, NVIDIA RTX A2000 12GB per rank
Run: mpiexec -np 2 julia --project=. demos/krylov_mpi_gpu_demo.jl
New 322-line demonstration script proving type-stable data flows to GPU and MPI:
Part 1: Type-stable data structures (lines 54-77)
- Creates nodes, connectivity, displacement as typed arrays
- Material properties E, ν as Float64
- All structures explicitly typed (Matrix{Float64}, not Dict)
Part 2: MPI communication (lines 79-123)
- Rank 0 sends 24KB displacement data to rank 1
- Transfers material properties
- Validates data integrity with checksum
- Uses MPI.Send/Recv with typed buffers
Part 3: GPU kernel execution (lines 125-192)
- Defines assemble_element_kernel! for CUDA
- Type-stable kernel: Float64, Int32, no allocations
- Transfers data to GPU (CuArray)
- Launches kernel with thread blocks
- Validates results against expected values
Part 4: Combined GPU+MPI workflow (lines 194-271)
- Rank 0 computes on GPU
- Transfers results via MPI to rank 1
- End-to-end validation
Summary section (lines 273-322):
- Reports hardware used (GPU model, MPI ranks)
- Key insights: type stability required for GPU, enables fast MPI
- Conclusion: type-stable fields are foundation for modern FEM
Requirements: MPI (required), CUDA (optional, detects and uses if available)
Run: mpiexec -np 2 julia --project=. demos/gpu_mpi_demo.jl
New 77-line guide documenting:
- Prerequisites (MPI and CUDA globally installed)
- Running commands for MPI communication test
- Running commands for combined GPU+MPI test
- Single-process GPU test instructions
- What gets demonstrated (type stability requirement, MPI fast transfer, real hardware)
- Success indicators and result interpretation
- Key insight: same patterns enable CPU speedup, GPU execution, and MPI efficiency
New 125-line README documenting:
- Two main demonstrations (GPU+MPI and Krylov solver)
- Requirements (Julia 1.9+, MPI, optional CUDA)
- Key insights: type stability required for GPU/MPI/Krylov
- Nodal assembly pattern explanation
- Architecture validation (v0.5.1 vs v1.0 comparison)
- References to benchmarks and design docs
- Contributing guidelines for new demos
Major documentation update (380 additions, 167 deletions):
- Explained topology is pure geometry (NO hardcoded node counts)
- Clarified basis determines BOTH polynomial degree AND node count
- Distinguished node count (connectivity) vs DOF count (unknowns)
- Added examples: Nedelec (edge DOFs), Raviart-Thomas (face DOFs)
- Documented Lagrange{Topology, P} parametric architecture
- Showed why Tri3/Quad4/Tet10 names are anti-pattern
- Updated all code examples to use new architecture
- Explained Serendipity vs full Lagrange tensor products
- Added performance implications and trade-offs
- Showed how one Triangle topology works for P1/P2/P3/Nedelec/etc
- Commented out assemble_mass_matrix! specialization for Element{Tet10}
- Tet10 is now a topology type, not a basis type (name conflict)
- Needs refactoring to use Tet10Basis or new parametric architecture
- Fixed code formatting (spacing around operators, indentation)
- Added TODO comment explaining the issue
- Removed nnodes from topology exports (now in basis module)
- Added exports for new topology names (Triangle, Quadrilateral, etc.)
- Kept old names as exports (they're aliases for backwards compatibility)
- Added explanatory comments about topology vs basis separation
- Added examples showing how node count comes from basis now
- Organized exports by dimension (0D/1D/2D/3D)
- Commented out topology_to_basis() helper and old Element constructors
- These mapped old names (Tri3→Tri3Basis) which no longer exist
- Need to update for new Lagrange{Topology, P} parametric architecture
- Added TODO comment explaining migration needed
- Temporary measure until new Element constructors are implemented
- Changed struct name from Wedge6 to Wedge
- Removed nnodes() method (node count now determined by basis)
- Updated all function signatures to use Wedge
- Added Wedge6 as backwards compatibility alias
- Added note explaining basis determines node count (P1=6, P2=15 nodes)
- Changed struct name from Pyr5 to Pyramid
- Removed nnodes() method (node count now determined by basis)
- Updated all function signatures to use Pyramid
- Added Pyr5 as backwards compatibility alias
- Added note explaining basis determines node count (P1=5, P2=13, P3=29)
- Changed struct name from Hex8 to Hexahedron
- Removed nnodes() method (node count now determined by basis)
- Updated all function signatures to use Hexahedron
- Added Hex8 as backwards compatibility alias
- Added note explaining basis determines node count (Q1=8, Q2=27 nodes)
- Changed struct name from Tet4 to Tetrahedron
- Removed nnodes() method (node count now determined by basis)
- Updated all function signatures to use Tetrahedron
- Added Tet4 as backwards compatibility alias
- Added note explaining basis determines node count (P1=4, P2=10 nodes)
- Changed struct name from Seg2 to Segment
- Removed nnodes() method (node count now determined by basis)
- Updated all function signatures to use Segment instead of Seg2
- Added Seg2 as backwards compatibility alias
- Added note explaining basis determines node count
- Changed struct name from Quad4 to Quadrilateral
- Removed nnodes() method (node count now determined by basis)
- Updated documentation to explain topology vs basis separation
- Added examples showing Lagrange and Serendipity differences
- Added Quad4 as deprecated alias for backwards compatibility
- Clarified that topology defines geometry only, basis determines nodes
- Changed struct name from Tri3 to Triangle
- Removed nnodes() method (node count now determined by basis)
- Updated documentation to explain topology vs basis separation
- Added examples showing Lagrange{Triangle, P} for different degrees
- Added Tri3 as deprecated alias for backwards compatibility
- Clarified that topology defines geometry only, basis determines nodes
Deleted: scripts/generate_lagrange_basis.jl (720 lines)
Reason: Functionality merged into src/basis/lagrange_generator.jl
The generator is now both a library (for inclusion) and a script (for execution).
Run as: julia --project=. src/basis/lagrange_generator.jl
Consolidates scripts/generate_lagrange_basis.jl into src/basis/lagrange_generator.jl
Changes:
- Added Vecish type alias handling for standalone/included execution
- Added vandermonde_matrix() function (~40 lines) for polynomial basis construction
- Added ElementDescription struct with keyword constructor for readability
- Added 15 element definitions with reference coordinates and polynomial ansatz:
* 1D: Seg2, Seg3
* 2D triangles: Tri3, Tri6
* 2D quads: Quad4, Quad8, Quad9
* 3D tets: Tet4, Tet10
* 3D hexes: Hex8, Hex20, Hex27
* 3D pyramid: Pyr5
* 3D wedges: Wedge6, Wedge15
- Added generation script block (~550 lines) that runs when file executed directly
- Generator now appends "Basis" suffix to all types (Tri3Basis, Quad4Basis, etc.)
- Outputs to src/basis/lagrange_generated.jl with clean formatting
- Includes progress reporting and next steps guidance
Total: 254 → 813 lines (+559 lines)
Run as: julia --project=. src/basis/lagrange_generator.jl
Jukka Aho