New 518-line demo showing ElementSet + immutable fields architecture:
- ElementSet struct groups elements with type-stable fields
- Elements contain only connectivity (NTuple, zero-cost)
- Fields live in ElementSet as NamedTuple (immutable, type-stable)
- Mock GPU execution showing kernel compatibility
Three execution modes demonstrated:
1. CPU assembly with zero allocations
2. Mock GPU assembly (simulates CUDA pattern)
3. Time stepping with field container recreation
Key validations:
- Zero allocations in assembly loop (verified with @benchmark)
- GPU kernel accesses element.connectivity directly
- Creating new NamedTuple ~1000× faster than deepcopy (10ns vs 10μs)
- CPU/GPU results match within 1e-10 relative error
Features:
- Mock GPU module (simulates CUDA.jl without dependency)
- AssemblyCache for pre-allocated buffers
- Time stepping simulation (5 steps with displacement updates)
- Memory usage comparison (immutable vs mutable patterns)
- Complete validation suite with performance metrics
Run with: julia --project=. demos/gpu_elementset_demo.jl
New 203-line GPU-only demonstration (simplified without MPI complexity):
Setup and validation (lines 1-52):
- Checks for CUDA availability, exits if not found
- Reports GPU model and memory
- Sets up problem: 10000 nodes, 1000 elements
- Type-stable data: nodes (Float64), connectivity (Int), E, ν
GPU kernel (lines 54-120):
- assemble_element_kernel! for CUDA
- Type-stable: Float64, Int32, CuDeviceMatrix/Vector
- No allocations in kernel
- Computes simplified assembly: K_local = E * (1-ν²)
Execution (lines 122-170):
- Transfers data to GPU (nodes, connectivity)
- Reports bytes transferred
- Launches kernel with thread blocks
- Transfers results back from GPU
Verification (lines 172-203):
- Compares computed vs expected values
- Reports success/failure
- Key achievements summary:
* Type-stable kernel compiled
* Fast GPU memory transfer (typed arrays)
* Zero allocations in kernel
- Why it matters: Dict-based storage CANNOT compile for GPU
- Conclusion: type stability required for modern HPC
Purpose: Simpler test than full MPI version, focuses on GPU capability
Run: julia demos/gpu_only_demo.jl (requires CUDA GPU)
New 327-line mock demonstration (prototype before real hardware version):
MockCUDA module (lines 20-64):
- Mock CuArray type wrapping CPU arrays
- Mock cu() transfer (simulates CPU→GPU)
- Mock @cuda macro (simulates kernel launch)
- Mock thread/block indexing functions
- Demonstrates API without requiring CUDA.jl dependency
GPU kernel example (lines 66-180):
- Type-stable element assembly kernel
- Shows concrete types required (Float64, Matrix{Float64})
- Demonstrates zero-allocation pattern
- Mock execution showing what real CUDA would do
MPI communication examples (lines 182-280):
- Mock MPI module with Send/Recv
- Type-stable data transfer patterns
- Demonstrates fast vs slow paths
Summary (lines 282-327):
- Why type stability matters for GPU/MPI
- GPU: type-unstable code FAILS to compile
- MPI: typed arrays 100× faster than serialization
- Zero allocations required in GPU kernels
- Pattern: typed structures → pre-allocated buffers → type-stable code
- Critical insight: type stability is REQUIREMENT not optimization
Purpose: Educational prototype demonstrating concepts before real hardware.
Superseded by: gpu_mpi_demo.jl (real CUDA and MPI)
New 400-line distributed FEM solver demonstration with 6 parts:
Part 1: Generate test problem (lines 61-100)
- 10×10 SPD system, condition number ~3.45
- Distributed nodal assembly: each rank owns nodes
- Exact solution x=[1,2,...,10], RHS b=A*x
Part 2: Nodal assembly pattern (lines 101-140)
- get_row(i) and get_rhs(i) abstractions
- Row-by-row matrix construction
- Each rank assembles its local rows
Part 3: GPU transfer (lines 141-167)
- Transfer local data to GPU if CUDA available
- Falls back to CPU arrays if no GPU
- Reports bytes transferred per rank
Part 4: Distributed matrix-vector product (lines 168-203)
- matvec_distributed! function
- Each rank computes y_local = A_local * x_global
- GPU acceleration if available, CPU fallback
Part 5: Conjugate Gradient solver (lines 204-318)
- cg_distributed() with MPI collectives
- Allreduce for global dot products
- Allgatherv for vector assembly
- Reports convergence progress per iteration
Part 6: Verification (lines 319-345)
- Compare computed vs exact solution
- Report relative error
- Pass/fail verification (threshold 1e-6)
Summary (lines 346-400):
- Reports what was demonstrated on real hardware
- Nodal assembly, distributed computing, multi-GPU, Krylov CG
- Key insight: type-stable + nodal → scalable
- Relevance to JuliaFEM contact mechanics
Results: Converges in 9 iterations, 7.73×10⁻¹⁴ relative error
Hardware: 2 MPI ranks, NVIDIA RTX A2000 12GB per rank
Run: mpiexec -np 2 julia --project=. demos/krylov_mpi_gpu_demo.jl
New 322-line demonstration script proving type-stable data flows to GPU and MPI:
Part 1: Type-stable data structures (lines 54-77)
- Creates nodes, connectivity, displacement as typed arrays
- Material properties E, ν as Float64
- All structures explicitly typed (Matrix{Float64}, not Dict)
Part 2: MPI communication (lines 79-123)
- Rank 0 sends 24KB displacement data to rank 1
- Transfers material properties
- Validates data integrity with checksum
- Uses MPI.Send/Recv with typed buffers
Part 3: GPU kernel execution (lines 125-192)
- Defines assemble_element_kernel! for CUDA
- Type-stable kernel: Float64, Int32, no allocations
- Transfers data to GPU (CuArray)
- Launches kernel with thread blocks
- Validates results against expected values
Part 4: Combined GPU+MPI workflow (lines 194-271)
- Rank 0 computes on GPU
- Transfers results via MPI to rank 1
- End-to-end validation
Summary section (lines 273-322):
- Reports hardware used (GPU model, MPI ranks)
- Key insights: type stability required for GPU, enables fast MPI
- Conclusion: type-stable fields are foundation for modern FEM
Requirements: MPI (required), CUDA (optional, detects and uses if available)
Run: mpiexec -np 2 julia --project=. demos/gpu_mpi_demo.jl
New 77-line guide documenting:
- Prerequisites (MPI and CUDA globally installed)
- Running commands for MPI communication test
- Running commands for combined GPU+MPI test
- Single-process GPU test instructions
- What gets demonstrated (type stability requirement, MPI fast transfer, real hardware)
- Success indicators and result interpretation
- Key insight: same patterns enable CPU speedup, GPU execution, and MPI efficiency
New 125-line README documenting:
- Two main demonstrations (GPU+MPI and Krylov solver)
- Requirements (Julia 1.9+, MPI, optional CUDA)
- Key insights: type stability required for GPU/MPI/Krylov
- Nodal assembly pattern explanation
- Architecture validation (v0.5.1 vs v1.0 comparison)
- References to benchmarks and design docs
- Contributing guidelines for new demos
Jukka Aho