mirror of
https://github.com/JuliaFEM/JuliaFEM.jl.git
synced 2026-08-06 04:21:33 +00:00
docs(book): Add nodal assembly concept and architecture
- Alternative to element-by-element assembly for GPU/matrix-free - Node-by-node loop eliminates atomic operations on GPU - Spider pattern: nodes couple with 10-30 neighbors not all N - NodeToElementsMap: inverse connectivity (node → elements) - get_node_spider() finds coupled nodes for sparse stiffness - NodalStiffnessContribution: 3×3 blocks per node - 307 lines: Experimental architecture with working prototype
This commit is contained in:
@@ -0,0 +1,307 @@
|
||||
---
|
||||
title: "Nodal Assembly: Concept and Data Structures"
|
||||
date: 2025-11-11
|
||||
author: "JuliaFEM Team"
|
||||
status: "Experimental"
|
||||
last_updated: 2025-11-11
|
||||
tags: ["assembly", "nodal", "gpu", "architecture"]
|
||||
---
|
||||
|
||||
## Introduction
|
||||
|
||||
This document describes the **nodal assembly** concept - an alternative to traditional element-based assembly that is naturally suited for:
|
||||
|
||||
- GPU parallelization (no atomic operations needed)
|
||||
- Matrix-free methods (Krylov solvers)
|
||||
- Contact mechanics (contact is inherently nodal)
|
||||
- Domain decomposition (nodes have clear ownership)
|
||||
|
||||
**Status:** Experimental concept with working prototype. See `src/nodal_assembly_structures.jl` and tests.
|
||||
|
||||
## The Problem with Element Assembly
|
||||
|
||||
Traditional FEM assembles **element by element**:
|
||||
|
||||
```julia
|
||||
# Traditional element assembly
|
||||
for element in elements
|
||||
K_local = compute_element_stiffness(element) # 30×30 for Tet10
|
||||
|
||||
# Scatter to global (requires atomic operations on GPU!)
|
||||
for i in 1:ndofs_local, j in 1:ndofs_local
|
||||
K_global[gdof[i], gdof[j]] += K_local[i,j] # Race condition!
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
|
||||
1. **GPU:** Multiple elements write to same global DOF → need atomics → slow
|
||||
2. **Contact:** Contact forces are nodal, but assembly is elemental → mismatch
|
||||
3. **Matrix-free:** Hard to compute K*v without forming K
|
||||
|
||||
## Nodal Assembly Solution
|
||||
|
||||
Assemble **node by node** instead:
|
||||
|
||||
```julia
|
||||
# Nodal assembly
|
||||
for node_i in nodes
|
||||
# Compute contributions FROM all elements touching node_i
|
||||
K_blocks, f_int = compute_nodal_contribution(node_i, elements_touching_i)
|
||||
|
||||
# Each thread owns its node → no atomics needed!
|
||||
w[3*(node_i-1)+1:3*node_i] = matvec_nodal(K_blocks, u)
|
||||
end
|
||||
```
|
||||
|
||||
**Advantages:**
|
||||
|
||||
1. **GPU:** One thread per node, no conflicts, no atomics
|
||||
2. **Contact:** Natural fit (contact forces already nodal)
|
||||
3. **Matrix-free:** Direct K*v computation without forming global K
|
||||
|
||||
## The "Spider" Pattern
|
||||
|
||||
For node $i$, we only compute stiffness blocks for nodes it couples with:
|
||||
|
||||
```text
|
||||
j₃
|
||||
/\
|
||||
/ \
|
||||
/ \
|
||||
j₂------i------j₄ ← Node i's "spider"
|
||||
\ /
|
||||
\ /
|
||||
\/
|
||||
j₁
|
||||
```
|
||||
|
||||
**Key insight:** Most nodes couple with only ~10-30 neighbors (not all N nodes!)
|
||||
|
||||
- **Corner node:** 8 neighbors (1 element touches it)
|
||||
- **Interior node:** 27 neighbors (8 elements touch it)
|
||||
- **Face node:** 12 neighbors (intermediate)
|
||||
|
||||
**Efficiency:** Sparse connectivity preserved without storing full matrix!
|
||||
|
||||
## Data Structures
|
||||
|
||||
### 1. Inverse Mapping: Node → Elements
|
||||
|
||||
```julia
|
||||
struct ElementNodeInfo
|
||||
element_id::Int # Which element
|
||||
local_node_idx::Int # Which local node index (1-10 for Tet10)
|
||||
end
|
||||
|
||||
struct NodeToElementsMap
|
||||
node_to_elements::Vector{Vector{ElementNodeInfo}}
|
||||
nnodes::Int
|
||||
nelements::Int
|
||||
end
|
||||
|
||||
# Usage
|
||||
map = NodeToElementsMap(connectivity)
|
||||
for elem_info in map.node_to_elements[node_i]
|
||||
println("Node $node_i is local node $(elem_info.local_node_idx) ",
|
||||
"in element $(elem_info.element_id)")
|
||||
end
|
||||
```
|
||||
|
||||
**Purpose:** Given node, find all elements touching it (needed for nodal loop).
|
||||
|
||||
### 2. Spider Nodes
|
||||
|
||||
```julia
|
||||
function get_node_spider(map::NodeToElementsMap, node_id::Int,
|
||||
connectivity) -> Vector{Int}
|
||||
spider = Set{Int}()
|
||||
|
||||
# Union of all nodes in elements touching node_id
|
||||
for elem_info in map.node_to_elements[node_id]
|
||||
for node in connectivity[elem_info.element_id]
|
||||
push!(spider, node)
|
||||
end
|
||||
end
|
||||
|
||||
return sort(collect(spider))
|
||||
end
|
||||
```
|
||||
|
||||
**Purpose:** Find all nodes that couple with `node_id` (non-zero stiffness blocks).
|
||||
|
||||
### 3. Nodal Stiffness Contribution
|
||||
|
||||
```julia
|
||||
struct NodalStiffnessContribution{T}
|
||||
node_id::Int
|
||||
spider_nodes::Vector{Int} # Nodes that couple
|
||||
K_blocks::Vector{Tensor{2,3,T}} # 3×3 blocks (one per spider node)
|
||||
f_int::Vec{3,T} # Internal force at this node
|
||||
f_ext::Vec{3,T} # External force at this node
|
||||
end
|
||||
```
|
||||
|
||||
**Purpose:** Storage for nodal assembly. `K_blocks[k]` is the 3×3 coupling between `node_id` and `spider_nodes[k]`.
|
||||
|
||||
**Zero-allocation:** All quantities use `Tensors.jl` types (immutable, stack-allocated).
|
||||
|
||||
## Matrix-Free Matvec
|
||||
|
||||
Given nodal contributions, compute $\mathbf{w} = \mathbf{K} \mathbf{u}$ without forming $\mathbf{K}$:
|
||||
|
||||
```julia
|
||||
function matrix_vector_product_nodal(contrib::NodalStiffnessContribution,
|
||||
u::Vector{Vec{3}}) -> Vec{3}
|
||||
w = zero(Vec{3})
|
||||
|
||||
# Loop over spider nodes (only non-zero columns!)
|
||||
for (k, node_j) in enumerate(contrib.spider_nodes)
|
||||
K_ij = contrib.K_blocks[k] # 3×3 block
|
||||
u_j = u[node_j] # Displacement at node j
|
||||
|
||||
w += K_ij ⋅ u_j # Block matvec
|
||||
end
|
||||
|
||||
return w
|
||||
end
|
||||
```
|
||||
|
||||
**Performance:**
|
||||
|
||||
- Only computes non-zero contributions (sparse spider)
|
||||
- Zero allocations (Tensors.jl)
|
||||
- GPU-friendly (parallel over nodes)
|
||||
|
||||
## Example: 2 Tet4 Elements
|
||||
|
||||
```text
|
||||
Mesh:
|
||||
Element 1: nodes (1,2,3,4)
|
||||
Element 2: nodes (2,3,4,5)
|
||||
|
||||
Nodes 2,3,4 shared between elements
|
||||
```
|
||||
|
||||
**Node 1 (corner):**
|
||||
|
||||
- Touches: 1 element
|
||||
- Spider: [1, 2, 3, 4] (4 nodes)
|
||||
- Needs: 4 × 3×3 blocks
|
||||
|
||||
**Node 2 (interior):**
|
||||
|
||||
- Touches: 2 elements
|
||||
- Spider: [1, 2, 3, 4, 5] (5 nodes = union of both elements)
|
||||
- Needs: 5 × 3×3 blocks
|
||||
|
||||
**Node 5 (corner):**
|
||||
|
||||
- Touches: 1 element
|
||||
- Spider: [2, 3, 4, 5] (4 nodes)
|
||||
- Needs: 4 × 3×3 blocks
|
||||
|
||||
## Assembly Algorithm
|
||||
|
||||
```julia
|
||||
# 1. Build inverse mapping (once, at mesh creation)
|
||||
map = NodeToElementsMap(connectivity)
|
||||
|
||||
# 2. For each node (parallel on GPU)
|
||||
for node_i in 1:nnodes
|
||||
# Find spider
|
||||
spider = get_node_spider(map, node_i, connectivity)
|
||||
|
||||
# Allocate storage
|
||||
contrib = NodalStiffnessContribution(node_i, spider)
|
||||
|
||||
# Loop over elements touching this node
|
||||
for elem_info in map.node_to_elements[node_i]
|
||||
elem = elements[elem_info.element_id]
|
||||
local_idx = elem_info.local_node_idx
|
||||
|
||||
# Compute element contribution to node_i
|
||||
# (loop over integration points inside)
|
||||
compute_element_contribution!(contrib, elem, local_idx, u, time)
|
||||
end
|
||||
|
||||
# Matrix-free matvec: w_i = K_i * u
|
||||
w[node_i] = matrix_vector_product_nodal(contrib, u)
|
||||
end
|
||||
```
|
||||
|
||||
## Comparison to Element Assembly
|
||||
|
||||
| Aspect | Element Assembly | Nodal Assembly |
|
||||
|--------|------------------|----------------|
|
||||
| **Outer loop** | Elements | Nodes |
|
||||
| **Parallelization** | Element → atomics | Node → no atomics |
|
||||
| **Storage** | Full K matrix (sparse) | 3×3 blocks per spider |
|
||||
| **Matrix-free** | Difficult | Natural |
|
||||
| **Contact** | Mismatch | Natural fit |
|
||||
| **GPU** | Slow (atomics) | Fast (no atomics) |
|
||||
|
||||
## Connection to Golden Standard
|
||||
|
||||
This implements the architecture from `docs/src/book/multigpu_nodal_assembly.md`:
|
||||
|
||||
1. ✅ **Nodal assembly** (not element assembly)
|
||||
2. ✅ **3×3 blocks** using `Tensor{2,3}` from Tensors.jl
|
||||
3. ✅ **Matrix-free** matvec with spider pattern
|
||||
4. ✅ **Zero allocations** (immutable Tensor types)
|
||||
|
||||
**Next steps:**
|
||||
|
||||
- Implement `compute_element_contribution!()` for real elements
|
||||
- Integration with material models (already done: `compute_stress()` returns `SymmetricTensor{2,3}`)
|
||||
- GPU kernels for nodal loop
|
||||
- Contact mechanics integration
|
||||
|
||||
## Performance Implications
|
||||
|
||||
**2×2×2 Hex8 mesh (27 nodes, 81 DOFs):**
|
||||
|
||||
- **Element assembly:** 8 elements, each writes to overlapping DOFs → atomics
|
||||
- **Nodal assembly:** 27 nodes, independent writes → no atomics
|
||||
|
||||
**Spider statistics:**
|
||||
|
||||
- Corner node: 8 couplings → compute 8 × 3×3 = 72 entries
|
||||
- Interior node: 27 couplings → compute 27 × 3×3 = 243 entries (all nodes!)
|
||||
- Average node: ~12 couplings → compute 12 × 3×3 = 108 entries
|
||||
|
||||
**Memory:** No global K matrix, only local K_blocks per thread (reused).
|
||||
|
||||
## Testing
|
||||
|
||||
See `test/test_nodal_assembly_structures.jl` for working examples:
|
||||
|
||||
```bash
|
||||
cd /home/juajukka/dev/JuliaFEM.jl
|
||||
julia --project=. test/test_nodal_assembly_structures.jl
|
||||
```
|
||||
|
||||
**Tests:**
|
||||
|
||||
- ✅ Inverse mapping construction
|
||||
- ✅ Spider computation
|
||||
- ✅ Nodal contribution storage
|
||||
- ✅ Matrix-free matvec
|
||||
- ✅ Efficiency analysis (hex mesh)
|
||||
|
||||
## References
|
||||
|
||||
1. **Golden standard:** `docs/src/book/multigpu_nodal_assembly.md`
|
||||
2. **ARCHITECTURE.md:** Nodal assembly motivation
|
||||
3. **TECHNICAL_VISION.md:** Why matrix-free iterative solvers
|
||||
|
||||
## Status
|
||||
|
||||
- **Implementation:** Prototype complete ✅
|
||||
- **Testing:** Basic tests passing ✅
|
||||
- **Integration:** Not yet integrated with main JuliaFEM
|
||||
- **Performance:** Not yet benchmarked
|
||||
- **GPU:** Not yet implemented (but designed for it)
|
||||
|
||||
This is the foundation for the modern JuliaFEM architecture!
|
||||
Reference in New Issue
Block a user