See

discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/317935582

Mesh generation for HPC problems: the potential of

SnappyHexMesh

Presentation June 2017

DOI: 10.13140/RG.2.2.25007.53923

CITATIONS READS

0 21

1 author:

Andrea Montorfano
Politecnico di Milano
45 PUBLICATIONS 173 CITATIONS

SEE PROFILE

Some of the authors of this publication are also working on these related projects:

dynamic VOF simulation of GDI injector opening View project

Developments in transient modeling, moving mesh, turbulence and multiphase methodologies in OpenFOAM View project

All content following this page was uploaded by Andrea Montorfano on 27 June 2017.

The user has requested enhancement of the downloaded file.

Mesh generation for HPC problems:
the potential of snappyHexMesh

Dr. Andrea Montorfano

dip. di Energia, Politecnico di Milano
andrea.montorfano@polimi.it
Motivation
CFD work-flow
Pre-processing
Physical Geometrical Geometrical
MESH
problem model (CAD) model (CFD)

Solver setup

solution

Data
processing

Is solution
END
acceptable?

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

Motivation

CFD Vision 2030:

Physical Geometrical Geometrical
MESH
problem model (CAD) model (CFD)

Solver setup

solution

Data
processing

Is solution
END acceptable
?

A single engineer/scientist must be able to conceive, create, analyze, and

interpret a large ensemble of related simulations in a time-critical period
(e.g. 24 hours), without individually managing each simulation.

Source: NASA Contractor Report CR2014-218178 (2014)

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

Motivation
Mesh generation and adaptivity continue to be significant bottlenecks in
the CFD workflow

NASA should develop, fund, and sustain a base research and technology
(R&T) development program for simulation-based analysis and design
technologies:
1. High Performance Computing (HPC)
2. Physical Modeling
3. Numerical Algorithms
4. Geometry and Grid Generation
5. Knowledge Extraction
6. MDAO

Source: NASA Contractor Report CR2014-218178 (2014)

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

Motivation

Panel Discussion at AIAA Aviation 2014

(participants poll)

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

snappyHexMesh

First version released with OpenFOAM-1.5 (2008)

quite a low-profile announcement for the main non-

commercial meshing tool used along with OpenFOAM!

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

Working principle
The Three Stages of SHM

0. A rigorously hexahedral base mesh is generated with an

external tool (e.g. blockMesh)

1. Mesh is refined according to surface intersection and

non-fluid regions are removed

2. Points are snapped onto the surface

3. Hex layers are extruded on specified solid

surfaces

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

Working principle

SHM is based on the octree-refinement algorithm

Cells are refined based on:
Intersection with model surface
Intersection with model edges (aka feature edges)
Distance from surface
Refinement boxes/spheres/cylinders/etc

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

Working principle
Each stage is iterative (until quality constraints are met)
Each stage can be run separately

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

Working principle
The whole process is fully automatic
once the user has
Defined the input geometry
Properly compiled the input dictionary

These actually represent the hard points in the workflow

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

Input geometry

SHM reads:
Pre-defined OpenFOAM primitives (box, sphere, cylinder, etc.)
Triangulated surface formats (STL, OBJ)

The geometry file has to be:

Closed (no holes, crevices)
Clean (no illegal faces)
Properly subdivided (each surface in the geometry file will turn into an
OpenFOAM boundary patch)

Usually the file supplied by the CAD guy cannot be given as-it-is to
SHM

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

Input geometry

Solution #1: Talk with the CAD guy

Solution #2: Use OpenFOAM utilities for

surface manipulation

Solution #3: Use a non-commercial tool

(e.g. SALOME)

Solution #4: use a commercial grid-

generator to clean your STL

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

Input dictionary

All parameters needed for the execution are specified in the

snappyHexMeshDict file
example file has 149 lines (not counting comments, of course)
and with case complexity the dictionary length increases as well
but most parameters can be found once for all, for a given case family!
The lack of a GUI allows an easy coupling with automation
scripting and the deployment of SHM in an HPC context

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

snappyHexMesh in parallel
The three stages can be executed in parallel on several subdomains

Communication is needed to sync points on processor interfaces

Cell number can grow differently among processors

need for dynamic rebalancing

The decomposed mesh is ready to be run, without the need to reconstruct it

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

snappyHexMesh in parallel
Dynamic load balancing is done automatically by ptscotch

Load balancing is time-consuming. Need for a trade-off between

slowdown due to imbalance and rebalancing overhead
A. Montorfano, mesh generation for HPC problem with snappyHexMesh
What you need
An OpenFOAM installation
MPI environment
ptscotch support in OpenFOAM
A good amount of memory: SHM uses a lot of RAM during its operation. A rough guide
is 4 GB per million cells.
Good-level I/O
A remote visualization facility (to debug your meshing process) operating in
client/server mode

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

SHM workflow
Time to prepare the case is
negligible
Castellated mesh phase:
needs few iterations but causes
huge increase in cell count

increasing setup difficulty

increasing wall-clock time
(rebalancing overhead)
always converges
Snap phase:
needs more iterations, does not
increase number of cells
usually converges, some spots
may need attention
Add layer phase
needs still more iterations
adds significant number of cells
does not always converge

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

SHM workflow
Looks like SHM is an user-unfriendly, difficult-to-setup, resource-demanding
tool, however it is used by a large part of the OF community:
considerable cells savings with respect to purely-tet mesh: up to 40% less
elements, due only to the element shape (cut hex vs tet)
Cut-hexes are better handled by OF solvers than tets (better accuracy)
completely automatic run
works in parallel on any number of processors, by default
no license cost
The real bottleneck is preparing the dictionary (requires some iterations),
however:
The same dictionary can be used for different cases of the same family
A large part of the dictionary can be used even for different case families
The really discriminating factor is the users expertise (open-source is not free!)

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

SHM in industrial contexts #1
Scenario: parameter study
find Cx and Cz of vehicle as a function of wing pitch

input: car snappyHexMesh

geometry@ (in parallel)

simpleFoam
(in parallel)

Cx, postProcessing
Cz (in parallel)

repeat

On an HPC cluster the user can run N cases, with Np processor each, at the same
time

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

SHM in industrial contexts #2
Scenario: shape optimization
Optimize aerodynamic surface to have max(Cz), with constraints C1CN

input: parametric
start surface profile:
STL generator
(or morpher)
snappyHexMesh
(in parallel)
(x, y, z) = f(p1pn)

p1.pn simpleFoam
(in parallel)

C1CN

optimization
end driver
Cx
Cz
postProcessing
(in parallel)
(e.g. DAKOTA)

The driver can solve multi-parameter optimization running several cases at the
same time

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

Summary
The combination of:
high parallelism
no license cost
automatic mesh generation
easily interfacing with external scripts/drivers
makes snappyHexMesh a handy instrument for large-scale simulations in HPC
environments
The setup phase requires highly skilled operators
The production phase allows you to fire and forget

A single engineer/scientist must be

able to conceive, create, analyze,
and interpret a large ensemble of
related simulations in a time-
critical period (e.g. 24 hours),
without individually managing
each simulation.

A. Montorfano, mesh generation for HPC problem with snappyHexMesh

View publication stats

Thank
you!

Contact:
Andrea Montorfano, PhD
Dip. di Energia, Politecnico di Milano
+39 02 2399 3804
andrea.montorfano@polimi.it