Multi-Resolution Climate

Modeling
Principle Investigators
F. Baer (UMCP)
J. Tribbia(NCAR)
A. Fournier (UMCP)

Collaborators
Mark Taylor (Los Alamos)
Steve Thomas (NCAR)
Rich Loft (NCAR)
M. Fox-Rabinovitz (Fac. Assoc.)
 Goal of the Project

• To develop a climate model

methodology which allows for
seamless concurrent integration of the
planetary scales together with regional
scales;
• To optimize the capabilities of
currently available computer hardware
(parallel processors) with this model;
• To incorporate the best features of
current state-of the-art climate models.

2
 Rationale
• Variable resolution in a model helps
define climate;
• Nonlinear effects introduced by
regional scales must be incorporated
into a climate;
• Smaller scale effects often grow on
shorter time scales;
• Identification and prediction of
regional climate should help in
understanding the evolution of the
global climate.
• Integrations must be sped up to
perform all computations needed for
solving the climate modeling problem.
3
 The Method and Model

SEAM: Spectral Element

Atmospheric Model
• A global model offering great
flexibility and advantages in:
– Using geometric properties of finite
element methods;
– Incorporating local mesh refinement and
regional detail;
– Utilization of parallel processing;
– Maintaining the accuracy of spectral
models;
– Computational efficiency;
– Having no pole problem.
4
 Plans for the Model

• Use as the dynamical core with

a forced model like the CCM4;
• Consider the finite-volume
technique;
• Use semi-implicit integration
(currently under study);
• Consider semi-Lagrange
integration;
• Create an option for linking to
an OGCM.

5
 Model Domain

• Tile spherical surface with arbitrary

number and size of rectangular
elements;
– Inscribe a polyhedron with rectangular
faces inside sphere,
– Map surface of polyhedron to surface of
sphere with a gnomonic projection,
– Use the cube (most elementary
polyhedron),
– Subdivide each of the six faces of the
cube as desired.
– Can use Local Mesh Refinement (LMR)
as desired.
6
Cube Subdivision 1

Uniform Resolution Rectangles

An example of Local Mesh-refinement
on the globe.
 Application of the method

Use the shallow water equations as an example:

V 1 
   f k  V -  V  V +gH 
t 2 
h
   hV
t
H = h +hs   k    V f  2sin 

 Generate an Integral form of the equations;

 Multiply by a global test function () and
integrate over the entire domain (spherical
surface).
V  1 
A t dA   A   f k  V 2 V  V +gH  dA
h
A t dA   A   hVdA
 spectral element discretization
• Within each rectangle, use Legendre cardinal
functions as basis functions ( k, j ) in each direction.

 k, j  k (x) j (y)
h(x, y)   hk , j k (x) j (y)
k, j

V(x,y)   Vk, j k (x) j (y)

k, j

 k (x i )  Legendre Cardinal functions = k ,i

xi  Gauss - Lobatto Points

 spectral element discretization

•The Gauss- Lobatto quadrature points

distribute as follows on the local rectangles:

•Within each rectangle, estimate the integral

equations by Gauss- Lobatto quadrature .
 Global Test Functions

•Simple combinations of Legendre cardinal functions;

•One global test function for each grid point.

Element interior points

Element boundary points

 Example of the computational system

The system of integral equations,

h
A t dA     hVdA
A

Reduces to:

A set for element interior points:

h
J (x i )  J  hV
t
And a set for element boundary points:
h
J  J  t xi    hVJ   hVJ 
   

where J is the area integration weight.

 In Summary

• Tile the sphere with rectangles of

arbitrary number and size;
• Represent the prediction equations
in integral form;
• Use Gauss-Lobatto quadrature for
integration;
• Use Legendre cardinal functions for
the basis functions;
• Use test functions based on the
Legendre cardinal functions.

These choices result in an extremely

simple finite element method with a
diagonal mass matrix.
14
 Does the model work?

•Tests with the shallow water

equations:
–With test suite (Williamson et al.,
1992);
–Efficiency;
–With local mesh refinement;
–With semi-implicit integration;
–With high resolution turbulence
problem;
–With vortex studies.
•Tests with a 3-D PE dynamical
core and Held-Suarez forcing.

15
 Shallow water test case 7
•ICs taken from 500 mb observed data;
•Compare spectral element model to NCAR
spectral, CSU twisted icosohedral grid, and A-L
grid point models.
•L2 is normalized error between computed and
true height field at end of run.
 Shallow water test case 7

Efficiency
•Curves show increasing number of elements
(M) for fixed spectral degree (N);
•Increasing N is more costly than increasing M;
•N = 8 appears to be the most efficient index.
•Select M for desired resolution.
 488x8x8

Mesh
Refine
1784x8x8

Global
6936x8x8

•Test case 5
•Ht. field 150x8x8

210x8x8

Local

310x8x8
Mesh Refine 488x8x8

Zoom

1784x8x8

Global

6936x8x8

150x8x8

210x8x8

Local

310x8x8
 Shallow water test case 6
•Errors comparing the explicit and semi-
implicit integrations of SEAM.

Semi-implicit

(Thomas and Loft)

Explicit
Linf

(Taylor)
 Shallow water test case 2

•Ratio of semi-implicit to explicit

integrations of SEAM with the shallow
water equations for various truncations:
C refers to number of elements;
N refers to spectral degree.
•Note that semi-implicit is somewhat faster as
well as more accurate.

Thomas and Loft

 Shallow water equations / Jupiter

• A study of decaying turbulence with high

resolution using SEAM;
• Equivalent depth on Jupiter = 20000 m;
• Use very weak dissipation;
• T170, T360, T533, T1033 runs on a CRAY T3E
with 128 processors;
• T1033 has 60000 elements (8x8) ~ 3000
equatorial pts.;
• Potential vorticity at 276 Jupiter days, T1033.
 Shallow water equations / Jupiter

•Zonal wind at 700 Jovian days (left column);

•Equatorial jet strength in time (right column).
•Both demonstrate the effect of resolution.

Zonal Wind Equatorial Jet Strength

0

100

T170
200

100

T1033
200
0 500 1000

Latitude Jovian days

 Dynamical Core/SEAM/LMR

Local mesh-refine Andes topography

Global

Zoom
 Dynamical Core/SEAM
•Held-Suarez forcing; 384x8x8 ~ T85/L20 SEAM;
• Lat. vs Ht. zonal mean wind and eddy variance of T;
•Compare T63, G72 and SEAM.
T*2 (K2) U (m/s)

T63

T*2 (K2)

G72

Latitude Latitude

SEAM
 Dynamical Core/SEAM
•Held-Suarez forcing;
•SEAM with uniform grid;
•Scaling results for various resolutions; almost
insensitive to processor number change.

HP Exemplar SPP2000

320km/L20
160km/L20
80km/L20

320km/L20 (dotted)
160km/L20 (solid)
80km/L20 (dashed)
 Dynamical Core/SEAM

•Parallel scaling on various computers

•Triangles denote SEAM
•Horizontal resolution-T181, (g)seaborg- T533
•Other symbols for other models
•Log-log plot, flops vs processors.

Mflops per processor Gflops

# of processors
 The Research Plan

• First and foremost, incorporate

SEAM as a dynamical core into
CCM4.
• Experiments to be undertaken
concurrently with this development:
– Apply the H-S forced version of
SEAM (already running) to
meaningful simulations such as
turbulence decay;
– Develop algorithmic refinements of
the 3-D simple forced model to
include semi-implicit integration.
This is currently under study by our
collaborators Thomas and Loft and a
test version is running.

28
 Research Plan, cont.
• Mesh refinement studies:
– Test effectiveness of LMR using 3-D
PE version of SEAM with H-S test case
forcing model including real
topography;
– Test a suite of long integrations at
various uniform resolutions;
– Test high resolution over the Andes and
Himalayas with lower uniform
resolution elsewhere;
– Undertake comparison studies of LMR
with the stretched grid model of Fox-
Rabinovitz (already begun);
– Undertake the same comparison with a
global model which uses an imbedded
fine-scale model (say MM5) if
available.
29
 Research Plan, cont.
• Vertical representation:
– Currently we use -coordinates; we will consider
the use of the finite volume method together with
spectral elements in the horizontal.
• Time extrapolation:
– The model currently uses explicit integration;
– We will study the feasibility of using semi-
Lagrange integration;
– The semi-implicit scheme is currently under study;
it appears to be between 2 to 3 times faster than
the explicit scheme for the SWE.
• Time filtering;
– We have developed a time filtering scheme which
we will test with SEAM.
• Storm tracks with LMR:
– We will put LMR into SEAM in the vicinity of
quasi-stationary storm tracks to test sensitivity.

30
 Research Plan, cont.

Studies with SEAM and full physics

(CCM4)
• Using this model, repeat tests described
above with the simple forcing model and
compare to those results;
• Space resolution:
– Test quality of predictions against the
standard CCM4 with various resolutions on
a uniform grid.
• Apply LMR:
– One region at a time;
– Multiple regions;
– Consider LMR use in tropics for tropical
wave evolution;
– Compare LMR integrations with other
models using LMR, such as Fox-Rabinovitz’
stretched grid.

31
 Research Plan, cont.

• Time resolution:
– We will determine the effect of SEAM
methodology on the climate period
selected; i.e., seasonal vs. decadal, etc.
– We will consider this effect using LMR.
• Computational efficiency:
– We will attempt to run the various
experiments on machines with largest
number of processors to determine
scaling.

32
• Interactions:
– NCAR
• CCM4 staff
• SCD staff
• Scientists associated with the project
• Computers
– NERSC
• Computers
• Support staff
– UMCP
• Scientists and students associated
with the project
• Stretched-grid development group.
• Staff:
– PIs: Baer, Tribbia, Fournier
– Co-Investigator: Taylor
– Faculty Affiliate: Fox-Rabinovitz
– Collaborators: Thomas, Loft
33
The End

34