caffa3d.

MB
A fully implicit, finite volume, flow solver for the incompressible 3D Navier-Stokes equations with generic scalar transport, in block structured, non-orthogonal, body fitted grids.

Gabriel Usera
1

1,2

Dep. Enginyeria Mecnica, Universitat Rovira i Virgili, Av. Pasos Catalans 26, 43007 Tarragona, Espaa <gabriel.usera@urv.net> Instituto de Mecnica de los Fluidos IMFIA, Universidad de la Repblica, J. Herrera y Reissig 565, 11300, Montevideo, Uruguay

Content 1. Introduction 2. Grid generator: grid3d.MB 3. Block pre-processor: block3d.MB 4. Flow solver: caffa3d.MB 5. Post-processing tools 6. Tutorial example 7. Acknowledgements 8. References

1. Introduction 1.1 Acknowledgement and disclaimer. This documentation describes the software package caffa3d.MB, which is an extension to 3D and block-structured grids of the original 'caffa.f' 2D code by M. Peric et al. Throughout this documentation, the user is assumed to be familiar with that code (which is freely available on the Internet), and its documentation, and with the general concepts of CFD as presented in the book by J. H. Ferziger and M. Peric, Computational Methods for Fluid Dynamics [3] (referred to in this documentation as the book). A self-contained version of this documentation is planed for the (near) future. The use, modification and/or distribution of this software are free. 1.2 General description The software caffa3d.MB is a Fortran95 implementation of a fully implicit finite volume method for solving the 3D incompressible NavierStokes equations in complex geometry. Spatial discretization is based on block-structured, non-orthogonal, body fitted, collocated grids with first order (UDS) and second order (CDS) schemes for the convective terms. For the time discretization fully implicit two-level first order (implicit backward Euler) and three-level second order schemes are available. The SIMPLE algorithm is implemented for the coupling between velocity and pressure. A generic transport equation is provided for solving transport of different scalars. In particular, heat transport is implemented together with buoyancy effects on the momentum equations. Unlike the original 2D version, only one grid level has been retained, although a future extension to multi-grid is planned. The k-e turbulent model has been retained and adapted to 3D. Also a simple Smagorinsky type LES model has been introduced. Please note however that turbulence models are not currently covered by this documentation. 1.3 Software components. The software is in fact composed of three modules: 1. Grid generator (grid3d.MB). 2. Block pre-processor (block3d.MB). 3. Flow solver (caffa3d.MB). A chapter is devoted to the description of each of these modules. Additionally, a set of Matlab post-processing tools is provided for visualisation of grid and results. 1.4 Block interfaces The following interfaces between grid blocks are currently supported: 1. 2. Exact, one-to-one interfaces. These are automatically detected and processed by the block pre-processor. Exact, many-to-one interfaces. These require user input for pairing, which is done afterwards by the block pre-processor (not currently covered in this documentation).

3.

Sliding interfaces. These are supported through user programming of appropriate handling routines (not currently covered in this documentation). Arbitrary, many-to-many, non-matching interfaces. These require user input for computing, which is done afterwards by the block preprocessor (not currently covered in this documentation).

4.

Please note that same-block interfaces are treated in the same way, so Otype, H-type, and C-type grids are supported, through the use of one or many blocks. 1.5 Input and Output The input and output files used by the software package are all formed by a six character root plus a three character extension (except for result post processing files which change the three character extension for a correlative number). The following are the conventions used for naming files: Grid input files (.ing): mygrid.ing Grid (output) files (.grd): mygrid.grd Block input file (.kin) : myname.kin Block output file (.bck) : myname.bck Solver input file (.cin) : myname.cin Solver output file (.out) : myname.out Solver restart file (.res) : myname.res Post-processing : myname.1, myname.2, ... , myname.xxx

1.6 Undocumented features A number of features are currently supported by the software package but not documented here (although they will be in future versions of this documentation). These are: Many-to-one, non-matching many-to-many, and sliding block interfaces. Turbulence models Inner boundary conditions, as thin walls placed inside the domain. OpenMP capability.

The curious user will not find it difficult to guess how to use some of these features; the code should be self-explanatory. The OpenMP capability, for example, is transparent if no openmp switch is used when compiling, and requires little more than the openmp switch to work. The code is extensively annotated and should help in this task.

2. Grid generator: grid3d.MB 2.1 Introduction The grid generator was adapted from the 2D grid generator accompanying the original caffa package (gridgen.f). As stated earlier the reader of this documentation is assumed to be familiar with the original package. grid3d.MB currently only supports the generation of 3D grids as rectilinear extrusions of a 2D grid into the third direction. Thus, it is not able to generate a spherical grid for example. Please note however that the flow solver accepts general 3D block-structured grids, so the user can provide its own grid generator. In section 2.4 the required format for the binary grid files is explained. For block-structured grids, each grid block is generated separately. Thus there will be one grid input file (.gin) for each grid block, the grid generator needs to be run once for each grid block and one grid file (.grd) will be created for each grid block. The block pre-processor must be run afterwards to compile the information about block interfaces (even if there is to be only one grid block, with no interfaces!). 2.2 Description The grid generation starts with the basic 2D section very much the same as for the original gridgen.f. Also the input for this section is almost the same, except for the boundary conditions, which are specified at the end in a separate block. Once the 2D section is generated it is 'extruded' in the 'z' direction using node distributions which are prescribed by the user in the same manner as for 2D grid edges. Finally the boundary conditions are specified in a separate input block after the geometry is defined. Boundary conditions can be specified in as many 'sections' as required. Each 'section' is composed of two lines. The first line defines the region where the given boundary condition is to be applied, by specifying the index interval. The second line simply gives the boundary condition code (just as for the 2D version: 1.-Inlet, 2.Outlet, 3.-Symmetry, 4.-Isothermal Wall, 5.-Adiabatic Wall, 10.Interface) and a numeric tag which can be used afterwards to identify the given boundary condition (very useful when programming boundary conditions, sliding interfaces, etc). Please note that if any boundary cell face appears repeated in more than one boundary condition specification, only the last one counts (this is useful to define windows of a given boundary condition within regions that were assigned previously a different BC.) Please see annotations in cavmpA.gin and/or source code for further details. 2.3 Geometry computations The 3D flow solver requires the knowledge of several geometrical properties of the grid which are computed by the grid generator. These are the volumes of each cell and the location of its centre, and for each boundary face the location of the cell face centre and the normal surface vector. All these are computed as described in section 8.6.4 of the book. (Note however that the flow solver itself currently computes some of these properties, something I intend to sort one way or the other in the future)

2.4 Grid file format In order to allow the use of other grid generators, the required format for the grid file(s) is described in this section. To do so in an expedited way the statements that write the grid file are presented and annotated next:
OPEN (UNIT=7,FILE=FILGRD,FORM='binary') REWIND 7 IUNIT=7 WRITE(IUNIT) * NI,NJ,NK,NIJK,NINL,NOUT,NSYM,NWAL,NOC, * NWALI,NWALA,NINX,NOUX,NSYX,NWAX,NOCX WRITE(IUNIT) * (LI(I),I=1,NI),(LK(K),K=1,NK) WRITE(IUNIT) * (IJI(I) ,I=1,NINX),(IJPI(I) ,I=1,NINX), * (IJI1(I) ,I=1,NINX),(IJI2(I) ,I=1,NINX), * (IJI3(I) ,I=1,NINX),(IJI4(I) ,I=1,NINX), * (ITAGI(I),I=1,NINX) WRITE(IUNIT) * (IJO(I) ,I=1,NOUX),(IJPO(I) ,I=1,NOUX), * (IJO1(I) ,I=1,NOUX),(IJO2(I) ,I=1,NOUX), * (IJO3(I) ,I=1,NOUX),(IJO4(I) ,I=1,NOUX), * (ITAGO(I),I=1,NOUX) WRITE(IUNIT) * (IJW(I) ,I=1,NWAX),(IJPW(I) ,I=1,NWAX), * (IJW1(I) ,I=1,NWAX),(IJW2(I) ,I=1,NWAX), * (IJW3(I) ,I=1,NWAX),(IJW4(I) ,I=1,NWAX), * (ITAGW(I),I=1,NWAX) WRITE(IUNIT) * (IJS(I) ,I=1,NSYX),(IJPS(I) ,I=1,NSYX), * (IJS1(I) ,I=1,NSYX),(IJS2(I) ,I=1,NSYX), * (IJS3(I) ,I=1,NSYX),(IJS4(I) ,I=1,NSYX), * (ITAGS(I),I=1,NSYX) WRITE(IUNIT) * (IJL(I) ,I=1,NOCX),(IJR(I) ,I=1,NOCX), * (IJOC1(I),I=1,NOCX),(IJOC2(I),I=1,NOCX), * (IJOC3(I),I=1,NOCX),(IJOC4(I),I=1,NOCX), * (ITAGOC(I),I=1,NOCX) WRITE(IUNIT) * (X(I) ,I=1,NIJK),(Y(I) ,I=1,NIJK),(Z(I) ,I=1,NIJK), * (XC(I),I=1,NIJK),(YC(I),I=1,NIJK),(ZC(I),I=1,NIJK), * (FX(I),I=1,NIJK),(FY(I),I=1,NIJK),(FZ(I),I=1,NIJK), * (VOL(I),I=1,NIJK), * (SRDW(I),I=1,NWAX), * (XNW(I),I=1,NWAX),(YNW(I),I=1,NWAX),(ZNW(I),I=1,NWAX), * (SRDS(I),I=1,NSYX), * (XNS(I),I=1,NSYX),(YNS(I),I=1,NSYX),(ZNS(I),I=1,NSYX), * (FOC(I),I=1,NOCX) CLOSE(UNIT=7)

01. NI,NJ,NK... Number of nodes in I, J and K directions 02. NIJK... Total number of nodes. NIJK=NI*NJ*NK 03. NINL... Num. of boundary CV faces assigned with Inlet BC. 04. NOUT... Num. of boundary CV faces assigned with Outlet BC. 05. NSYM... Num. of boundary CV faces assigned with Symmetry BC. 06. NWAL... Num. of boundary CV faces assigned with Wall BC.

07. NOC ... Num. of boundary CV faces implied in O-C cuts. 08. NWALI... Num. of boundary CV faces assigned with Isothermal Wall BC. 09. NWALA... Num. of boundary CV faces assigned with Adiabatic Wall BC. (Please note that NWAL=NWALI+NWALA) 10. NINX, NOUX, NSYX, NWAX, NOCX... Used to prevent arrays of null length: NINX=MAX(NINL,1), NOUX=MAX(NOUT,1), NSYX=MAX(NSYM,1), NWAX=MAX(NWAL,1), NOCX=MAX(NOC ,1) #################### 11. LI(I)... Offset indices for I columns. LI(I)=(I-1)*NJ 12. LK(K)... Offset indices for K columns. LK(K)=(K-1)*NIJ #################### Then we have the blocks for BC. specifications. There is one block for each BC.: Inlet, Outlet, Symmetry, Wall, OC-cuts or Interfaces. They are all similar. The block for Inlet goes like this: 13. IJI(I)... Integer array. IJK indexes to each cell face assigned as Inlet 14. IJPI(I)... Idem. for the interior node neighbour to the boundary face 15. IJI1(I)...IJI4(I)... Arrays with the IJK indexes to each corner of the boundary face given CLOCKWISE ( so that the normal will point outwards ). 16. ITAGI(I)... Array with the numeric boundary identification tag #################### 17. X(I),Y(I),Z(I)... Co-ordinates of grid corners 18. XC(I),YC(I),ZC(I)... Coord. of cell centres 19. FX(I), FY(I), FZ(I)... Interpolation factors 20. VOL(I)... Volume of each cell 21. SRDW(I), XNW(I), YNW(I), ZNW(I)... Info about wall BC. faces (see code) 22. SRDS(I), XNS(I), YNS(I), ZNS(I)... Info about symmetry BC. faces (see code)

3. Block pre-processor Block3d.MB is a new part of the software package, in the sense that there was not an equivalent module in the original 2D package caffa (no need for one). This module assembles the grid blocks produced by grid3d.MB, building the information structure required by the flow solver caffa3d.MB for handling block-structured grids. In particular, the interfaces between blocks are matched and listed with the required information for the flow solver. The block pre-processor must be run for all problem set-ups, even if it is a one-block grid problem. For one-to-one interfaces no additional input is required from the user other than the number of grid blocks and the list of file names corresponding to the grid files (.grd) for each block. Thus, the block input file (.kin) myname.kin must have the number of grid blocks in the first line, and then one line for each grid block giving the corresponding file name (6 characters the extension .grd is assumed). For example: 3 blockA blockB blockC 0 0

The two 0s in the last lines indicate that there is no further input. (The opposite would be the case for many-to-one and non-matching many-tomany interfaces, but those are not currently covered by this documentation). A block information file (.bck) 'myname.bck' is generated. The flow solver will then read this file first, and after that it will read each grid block file (the geometry info for each block remains in the original .grd file, the .bck file only contains indexing and connectivity information). Also the file param3d.inc is generated with appropriate dimension parameters for compiling the flow solver caffa3d.MB. This file should be copied to the folder containing the source files before compiling caffa3d.MB. The process of matching one-to-one interfaces runs automatically, with no user input and no screen output. However it will fail if the grid blocks don not actually match (due to incorrect geometry or BC. specification of one or more blocks). In this case the failing boundary faces will be listed and no block information file .bck will be generated.

4. Flow solver The use of the flow solver is little changed from the original 2D version. Most of the changes, from a user point of view, are in the grid generation and the (new) block assembling step. The input file for the flow solver (.cin) has only minor changes. Since the multiple grid levels have been dropped for the time being, so was the KIN parameter. Also the LSG array (number of outer iterations, 'in each grid') is now a scalar. An extra logic parameter LGRAD has been added which controls the output of gradient fields for post-processing. Other than that some inputs have been re-arranged in different lines. Of course there is an extra equation (W momentum equation) which was given number 3 (so now pressure has number 4 and temperature number 5). Equations 6 to 9 are for the turbulence models, but those are not covered in this documentation. At the end of the file, some information about the grid blocks is added. The number of blocks, and the file name ('.grd' assumed) for each grid block file must be listed (just as for the '.kin' file for the block preprocessor, please see the previous section). Again, a final 0 after the last line is required (in this case to indicate that no inner walls are to be considered; this feature will be documented in future versions). As for the solver, an option of linear equation solver has been added. In addition to the SIP solver (now in 3D version), a CGSTAB solver has been adapted from the code library that comes with the book. The choice of linear solver is made at compilation time with the appropriate -I flag. Please see the tutorial section. The SIP solver has been adapted for fully implicit treatment of the interface terms between blocks, as described in section 8.6.5 of the book. The CGSTAB solver however, currently only supports a semi-explicit treatment of these terms within the inner iterations. Thus, care is advised when solving multi-block domains with the CGSTAB solver. The user may prefer to stick to the SIP solver in these cases. Still, CGSTAB can be used safely for single block domains, where it may give some performance advantage over the SIP solver, especially in very fine grids. Other than the screen output, the solver produces the following files: myname.out: Convergence and monitor point output. myname.xxx: Results for post-processing. myname.res: Restart file for use in following computations.

5. Post-processing tools A set of Matlab scripts and functions is provided for post-processing of the results, as well as grid visualisation previous to flow solving. 5.1 Grid visualisation The Matlab script checkGrid.m reads a given grid output file (.grd) and allows the user to visualise the grid prior to flow calculation for verification or debugging purposes. The user only needs to provide the correct filename (and path if necessary) in the first line of the script file: GridFileName=['mygrid.grd']; After execution all the grid information is loaded into the Matlab workspace. The grid boundary surfaces can be visualised directly by using the provided gridsurf.m function: h=gridsurf(XC,YC,ZC,red,gcf);

5.2 Results visualisation The Matlab script caffa3dMBs.m reads a given caffa3d.MB post processing file (myname.xxx) and loads into the Matlab workspace the selected grid and flow fields. The user needs to provide the correct filename (and path if necessary) in the first line: Filnam=['myname.1']; Additionally the user may want to comment the lines that correspond to those flow fields not wanted. Grid data is loaded into the structure GR with one level (GR(i)) for each grid block, and fields for each grid field (thus GR(3).XC is the array with the x-coordinate of the cell centres in the 3rd grid block ). Analogously, the structure FL keeps the flow fields (i.e. FL(2).UU is the array with the U-component of the velocity in the 2nd block). Please see the tutorial section (next section) for working examples on the visualisation of results.

6. Tutorial example This section is intended to guide the user through a first example and test case, which also serves as a validation case. The set-up, compilation, grid generation, block pre-processing, and flow solving steps will be explained. The test case selected is the 3D lid driven cavity at Re=1000. A uniform grid with 81x81x81 cells is used (5.3x105 cells). This problem runs in about 197 MB of RAM and requires, to reach steady state, about 1hr in an AMD Athlon(tm) XP 2800+ CPU. The output and results files require about 500 MB of disk space. Results have been compared successfully with those from other authors ([1], [2], please see below). To demonstrate the set-up of multi-block grids, the domain has been splited in two along a vertical plane (not exactly the middle plane, the left block keeps 39 cells while the right block keeps the other 42 cells). There are then two blocks, matched through a one-to-one interface. The tutorial is intended for a Linux (or Unix) system. Also the free noncommercial version of Intel Fortran Compiler v 7.1 is used (appropriate compiler flags may differ for other versions or compiler brands). For the visualisation examples a graphics enabled system with Matlab 6.x or later is required. The setup files for this example are included in the distribution, in the example folder 6.1 Unpacking the distribution Copy the tar.gz file caffa3d.MB.v.1.01.tar.gz to an empty folder that will host the caffa3d.MB package (assume this folder is named caffa3d). Decompress and extract the package with:
caffa3d $ gunzip caffa3d.MB.v.1.01.tar.gz caffa3d $ tar xvf caffa3d.MB.v.1.01.tar ...tar output... caffa3d $

The following files and folder structure is created as a result:

caffa3d $ ls block3d.MB caffa3d.MB documentation example

caffa3d.MB.readme grid3d.MB

caffa3d.MB.v.1.01.tar

6.2 Compiling grid3d.MB and block3d.MB The files param3d.ing and param3d.inb control de dimension parameters for grid3d.MB and block3d.MB respectively. The current parameters are generous, but for very large problems the user may need to increase them. To compile grid3d.MB change to the corresponding folder and call the compiler:

caffa3d $ cd grid3d.MB grid3d.MB $ ifc o ../example/grid3d.MB.lnx ./grid3d.MB.f ...compiler output... 2095 Lines Compiled grid3d.MB $

Analogously, to compile block3d.MB:

grid3d.MB $ cd ../block3d.MB block3d.MB $ ifc o ../example/block3d.MB.lnx ./block3d.MB.f ...compiler output... 1205 Lines Compiled block3d.MB $

The example folder should now have both .lnx executables (in addition to the distribution provided set-up files):
block3d.MB $ cd ../example example $ ls

block3d.MB.lnx cavmp1.cin cavmpA.gin checkGridMB.m gridsurf.m caffa3dMBs.m cavmp1.kin cavmpB.gin grid3d.MB.lnx

6.3 Grid generation Since the grid for this problem is made up from two blocks, we need to run the grid generator twice in the example folder:
example $ ./grid3d.MB.lnx ENTER PROBLEM NAME (SIX CHARACTERS): cavmpA ...output... example $ ./grid3d.MB.lnx ENTER PROBLEM NAME (SIX CHARACTERS): cavmpB ...output... example $

As a result, the grid files cavmpA.grd and cavmpB.grd are created in the example folder. To check that grid blocks were correctly generated, the following Matlab commands might be issued in a graphics enabled system:
example $ matlab -nodesktop nosplash < M A T L A B > Using Toolbox Path Cache. Type "help toolbox_path_cache"... To get started, type one of these: helpwin, helpdesk, or demo.

For product information, visit www.mathworks.com. >> >> >> >> >> >> >> >> GridFileName='cavmpA.grd'; checkGridMB; h=gridsurf(XC,YC,ZC,'red',gcf); hold on; GridFileName='cavmpB.grd'; checkGridMB; h=gridsurf(XC,YC,ZC,'blue',gcf); view(3)

A Matlab graphics window should appear with an image like this one:

Please quit Matlab to continue the tutorial.

>> quit example $

6.4 Block pre-processing To run the block pre-processor :

example $ ./block3d.MB.lnx ENTER PROBLEM NAME (SIX CHARACTERS): cavmp1 example $

No screen output is produced, unless an error occurs. Two new files are created in the example folder : cavmp1.bck and param3d.inc.

6.5 Compiling caffa3d.MB The parameter file param3d.inc created by block3d.MB should be copied/moved to the caffa3d.MB folder before compiling. Afterwards, the compiler is called:
example $ mv ./param3d.inc ../caffa3d.MB/param3d.inc example $ cd ../caffa3d.MB caffa3d.MB $ ifc o ../example/caffa3d.MB.lnx ./caffa3d.MB.f IsolverSIP ...compiler output... 11592 Lines Compiled example $

6.6 Running the flow solver To run the flow solver the following command needs to be issued:
caffa3d.MB $ cd ../example example $ ./caffa3d.MB.lnx ENTER PROBLEM NAME (SIX CHARACTERS): cavmp1 11 2.9E-02 1.8E-02 6.9E-04 1.9E-02 0.0E+00 .... 11 1.5E-02 1.0E-02 6.8E-04 1.7E-02 0.0E+00 .... ....

Please note that the first line of output takes about 1 min to appear in the CPU referred above for the given setup. Output from the solver is better described in the .out file (cavmp1.out) which can be checked during the computation. 6.7 Results visualisation The provided Matlab script may be used to visualise the results. As provided, the script is configured to load the last post-processing output in the current example (cavmp1.10, see first line in Caffa3dMBs.m). The following Matlab commands may be issued in a graphics enabled system:
example $ matlab -nodesktop nosplash < M A T L A B > Using Toolbox Path Cache. Type "help toolbox_path_cache"... To get started, type one of these: helpwin, helpdesk, or demo. For product information, visit www.mathworks.com. >> >> >> >> >> >> >> filnam=cavmp1.10; caffa3dMBs; figure; hold on; grid on; plot(GR(1).XC(42,:,42)*2-1,FL(1).VV(42,:,42),'b','LineWidth',2) plot(GR(2).XC(42,:,42)*2-1,FL(2).VV(42,:,42),'b','LineWidth',2) plot(FL(2).UU(:,3,42),GR(2).YC(:,3,42)*2-1,'b','LineWidth',2) axis square; axis([-1,1,-1,1]);

A Matlab graphics window with a plot similar to the following one should appear. Please note that, in the figure featured here, results from other authors as well as for other grid resolutions are also presented for comparison (the Matlab window should only show a pair of blue U and V profiles).

7. Acknowledgements Gabriel Usera currently holds a research and teaching position at the Universidad de la Repblica (Uruguay). The caffa3d.MB software package (and this documentation) was wrote during a research visit in the Universitat Rovira i Virgili (Catalunya), where Gabriel Usera currently pursues his PhD studies. The co-operative support from both Universities has been invaluable. Additionally, this work was financially supported by grants 2002 SGR00189 from AGAUR Generalitat de Catalunya, DPI2003-06725-C02-01 from DGI, Ministerio de Ciencia y Tecnologa y Fondos FEDER, assigned to the ECOMMFIT research group of the Univeritat Rovira i Virgili. Gabriel Usera also received the support of pre doctoral grants from Ministerio de Ciencia y Tecnologa (Spain) and Programa de Desarrollo Tecnolgico (Uruguay), as well as a financial aid for research visits granted by the Universidad de la Repblica granted through the CSIC RRHH program. Finally the authorisation of Dr. Milovan Peric for the use of his caffa code as a basis for the caffa3d.MB package was greatly appreciated. 8. References [1] Iwatsu R., Hyun J.M., Kuwahara K., Analyses of three-dimensional flow calculations in a driven cavity Fluid dynamics research, 6(2):91102,1990. [2] Jimnez A., Interacci entre les estructures de flux i el transport de materia. Aplicaci a un doll pla turbulent PhD Thesis, Universitat Rovira i Virgili, 2003 [3] Ferziger J., Peric M., Computational methods for fluid dynamics Springer-Verlag, 2002