R

iley Cumberland is a senior in Nuclear Engineering at North Carolina State University. His hometown is Asheville, NC. Rileys research was completed as an intern during the summer of 2008 at the Idaho National Laboratory under the Science Undergraduate Laboratory Internship program. Riley is currently focusing on nishing his undergraduate courses. His interests involve amateur radio, electronics and the outdoors. Riley also likes repairing broken things. eorge Mesina received a Ph.D. in Applied Mathematics from the University of Pittsburgh in 1988. He then joined the Idaho National Laboratory (INL) and works in computational uid dynamics and thermal hydraulics and has over 30 publications in these areas. In 1993, he joined the RELAP5 Development Team and invented the BPLU algorithm for vector-parallel solution of linear equations and implemented it in RELAP5; it allowed RELAP5 to run 5 times faster on vector-parallel supercomputers on the AP600 nuclear power plant benchmark. In 1995, Dr. Mesina won a Small Business Innovative Research grant with Computer Simulation and Analysis

for distributed computing via the Parallel Virtual Machine protocol. From 19961997 he was PI for the Real-time RELAP5 Cooperative Research and Development Agreement that created the RELAP5-RT simulator version, licensed in 1997 and 1999, that is still being sold for use on nuclear power plant training simulators. From 19982000 he was PI and lead developer of the RELAP5 Graphical User Interface project that automatically generates a 3D rendering of a RELAP5 input model. Since then George has become code architect of RELAP5-3D. He has also been PI and lead developer of numerous projects including: RELAP5-3D parallelization for share-memory multiprocessors, RELAP5-3D code restructuring, and the 4.5 year FORTRAN 95 conversion project. Apart from his work with RELAP5, Dr. Mesina has chaired numerous conference sessions, served on many INL committees and is currently an INL Subject Matter Expert for Mathematics and the FORTRAN language. He has developed algorithms and implemented them as programs including: a FORTRAN statistics program for the Probability Risk Assessment group and a C++ program for doing least squares curve ts through sensor data for the INL Advanced Test Reactor.

IMPROVEMENTS TO THE TIME STEPPING ALGORITHM OF RELAP5-3D

RILEY CUMBERLAND AND GEORGE MESINA
ABSTRACT
The RELAP5-3D time step method is used to perform thermo-hydraulic and neutronic simulations of nuclear reactors and other devices. It discretizes time and space by numerically solving several differential equations. Previously, time step size was controlled by halving or doubling the size of a previous time step. This process caused the code to run slower than it potentially could. In this research project, the RELAP5-3D time step method was modied to allow a new method of changing time steps to improve execution speed and to control error. The new RELAP5-3D time step method being studied involves making the time step proportional to the material courant limit (MCL), while insuring that the time step does not increase by more than a factor of two between advancements. As before, if a step fails or mass error is excessive, the time step is cut in half. To examine performance of the new method, a measure of run time and a measure of error were plotted against a changing MCL proportionality constant (m) in seven test cases. The removal of the upper time step limit produced a small increase in error, but a large decrease in execution time. The best value of m was found to be 0.9. The new algorithm is capable of producing a signicant increase in execution speed, with a relatively small increase in mass error. The improvements made are now under consideration for inclusion as a special option in the RELAP5-3D production code.

INTRODUCTION Time stepping is an important component of numerical models. Choosing the appropriate time step is necessary to minimize both the execution time of the code and the error of the model. The purpose of this research project is to study the usage of a time step that is proportional to the material courant limit (MCL) of the model, and to consider this method of time step selection as a possible improvement to RELAP5-3D/Version 2.4.1.2 [1]. RELAP5 is a reactor multiphysics code that models both neutronics and thermo-hydraulics. It is used by engineers around the world to model these phenomena. Additionally, it is used at reactor facilities in the United States to run accident scenarios in training simulations. It was created at the Idaho National Laboratory (INL). 60

The code employs several mathematical models. For the uids of the reactor, it uses a nite difference approximation to solve the six equation model of two-phase uid ow. This model consists of a set of six partial differential governing equations that describe: liquid mass, vapor mass, liquid momentum, vapor momentum, liquid energy, and vapor energy. Appropriate closure relations are used to solve for additional variables. The code uses a nite difference method for multi-dimensional heat transfer equations and an analytical nodal method for neutron kinetics. Both semiimplicit and nearly-implicit nite difference time stepping schemes are available [1]. The focus of this work is on the former scheme, semi-implicit. Additionally, RELAP5 has models for both plant trips and controls [1]. It is used in reactor design, licensing and operator training simulators.

U.S. Department of Energy Journal of Undergraduate Research http://www.scied.science.doe.gov

The time step in RELAP5 is constrained by numerous conditions; namely user input, code stability and code error control. The user input includes the maximum and minimum time step sizes, the frequency of plots, printed output, restart record dumps and optional error control specications. The semi-implicit time step method keeps the time step below the MCL for code stability. To control mass error, the time step is adjusted to keep the error of each advancement sufciently small. In the current version, if error or stability conditions are not met, the time step is cut in half. If the conditions are met extremely well, the time step is doubled [1]. For the best code execution time, the time step ideally should be just below the MCL. However, the time step may never exceed the user-supplied maximum step size, which can be far below the MCL. Even when MCL is below the maximum step size, the halving/doubling scheme can select a time step nearly half the size of the MCL. It is thus necessary to replace both of these constraints on time step size in order to approach the ideal time step size. All of the functions previously described for time step control in RELAP5 are computed in subroutine DTSTEP. The subroutine is responsible for numerous other functions including determining when to create output, terminate a transient and communicate via the parallel virtual machine with other codes [1]. As the goal of this research project is to improve code efciency, i.e., reduce RELAP5 run time without compromising code accuracy, several alternate time step algorithms are developed and implemented. Measures of the code performance are explained and presented. METHODS Description of Original Time Stepping Method The MCL is an important constraint for calculating time steps for the semi-implicit time advancement method in RELAP5-3D. It is computed for volume i according to Carlson [1] as shown below. (1) Here xi is the characteristic length dimension of the control volume. v is the velocity of the uid. is the void fraction. Subscript f indicates a liquid phasic property. Subscript g indicates a gaseous phasic property. Superscript n is the time advancement number. n ti represents the time it takes for uid entering at one boundary of control volume i to cross to its exit boundary during time advancement n. The MCL for the entire ow region is given by: [1] (2) where NVOL is the number of control volumes in the ow region. This ensures that no control uid crosses more than one control volume in a single advancement. With the semi-implicit time stepping method, this is the stability limit since steps larger than this can cause instabilities. Despite the possibility of instability and in order to take larger time steps, DTSTEP does not strictly enforce the MCL condition, t < MCL. Rather, it sorts the control volume tn into i ve groups based on the control volume number. Thus, group 1 would contain volumes 1, 6, and 11 and group 2 would contain volumes 2, 7, and 12, etc. The smallest minimum of any group is denoted MCL1, the second smallest group minimum, MCL2, etc. Note that MCL1 = MCL in the above condition. By default, MCL2

is selected as the MCL for time stepping. The user can select to enforce equation (2) by turning on card 1, option 15. Either way, the MCL condition in the original halving/doubling algorithm is given by Carlson [1]. (3) Equations (1), (2) and (3) form the original algorithm, denoted A1. Because of (3), t is rarely larger than the actual MCL of equation (2). There are occasions, however, when t > MCL. In this project we study and implement methods that allow larger t while always satisfying equation (2). Another time step constraint is the fact that edit times (plot, major edit, and restart times) must be reached exactly. To accomplish this, the number of advancements needed to reach the next integer multiple of DTMAX is stored in the integer variable NREPET, an integer giving the number of permutations (all edit times are multiples of DTMAX). For example, if the current time step is 0.25*DTMAX, and the cumulative time is 5.25*DTMAX, then NREPET equals three. This is because three more time steps of the current size are needed to reach the next integer multiple of DTMAX; namely 6*DTMAX. When the time step is halved, NREPET is doubled, and when the time step is doubled, NREPET is halved. As a result of truncation due to integer division, the time step can only be doubled when NREPET is even which causes inefciency. In the previous example, doubling when t = DTMAX/4 is not allowed because the truncation of 3/2 to 1 would indicate that only one step of size DTMAX/2 is needed to reach from 5.25*DTMAX to 6*DTMAX. Thus, the use of NREPET causes the code to take more advancements than necessary to reach the end time of the transient. To increase code efciency, NREPET should be replaced. NREPET has two uses: to control time step size and to help determine when edits are necessary. When odd, NREPET prevents the time step from being doubled. An NREPET equal to zero indicates that current time is a multiple of DTMAX. For the NREPET system to work properly (allowing for the time step to hit time targets) the time step must increase and decrease by the same factor. If this is not the case, the time step will cease to hit time targets due to round-off error which prevents the time step size from approaching the MCL. Therefore, replacing the NREPET with an MCL-based time step control is a natural objective. The second purpose of NREPET is to help check if an edit needs to be performed since edits are only performed when NREPET = 0. The original version of DTSTEP implements time step control with oating point data. For accuracy, these are being replaced by an equivalent method of time tracking based on integer data. This allows NREPET, an integer variable, to be replaced naturally by the integer timekeeping data of the new integer algorithm. Implementation of the New Method The integer version of subroutine DTSTEP/Version 2.4.1.2 implements 11 major functions as 18 major sections and subsections. Jumps within sub-sections and between sections make the code difcult to read and modify. Therefore, to improve readability at the outset, 18 FORTRAN 90 internal subroutines were separated out of the original DTSTEP code. This reduced the amount of 61

U.S. Department of Energy Journal of Undergraduate Research http://www.scied.science.doe.gov

executable code in the main DTSTEP subroutine from an original 1 400 lines to only 350 lines; making the code much easier to read. Two functions of NREPET, control of halving/doubling and determination of whether the current time is a multiple of DTMAX, were implemented using an equivalent variable from the integer time algorithm. The time step control algorithm was then modied to use MCL to determine increased time step size. Time step halving, however, was continued in order to control error. The modied algorithm, A2, is:

algorithms consist of various combinations of the MCL following algorithms (A2 and A3) with the time targeting algorithm (A4). An additional distinction between the combined algorithms was whether MCL1 or MCL2 was being used. These combined algorithms are summarized in Table 1.
Combined Algorithm Description C1. A2 with A4 C2. A3 with A4 C3. A2 with A4 and Option 15 C4. A3 with A4 and Option 15 DTMAX condition enforced ignored enforced ignored Courant Control mMCL2 mMCL2 mMCL1 mMCL1

(A2)

Table 1. Description and comparison of the algorithms.

here, 0 < m < 1.0. The removal of NREPET allows the time step to be set to any value as long as targets continue to be hit and the model remains stable. Coding for algorithm A2 was placed at the end of DTSTEP rather than scattered throughout. This created implementation issues. For example, when conditions requiring a time step halving were met, a logical variable recorded it and it was used in the ifsection of A2. During testing, it was discovered that implementing A2 at the end could cause indenite repeating of the same time step. This was overcome by specifying that the time step would again be halved if there were three unsuccessful advancements in a row. To more closely approach MCL, a violation of DTMAX was necessary; leading to algorithm A3. For input models with no ow conditions, MCL is essentially innite. Algorithm A3 employs an upper limit, arbitrarily selected as 20-DTMAX, to prevent this.
(A3)

The elimination of NREPET created a DTSTEP output problem with algorithms A2 and A3. Since MCL is unpredictable, seldom did cumulative time ever match edit time. Dn, the difference between edit time and cumulative time, divided by three is compared with the time step. If tn Dn, three steps of nearly equal size are taken to reach the time target. The rst step may be slightly larger due to aspects of the integer algorithm beyond the scope of this report, but the nal two are equal. This is summarized in algorithm A4. (A4)

To examine the workings of the new algorithms, time step size and MCL1 were recorded throughout two runs of TYP12002, one with DTMAX imposed and one with it ignored. MCL2 was used to control the time step. The code, by default, outputs MCL1 even when MCL2 is in use. Additionally, a comparison of the four algorithm combinations was made by varying m between 0.4 and 1.2 by steps of 0.05. With each variation of m, the test cases were executed ve times. Measures of mass error and execution speed were recorded with each run. The recorded run times were adjusted to not include input processing time which will be constant for each model. The test cases run were: ANS79, CSTEST1, FWHTR, NC, NEPTUNUS20, SCW, and TYP12002. Brief descriptions can be found in Table 2. For more detail of these test cases, refer to Carlson [2]. This variety of test cases is intended to expose any trouble that might occur with the new algorithms, as well as to show that the new algorithm is an improvement in most cases. As a whole, they represent the range of operating conditions that the new algorithm might encounter. This is the case with the notable exception of the parallel virtual machine, which is beyond the scope of this research project.
Model ANS79 CSTEST1 FWHTR NC NEPTUNUS SCW TYP12002 Description Two volume pipe; tests long term decay heat study Two loop plant with steam generator, nuclear core, pump and pressurizer RELAP5-3D feedwater heater component model test Water/steam-lled piping network test of ve separate cases of non-condensable gases: Air, CO, CO2, O2 NEPTUNUS pressurizer test y05 Reactor vessel model run to steady-state testing of the supercritical water of properties Typical Pressurized Water Reactor; 1200 second transient

Table 2. Brief descriptions of each model used [2].

In some transients, conditions required a time step reduction by a factor of two during the A4 process. In those cases, the time targeting process, A4, was deactivated until the time target was again within three current time steps of the current time. Performance Measurements Four sets of combined algorithms, C 1C 4, were tested with several input models. The input models or test cases, are combinations of hydrodynamic components, heat structures, plant controls, initial conditions and boundary conditions. The combined 62

As a control, each test case was run twice, rst with MCL1 and then with MCL2, using the original integer algorithm. Measures of mass error and execution speed were recorded with each run. Central processing unit (CPU) times were, however, not recorded due to the ndings from the measurements previously described. Two different measures of mass error were employed. Cumulative mass error is too sensitive to small changes in the timing of events to provide an effective measure of performance; therefore

U.S. Department of Energy Journal of Undergraduate Research http://www.scied.science.doe.gov

two other measures were developed. These are shown in equations (4) and (5): (4)

The absolute mass error rate is the average amount of mass that either appears or disappears from the numerical model per unit time. It is calculated with the formula:

For C2, the time step remains slightly below MCL. At about 0.33 seconds, it levels out at one third of the time between edits and is 3.3 times as large as the time step size in the C1 algorithm where DTMAX is imposed. This can also be seen in Figure 1. Despite the fact that only the time step size changed, the MCL varies quite substantially between graphs A and B in Figure 1. Similar behavior occurs in TYP12002 with the original integer-time coding running below DTMAX. It was found that in all input models, the number of advancements was proportional to the run time as shown in Figure 2.

(5) The latter equation sums the mass appearing and disappearing in the system due to truncation. The cumulative mass error rate however, sums up mass error without regard to whether it appears or disappears. If mass disappears and then reappears, then cumulative mass error will be zero, but error will still exist in the rest of the solution. The absolute mass error rate measure circumvents this and treats all mass error as a positive quantity. This produces another measure of accuracy similar to the cumulative mass error rate. This rate allows comparison of test cases with different execution times. RESULTS The results show a signicant improvement in execution time when the time step is unbounded by DTMAX, using algorithm combinations C2 and C4. There is also occasional improvement with algorithm combinations C1 and C3, where DTMAX is enforced. It is also found that algorithm combinations C1C4 work best with m set to 0.9. Figure 1 shows two graphs of MCL vs. time for algorithms C1 and C2 running the TYP12002 model with m at 0.9. This test case is very sensitive to time step selection, because of the models layout. For C1, the time step remains constant well below the MCL with some reductions due to excessive mass error as seen in Figure 1.

Figure 2. CPPU time, and the number of attempted advancements vs. m using C3. Error bars represent one standard deviation of the CPU time data.

Figure 1. Graphs of the MCL and time step in two runs of TYP12002. Graph A shows a run with DTMAX enforced and set to 0.1s. Graph B shows the same run with DTMAX not enforced.

U.S. Department of Energy Journal of Undergraduate Research http://www.scied.science.doe.gov

63

Response to Changing m The test cases had three types of response to a changing m: constant, smooth, and chaotic. Constant Category In four of the seven test cases, ANS79, FWHTR, NEPTUNUS20 and SCW the number of advancements is unaffected by changes to m. This is named the constant category. Also, changing between MCL1 and MCL2 has very little effect on the results. Further, algorithm combinations C1 and C3, with DTMAX imposed, take approximately the same number of steps as integer-time DTSTEP, Version 2.4.1.2/Interim Version r5b. However, algorithm combinations C2 and C4, which ignore DTMAX, greatly reduce the number of attempted advancements for these tests. As seen in Figure 3, the number of steps is often reduced by about 80%.

Smooth Category Two cases, CSTEST1 and NC, comprise the smooth category. These are by far the most notable. The run of CSTEST1 best represents the smooth category. Consider Figure 4, graph A. On the right side, the curves for C1 and C3 (DTMAX enforced) converge. On the left, curves for C1 and C2 (using MCL2) converge, as do C3 and C4 (using MCL1). The mass error rate in this category was not as stable as that in the constant category. The curves with DTMAX enforced (C1 and C3) still converge at the left side, while the others do not. Also, when m approaches the courant limit, mass error gets very large. This happens with m equal to 1 when MCL1 is used. Additionally, the removal of DTMAX, caused an increase in the absolute mass error rate as seen in Figure 4, graph B.

Figure 3. Measures of execution speed and accuracy with respect to m in runs of the NEPTUNUS20 test case, which is representative of the constant category of test cases. Graph A shows the number of advancements taken. Graph B shows the absolute mass error rate.

Figure 4. Measures of execution speed and accuracy with respect to m in runs of the CSTEST1 test case, which is representative of the smoothly varying category of test cases.

64

U.S. Department of Energy Journal of Undergraduate Research http://www.scied.science.doe.gov

Chaotic Category Only one test case, TYP12002, responded noisily to a changing m. This can be seen in Figure 5. The numbers of advancements attempted are grouped according to whether DTMAX is imposed or not. The curves without DTMAX imposed decrease slightly with increasing m. These curves also contain some noise. This can be seen in Figure 5, graph A. Figure 5, graph B shows that the mass error also behaves chaotically. DISCUSSION AND CONCLUSION Figures 3 through 5 show that the value of 0.9 is an acceptable value for m for virtually all input models. For some input models, failures occur at m = 1.0 and for others, failures occur at smaller values. However, 0.9 is always shown to be safe. The value m = 0.9 is recommended as it generally produces the best results as well. The results show that it is possible to greatly increase the execution speed of RELAP5 by using algorithms C2 and C4 (which ignore DTMAX), while producing only a relatively small increase in the mass error rate. This is especially true in the smooth test cases where the number of attempted advancements is decreased by 80%, while the absolute mass error rate increased no more than 30%. This is often the same as in the DTSTEP/Version 2.4.1.2/Interim Version r5b. Since mass error is already so small relative to the massive amounts of uid in the systems, this 30% increase should be acceptable to most users. The smooth test cases show the relation between the competing forces of DTMAX and the MCL in the time step selection. As was seen in the results section by the convergence on the left; when m is small, the MCL determines behavior. As was seen by the convergence on the right; when m is large, DTMAX determines behavior. Thus, the larger m loosens the MCL constraint. It is conceivable that the rate of edits caused the constant curves to not change with m. Because of the way the new edit time targeting algorithm A4 works, the time step can never exceed one third of the time between edits. This is seen somewhat in Figure 1, graph B), although that test case is chaotic. If the MCL is always

larger than the time step imposed by the edit frequency, the curve will not depend on m. By reviewing Figures 3 through 5, it can be seen that algorithm C4 with MCL1 and DTMAX ignored is generally the best for reducing the number of advancements, and thereby increases the code run speed the most. This is because it never violates equation (2) and therefore, it always remains stable. It also allows the time step to approach the ideal time step size of the MCL. The nal conclusion of this study is that algorithm C4, which strictly imposes the MCL condition of equation (2) and ignores DTMAX, with m = 0.9, produces the best results for most of the test cases. It reduces the number of advancements and code run time by up to 80%, while increasing measures of mass error by no more than 30%. Algorithm C4 should be implemented in RELAP5-3D as a permanent user option. ACKNOWLEDGMENTS Special thanks to Dr. George Mesina for his help and support in all parts of this project. Additional thanks goes to the RELAP team at INL for their support of this project. This research was funded by the U.S. Department of Energy, Ofce of Science. REFERENCES
[1] K.E. Carlson and the RELAP5-3D Code Development Team, RELAP5-3D Code Manual Volume I: Code Structure, System Models, and Solution Methods, Idaho National Engineering and Environmental Laboratory, Idaho Falls, ID, INEEL-EXT-98-00834, June 2005. [2] K.E. Carlson and the RELAP5-3D Code Development Team, RELAP5/MOD3 Code Manual Volume III: Developmental Assessment Problems, Idaho National Engineering and Environmental Laboratory, EG&G, Idaho Falls, ID, NUREG/CR-5535, EGG-2596, June 1990.

Figure 5. Response of the TYP12002 test case to a varying m. It is the only chaotic test case.

U.S. Department of Energy Journal of Undergraduate Research http://www.scied.science.doe.gov

65