Proceedings of the IMAC-XXVII February 9-12, 2009 Orlando, Florida USA 2009 Society for Experimental Mechanics Inc.

Identifying Models of Numerical Solution Error Using Time Series Analysis

Franois M. Hemez 1 Applied Physics Division (X-Division) Los Alamos National Laboratory Los Alamos, New Mexico 87545, U.S.A. Marine Marcilhac 2 Communication and Systems (C-S) 22 Avenue Galile 92350 Le Plessis Robinson, France

ABSTRACT: The main goal of this research is to identify mathematical models that describe the behavior of truncation error for arbitrary systems of partial differential equations solved by numerical methods. Methods of interest include finite element, finite difference, finite volume and particle methods. Truncation error is the difference between the exact solution of continuous equations and the numerical solution of discretized equations. Understanding how truncation error behaves and describing it with a mathematical model are essential steps of code verification activities. We propose to apply a MATLAB-based time series analysis toolbox developed at the University of Lancaster, U.K., and known as CAPTAIN, to numerical solutions obtained by running a computer code. The analysis is framed such that the CAPTAIN toolbox identifies a transfer function between the discrete solution and its truncation error. Once identified, the transfer function is converted to an ordinary differential equation that describes the numerical error and, in fact, represents the (possibly unknown) modified equation of the algorithm. A proof-of-concept is provided using datasets generated with a linear, harmonic oscillator and a non-linear ordinary differential equation. The technique proposed performs well on these examples and a path forward is proposed to take it one step further and identify the rate-of-convergence of a numerical algorithm using mesh refinement.

1. INTRODUCTION
Code and solution verification are defined as a scientifically rigorous and quantitative process for assessing the mathematical consistency between continuum and discrete variants of partial differential equations used to represent a reality of interest, as stated in Reference [1]. Verifying the numerical accuracy of discrete solutions computed by simulation code is important because partial differential equations that govern the equations of motion or conservation laws in computational engineering and physics are discretized for resolution with finite-digit arithmetic. The main challenge of assessing the performance of an algorithm and, therefore, the numerical quality of discrete solutions it delivers is to understand the extent to which solutions of the discretized equations converge to the (often unknown) exact solution of the continuous equations. Other Verification and Validation (V&V) activities used to establish the level of confidence that can be placed in a Modeling and Simulation (M&S) capability are discussed in Reference [2].

Technical staff member in X-Division. Mailing: Los Alamos National Laboratory, X-3, Mail Stop B259, Los Alamos, NM 87545, U.S.A. Phone: (+1) 505-667-4631. E-mail: hemez@lanl.gov. 2 Research and development engineer at C-S. Formerly post-doctoral research assistant in X-Division, Los Alamos National Laboratory. Mailing: Communication and Systems, C-S, 22 Avenue Galile, 92350 Le Plessis Robinson, France. Phone: (+33) (0)-1-41-28-47-82. E-mail: marine.marcilhac@c-s.fr.

In short, code verification is the assessment that the computer code does what it is supposed to do for an application of interest. It includes activities such as software quality assurance, verifying that there is no programming deficiency, no significant round-off or truncation errors, etc. The activities of solution verification, on the other hand, focus on demonstrating that the computational discretization provides adequately converged solutions. Solution verification and self-convergence study the error originating from the reliance on finite arithmetic and predict the numerical error (or solution error) obtained from running the calculation using specific values of mesh or grid sizes x, time steps t, etc. These activities are discussed in Reference [3]. A thread common to the above activities, whether one deals with code or solution verification, is to assess how truncation error behaves. This is because numerical methods always provide approximations. Finite element methods, for example, solve weak formulations of the equations of motion. The theory then states that weak and strong solutions become equivalent as x 0. They are not, however, equal and the difference between the two is captured by truncation error within the domain of asymptotic convergence. The procedure most commonly encountered to assess the behavior of truncation error is the refinement of a computational mesh. A discussion is provided in Reference [4]. One difficulty, however, is that the functional form of the mathematical model that one seeks to identify for truncation error is often unknown. It is common practice to assume the form of truncation error using a low-order polynomial such as, for example, yExact y(x) + .(xp) where yExact denotes the exact solution of the continuous equations, y(x) is the discrete solution obtained on the basis of discretization x and the last term .(xp) represents truncation error. Clearly, the results of a verification study are vulnerable to the correctness of this critical assumption. We propose a novel approach to assess the leading-order term of truncation error without having to assume a priori what its functional form may be. The technique identified models of transfer functions between discrete solutions computed by a code and their truncation errors. These transfer functions can then be converted into ordinary differential equations that govern how truncation error behaves. The technique proposed can be applied to any set of equations of motion or conservation laws that are solved with an arbitrary numerical method.

2. THE TRUCATION ERROR OF A NUMERICAL METHOD

The accuracy of a numerical method is, by definition, the difference between the exact solution of the continuous equations and discrete approximation estimated using a mesh or grid size x. In the following, the exact solution is referred to as yExact and the discrete solution is denoted by the symbol y(x) to stress the fact that it results from discretization x. Solution error is defined simply as:

(x ) = y Exact y( x) .

(1)

The implementation of a numerical method, such as a finite element method, finite volume method or finite differencing scheme, always involves approximations whereby representation or basis functions are truncated for coding on finite-arithmetic architectures. Clearly the degree to which the numerical solution is an accurate approximation of the exact solution depends on the order of truncation of the method. Assuming that truncation error is the dominant source of error, this can be represented mathematically as:

y Exact = y( x) + . x p + O x p +1 ,

( ) (

(2)

where the term .(xp) represents the leading order of truncation error. The symbol p denotes the observed rate-of-convergence of the numerical method. For example, a problem that uses quadratic finite elements should recover p = 2. Comparing equations (1) and (2) results in:

(x) . x p .

( )

(3)

It is emphasized that equality (3) assumes that truncation error is the most significant source of numerical error. Other potential sources of error include the lack-of-convergence of an iterative solver, numerical round-off and inappropriate resolution, to name only a few. The above assumption is precisely what defines the regime of asymptotic convergence. It is the range of resolution levels, xMin x xMax, within which truncation error dominates all other sources of error. An example of asymptotic convergence is illustrated in Figure 1. It pictures the behavior of error (1) when the Burgers equation is solved with a Lax-Wendroff integration scheme. The level of spatial resolution of the calculation is varied within 103 x 1 (cm) and clear evidence of first-order convergence is observed, as indicated by the slope p 1.2 of the curve on Figure 1. First-order convergence matches the theoretical order of truncation of the numerical scheme for these calculations that feature discontinuous solutions.

p=1

Figure 1. Illustration of asymptotic convergence for the 1D, Burgers equation. A numerical method and the discrete solutions it provides derive their accuracy from the fact that truncation error is under control. Being able to derive a mathematical model of truncation error is at the heart of verification activities. Such models are needed to verify the order of convergence of a numerical method, extrapolate discrete solutions as x 0 and quantify solution uncertainty. Common practice is, unfortunately, to assume how truncation error behaves and postulate its mathematical form. This practice can lead to grave mistakes, should assumptions made about the mathematical function used to represent the error be incorrect. This work attempts to address this issue by proposing a technique that treats the analysis code as a black box and can inform on an appropriate choice of model for a particular simulation or numerical application.

3. IDENTIFYING A MODIFIED EQUATION THROUGH TIME SERIES ANALYSIS

Other than the technique known as Modified Equation Analysis (MEA), there is no method to systematically derive mathematical models of truncation error. MEA is a general-purpose technique capable of exactly identifying the mathematical form of truncation error. Overviews are provided in References [5-6]. But its applicability is limited to relatively simple equations and numerical scheme. The derivations rapidly become intractable when applied, for example, to systems of finite element or finite volume equations, even with the help of a symbolic solver. It is to address this limitation that an alternative technique based on signal processing is proposed. The approach that we propose takes advantage of the duality between transfer functions and (linear) ordinary differential equations. The procedure is articulated in two steps. The first step is to carefully select input and output responses from a numerical simulation and identify a transfer function that relates them. The second step is to convert this transfer function into an ordinary differential equation. If these inputs and outputs represent the discrete solution and its truncation error, we will have obtained a differential equation that connects them, which is nothing less than the modified equation that defines truncation error. The theoretical framework is summarized below. A linear, differential equation that relates an input function F(x) to an output function y(x) can be generically written as:

y n y F m F 0 y(x) + 1 (x) + L + n (x) = 0 F(x) + 1 (x) + L + m (x) , x xn x xm

(4)

where coefficients 0 n and 0 m are constant. Note that the symbol x of equation (4) does not necessarily represent a spatial dimension. It could, for example, mean time or energy. Using a Laplace transform, the differential equation can be converted to:

F(s) = H(s) y(s) ,

(5)

where y(s) and F(s) are the Laplace, or frequency-domain, transforms of functions y(x) and F(x), respectively. The transfer function is defined as:

H(s) =

0 + 1s + L + n s n . 0 + 1s + L + m s m

(6)

We propose to rely on the theory represented by equations (4-6) but to proceed backwards. The transfer function is, first, identified from the appropriate responses. It is used next to provide the leading-order term of the differential equation that controls how truncation error behaves. The implementation of a pth-order accurate numerical method defines a truncation error whose leading order takes, in general, the form:

(x; t) . x p .

y ) (x; t) , x
q q

(7)

where is a function that depends on properties of the system of partial differential equations solved, but that is independent of spatial discretization x. Solving, for example, a structural mechanics problem with a quadratic finite element method yields p = 2 and q = 3. Equation (7) suggests how the input F(x) and output y(x) can be selected to simplify the identification. If the input is a discrete solution provided by the code and the output is the corresponding truncation error, then the transform of equation (7) in the frequency domain becomes:

(s) = 0 + 1s + L + n s n . y(s) , 144 4 2444 3

H(s)

(8)

where coefficients 0, 1 n depend on the type of numerical method and discretization x.

By selecting the input and output responses as the discrete solution and its truncation error, the model of transfer functions is reduced to the numerator of equation (6), which implies fewer coefficients to identify. A flow-chart that defines this approach in the context of code verification is shown in Figure 2. Perform Numerical Simulations with Different (x; t)

Estimate the Solution Error, (x;t) = yExact y(x;t)

Exact Solution Code

System Identification Toolbox

Estimate the Transfer Function, (s) = H(s).y(s)

Inverse Laplace Transform

Estimate the Ordinary Differential Equation of Truncation Error Figure 2. Flow-chart of the method proposed to identify modified equations. The implementation of Figure 2 relies on a methodology, and associated toolbox, for time series analysis developed for over two decades at the University of Lancaster, U.K. The software is called CAPTAIN and comes in the form of a MATLAB-based toolbox that operates on arbitrary responses. The identification decomposes these responses into dominant modes, as explained in References [7-8], then, best-fits the generic model of transfer functions shown in equation (6). Given the orders (n; m) of equation (6) that must be specified by the user, in addition to a few other settings, the toolbox identifies the best possible transfer function. Statistics of goodnessof-fit are estimated, which assists the user in determining the quality of the model identified. Details about the toolbox and its application can be found in References [9-10]. The CAPTAIN toolbox is fed with responses that can originate from linear or non-linear, inputoutput systems. No particular knowledge is needed about the underlying system that generates these signals. The fact that the dynamic system is treated as a black box is appealing when considering the application of the toolbox to code and solution verification. It is noted that, in our application, one need not restrict the analysis to time series. Responses analyzed can represent simulation results defined as a function of time, space or any other discretization variable. The only limitation is that the procedure illustrated in Figure 2 can only be applied to test problems for which an exact solution of the continuous equations is known.

4. PROOF-OF-CONCEPT WITH A DAMPED, HARMONIC OSCILLATOR

A proof-of-concept is first discussed. The approach proposed is applied to the resolution of a linear, single degree-of-freedom (SDOF) harmonic oscillator with a first-order accurate finite differencing method. The Euler forward algorithm is used. This application is provided to verify how the method performs when everything (exact solution and modified equation) is known. MEA demonstrates that applying the Euler forward integration scheme to the harmonic oscillator solves the following ordinary differential equation more accurately that the original equation:

2 y y 2 y (t) + (t) + y(t) f(t) = t 2 (t ) , t2 t t4 14 2 3

(t)

(9)

meaning that the differential equation that governs truncation error is given by:

(t) = t

2 y (t) . t2

(10)

Next equation (10) is discretized with the Euler forward time integration scheme implemented to solve the original harmonic oscillator. Using the same scheme is important for consistency. The discrete version of equation (10) is:

y[k+ 2] 2 y[k+ 1] + y[k] [k] = . t

(11)

Using a control theory notation that mixes the sampled, time-domain quantities y[k] and [k] and continuous transfer function H(s), equation (11) can be described symbolically as:

[k] =

1 2 s + s 2 y[k] . t4 1 4 244 3
H(s)

(12)

It is emphasized that the notation of equation (12) is symbolic. It is meant to represent the filter H(s) that acts on sampled inputs y[k] to produce outputs [k]. The notation would not, clearly, be used for an implementation since it confuses time-domain and frequency-domain quantities. The point this derivation is to show that we are looking for a model of transfer function that can be expressed as a second-order polynomial:

H(s) = 0 + 1s + 2s 2 ,

(13)

and, because the modified equation is derived explicitly in equation (12), the triplet of polynomial coefficients (0; 1; 2) is also known:

0 =

2 , 1 = = 2 0 and 2 = = 0 . t t t

(14)

In general the transfer function and its coefficients are, of course, unknown. The user would, then, search for the polynomial form and coefficient values that best-fit the available data. The fact that the exact solution is, here, known is used to our advantage to verify that the procedure yields sensible results. The numerical application presented next integrates the harmonic oscillator over the time period 0 t 2 sec. Three solutions are obtained with time samples t = 4.103, 103 and 2.5 104 sec. Because the exact solution is known, the solution error (t) = yExact y(t) is computed for each response. The CAPTAIN toolbox is then provided with pairs of time series (y(t); (t)) for each one of the three runs. Figures 3 and 4 show the discrete solutions and truncation errors when

analysis is performed with time steps t = 4 milli-sec. (in Figure 3) and t = 0.25 milli-sec. (in Figure 4). The figures also compare the original truncation errors (dashed, red lines) to the reconstruction provided by CAPTAIN (green, solid lines). The identification of transfer functions leads to reasonable goodness-of-fit to data, both in terms of amplitude and phase.

Truncation Error, (t)

Time Samples

Discrete Solution, y(t)

Time Samples

Figure 3. Identification of truncation error at t = 4.103 sec. for the linear oscillator. (Top: Comparison of truncation errors (t); bottom: discrete solution y(t).)

Truncation Error, (t)

Time Samples

Discrete Solution, y(t)

Time Samples

Figure 4. Identification of truncation error at t = 2.5.104 sec. for the linear oscillator. (Top: Comparison of truncation errors (t); bottom: discrete solution y(t).)

Table 1 summarizes the results of fitting transfer function models (13) to the datasets. Values of the triplets of identified coefficients (0; 1; 2) are listed, together with goodness-of-fit statistics estimated by CAPTAIN. Table 2 scales coefficients 0, 1 and 2 according to equation (14) to eliminate their dependence on time step t, which makes it easier to compare them. It can be observed that the identification is consistent in the results it provides, as one would expect. Table 1. Identification results for the linear oscillator. Quantity of Interest Time Step t Identified Coefficient 0 Identified Coefficient 1 Identified Coefficient 2 Goodness-of-fit Statistic Discrete Solution, y(t) Coarse 4.103 sec. 94.7 -187.8 93.1 78.4% Medium 103 sec. 380.6 -754.2 373.6 87.4% Fine 2.5.104 sec. 1,458 -2,889 1,430 97.4%

(Legend: The goodness-of-fit statistic is an indicator that is scaled in the 0-to-1 range.)

Table 2. Scaled coefficients k Identified with CAPTAIN for the linear oscillator. Quantity of Interest Corresponding Coefficient (0t) Corresponding Coefficient (-1t/2) Corresponding Coefficient (2t) Discrete Solution, y(t) Coarse 0.379 0.376 0.372 Medium 0.381 0.377 0.374 Fine 0.365 0.361 0.357

The overall observation we draw from this first example is that the method performs according to expectation. Given sampled datasets, it is capable of identifying a transfer function that can be converted back to an ordinary differential equation of the modified equation. But this example only provides a crude sanity check that the approach is reasonable. A more complicated dataset is analyzed next to assess the performance of the method.

5. APPLICATION TO THE NON-LINEAR, HYPERBOLIC BURGERS EQUATION

The next level up in complexity is the analysis of a non-linear equation. The well-known, 1D Burgers equation is selected for its mathematical properties (it is a hyperbolic equation) and ability to generate discontinuous solutions, given appropriate initial conditions. After giving a brief summary of the problem and its modified equation, the results of a numerical simulation performed with a first-order method are discussed. 5.1 Description of the Burgers Equation and its Modified Equation The equation chosen for analysis is Burgers equation in 1D, Cartesian geometry. It is nonlinear and hyperbolic and bears resemblance with other systems of equations, such as the Euler equations of gas dynamics or Navier-Stokes equations of computational fluid dynamics, in its ability to develop discontinuous solutions. The continuous Burgers equation is defined as:

y 1 y2 2 y (x; t) + (x; t) = (x; t) , t 2 x x2

(15)

where a diffusive term is added in the right-hand side. Coefficient is a user-defined constant. A first-order, upwind method is used to integrate equation (15). Details of this numerical scheme

are discussed in Reference [11]. The upwind method, also known as Euler backward in other circles, is one of the standard techniques used in computational engineering and physics. Even tough equation (15) and the first-order upwind method are both simple, the derivations needed to carry out the MEA become intractable by hand. Using, however, the symbolic solver MathematicaTM, a solution can be obtained. The leading term of truncation error is shown below:

y 1 y2 2 y y 2 y 1 3 y 1 y 2 y 4 y 2 + = x + x 6 y x3 2 x x 2 + 6 x4 + L, t 2 x x2 x2 2 14444444444 4 2444444444 4 4 3
(x;t)

(16)

where the notation y = y(x;t) is used for simplicity. Equation (16) indicates that the numerical method applied to Burgers equation is first-order in space. A similar analysis leads to secondorder in time (not shown). The dominant term of spatial truncation error is defined as:

(x; t) =

y(x; t) 2

2 y x (x; t) . x2

(17)

Because the highest-degree derivative in equation (17) is 2nd-order, one can search for a transfer function H(s) = 0 + 1s + 2s2 that assumes the form of a 2nd-order polynomial. One advantage of analyzing this problem is that, again, the triplet of coefficients (0; 1; 2) is known exactly and can be shown to be equal to:

0 =

y 2 x

1 =

y x

= 2 0 and 2 =

y 2 x

= 0 .

(18)

The values of (0; 1; 2) identified with CAPTAIN can be compared to these exact solutions to assess the performance of the method when datasets are, here, generated from the analysis of a non-linear equation. 5.2 Identification of the Modified Equation for Burgers Equation A total of four discrete solutions are computed over the spatial domain 0 x 1 cm using successively refined meshes. The coarsest level of mesh resolution is a uniform grid sampled at x = 4.102 cm. This first mesh defines a total of 25 grid points at which the solution y(x;t) is evaluated. A refinement ratio of R = 2 is used, meaning that the grid is halved each time to generate the other three computational meshes. The Burgers equation is integrated to the final time of t = 1 sec., starting from a smooth initial condition. (It produces no discontinuity.) Table 3. Identification results for the non-linear Burgers equation (upwind method). Quantity of Interest Mesh Size x Number of Grid Points Time Step t Number of Time Samples Identified Coefficient 0 Identified Coefficient 1 Identified Coefficient 2 Goodness-of-fit Statistic Discrete Solution, y(x;t) Extra-coarse 4.102 cm 25 3 10 sec. 1,000 12.93 -28.02 14.60 85.4% Coarse 2.102 cm 50 3 10 sec. 1,000 26.05 -54.37 27.65 94.8% Medium 102 cm 100 3 10 sec. 1,000 55.15 -112.51 56.64 98.1% Fine 5.103 cm 200 3 10 sec. 1,000 117.40 -236.89 118.76 99.4%

(Legend: The goodness-of-fit statistic is an indicator that is scaled in the 0-to-1 range.)

Table 3 summarizes the results of fitting transfer functions H(s) = 0 + 1s + 2s2 to the datasets. In Table 4, the regression coefficients 0, 1 and 2 are scaled to eliminate their dependence on mesh size x and facilitate the comparison to the exact solution. Table 4. Scaled coefficients k Identified with CAPTAIN for Burgers equation. Quantity of Interest Corresponding Coefficient (0x) Corresponding Coefficient (-1x/2) Corresponding Coefficient (2x) Exact Solution (|y|/2) Discrete Solution, y(t) Extra-Coarse 0.517 0.560 0.584 0.500 Coarse 0.521 0.544 0.553 0.500 Medium 0.551 0.563 0.566 0.500 Fine 0.587 0.592 0.594 0.500

For this numerical application, the exact value of the term |y|/2 that appears several times in equation (18) is equal to ; it defines a reference for comparison with the identified coefficients. Coefficients (0; 1; 2) listed in Table 4 are in reasonable agreement with the exact value. It can be further observed that a 10-to-20% error in their values is tolerable while the identified transfer function remains capable of fitting the pair of (discrete solution; truncation error) responses with acceptable accuracy, as indicated by the high goodness-of-fit statistics of Table 3. What seems to be an inherent level of robustness in the method is welcome because it implies that errors can be tolerated in the identification of the coefficients. Even if they are present, these errors do not jeopardize the identification of the leading term(s) of the modified equation. Based on results of Table 3, one would conclude that the transfer function between a discrete solution and its truncation error is a 2nd-order polynomial. It corresponds to a truncation error whose dominant contribution obeys an ordinary differential equation of the type:

(x; t) (x ).

2 y (x; t) . x2

(19)

for Burgers equation discretized with the upwind method. The identified model (19) agrees with the dominant term of spatial truncation error (17). The coefficient (x) of equation (19) depends on the mesh size and this function defines the order of accuracy of the numerical method. The rate-of-convergence is, however, unknown at this point and an approach is suggested next to identify its value from a mesh refinement study.

6. A PATH FORWARD TO IDENTIFY THE ORDER OF ACCURACY

As explained above, we have so far focused on the identification of a transfer function that represents an unknown, dynamic system between discrete solutions and their truncation errors. The transfer function can, then, be converted into an ordinary differential equation that models the leading-order term of a modified equation. This procedure, however, says nothing about the formal order of accuracy (or rate-of-convergence) of truncation error. In Reference [12], it is proposed to augment the approach illustrated in Figure 2 by adding the estimation of the order of accuracy of the numerical method. To do so, multiple calculations are needed from a mesh refinement study. These runs provide several estimations of the transfer function coefficients. Because these coefficients are estimated from calculations performed with different mesh sizes (or time steps), a relationship can be established between their values and the rate-of-convergence of the numerical method. This is illustrated, for example, in equation (19) that features a coefficient (x) that depends on the mesh size. Performing, say, three runs with coarse (xC), medium (xM) and fine (xF) mesh

sizes yields a triplet of leading-order coefficients {(xC); (xM); (xF)} from which a power-law model such as (x) = .(xq) can be identified. The exponent q is directly related to the order of accuracy. Results obtained in Reference [12] indicate that this approach recovers rates-ofconvergence that are in excellent agreement with the formal order of accuracy of the method.

7. CONCLUSION
A novel approach is proposed to assess the leading-order term of truncation error without having to assume a priori what its functional form may be. The technique is based on identifying mathematical models of transfer functions between discrete solutions computed by a code and their truncation errors. These transfer functions can then be converted into ordinary differential equations that govern how truncation error behaves. The identification is carried out using a MATLAB-based toolbox for time series forecasting developed at the University of Lancaster, U.K., that decomposes time series into dominant modes. The approach can assess the dominant effect of truncation error for any system of conservation laws that are solved with an arbitrary numerical method. The computer code is treated as a black box. It is used simply to obtain discrete solutions from a mesh refinement study. Discrete solutions and truncation errors provided to the forecasting toolbox do not necessarily need to be signals defined in time. Likewise assumptions such periodicity, linearity or stationarity are not needed. These attributes make it attractive to apply the technique to a diversity of algorithms in computational engineering and physics. The main limitation is that an exact solution of the continuous equations is needed to estimate truncation error, which restricts applicability of the method to code verification test problems. The examples discussed in this report are simple proofs-of-concept. More work is warranted to apply the technique to other test problems and gain confidence in its performance. Future development could include application to coupled physics test problems where one algorithm may behave differently from another one in terms of truncation error. One example would be a structural-thermal problem where a 1st-order finite element method is used to calculate the mechanical stresses while a 2nd-order finite volume method is applied to the thermal field. Even though the modified equation of either algorithm may be well characterized, truncation error for the coupled calculation may behave in a manner that is different from the sum of its parts.

ACKNOWLEDGMENTS
This work is performed under the auspices of the Advanced Scientific Computing (ASC) Verification and Validation (V&V) program at Los Alamos National Laboratory (LANL). The first author is grateful to Mark C. Anderson, V&V program leader, for his continued support. Also, the contribution to this research of our colleagues Jerry S. Brock and James R. Kamm is recognized and greatly appreciated. LANL is operated by the Los Alamos National Security, LLC for the National Nuclear Security Administration of the U.S. Department of Energy under contract DEAC52-06NA25396.

BIBLIOGRAPHICAL REFERENCES
[1] Brock, J.S., ASC Level-2 Milestone Plan: Code Verification, Calculation Verification, Solution-error Analysis and Test-problem Development for LANL Physics Simulation Codes, Technical Report LA-UR-05-4212 of the ASC Code Verification Project, Los Alamos National Laboratory, Los Alamos, New Mexico, May 2005.

[2]

Hemez, F.M., Doebling, S.W., Anderson, M.C., A Brief Tutorial on Verification and Validation, 22nd SEM International Modal Analysis Conference, Dearborn, Michigan, January 26-29, 2004. Roache, P.J., Verification in Computational Science and Engineering, Hermosa Publishers, Albuquerque, New Mexico, 1998. Hemez, F.M., Challenges to the State-of-the-practice of Solution Convergence Verification, 9th AIAA Non-deterministic Approaches Conference, Oahu, Hawaii, April 23-26, 2007. Hirt, C.W., Heuristic Stability Theory for Finite Difference Equations, Journal of Computational Physics, Vol. 2, 1968, pp. 339-355. Warming, R.F., Hyett, B.J., The Modified Equation Approach to the Stability and Accuracy Analysis of Finite Difference Methods, Journal of Computational Physics, Vol. 14, 1974, pp. 159-179. Young, P.C., Dominant Mode Analysis: Simplicity Out of Complexity, Proceedings of the Sensitivity Analysis and Model Output (SAMO) Summer School, Venice, Italy, July 2006. Young, P.C., Data-based Mechanistic Modelling, Generalised Sensitivity and Dominant Mode Analysis, Computer Physics Communication, Vol. 117, 1999, pp. 113-129. Pedregal, D.J., Taylor, C.J., Young, P.C., System Identification, Time Series Analysis and Forecasting: The Captain Toolbox, Handbook Version 2, University of Lancaster, United Kingdom, February 2007. Web resource: http://www.es.lancs.ac.uk/cres/captain.

[3] [4]

[5] [6]

[7] [8] [9]

[10] Young, P.C., Data-based Mechanistic Modelling of Environmental, Ecological, Economic and Engineering Systems, Environmental Modeling and Software, Vol. 13, 1998, pp. 105-122. [11] van Leer, B., Upwind and High-Resolution Methods for Compressible Flow: From Donor Cell to Residual-Distribution Schemes, Communications in Computational Physics, Vol. 1, No. 2, April 2006, pp. 192-206. [12] Marcilhac, M., Hemez, F.M., Brock, J.S., Kamm, J.R., Identifying Models of Numerical Solution Error Using Time Series Analysis, Technical Report of the ASC Code Verification Project, Los Alamos National Laboratory, Los Alamos, New Mexico, October 2008.