Matlab programs for complete and

incomplete data exact VARMA

likelihood and its gradient

Kristján Jónasson

Report VHÍ-02-2006
Reykjavík, September 2006
Report VHÍ-02-2006, Reykjavík September 2006

Kristján Jónasson. Matlab programs for complete and incomplete data exact VARMA likelihood and its gradient.,
Engineering Research Institute, University of Iceland, Technical report VHI-02-2006, September 2006

Kristján Jónasson, Department of Computer Science, University of Iceland, Hjardarhagi 4, IS-107 Reykjavik,
Iceland. Email: jonasson@hi.is.

The author is responsible for the opinions expressed in this report. These opinions do not necessarily represent the
position of the Engineering Research Institute or the University of Iceland.

 Engineering Research Institute, University of Iceland, and the author.

Engineering Research Institute, University of Iceland, Hjarðarhagi 4, IS-107 Reykjavík, Iceland

CONTENTS

1. INTRODUCTION 3

2. FUNCTION VALUE AND GRADIENT OF LOG-LIKELIHOOD 3

2.1 VAR models 3
2.2 Complete data VARMA models 4
2.3 Missing value VARMA models 5
2.4 Model parameters given by a function 5

3. SIMULATION 5

4. TESTING 6

APPENDIX A. MAIN MATLAB PROGRAMS AND SUBFUNCTIONS 7

APPENDIX B. TEST PROGRAMS 21

REFERENCES 33
ÁGRIP
Matlab föll til að reikna út nákvæmt log-sennileikafall fyrir VAR- og VARMA-tímaraðalíkön eru sett fram og
lýst. Föllin ráða við göt í mælirunum. Þau reikna ennfremur stigul sennileikafallsins, en hann nýtist við stikamat
með tölulegri hámörkun sennileikafallsins. Föllin bjóða upp á sparnað í ýmsum sértilvikum t.d. við mat á árstíða-
bundnum líkönum. Fallasafninu fylgir forrit til að herma VARMA-tímaraðir, og hafa hermdu raðirnar rétta
dreifingu strax frá fyrsta lið. Fallasafninu fylgja prófunarforrit, bæði til að staðfestingar á villuleysi og til nota við
frekari þróun safnsins.

ABSTRACT
Matlab functions for the evaluation of the exact log-likelihood of VAR and VARMA time series models are
presented (vector autoregressive moving average). The functions accept incomplete data, and calculate analytical
gradients, which may be used in parameter estimation with numerical likelihood maximization. Allowance is
made for possible savings when estimating seasonal, structural or distributed lag models. Also provided is a
function for creating simulated VARMA time series that have an accurate distribution from term one (they are
spin-up free). The functions are accompanied by a test suite for verifying their correctness and aid further
development.
1. INTRODUCTION
Several Matlab functions to aid in the analysis of multivariate time series models have been written and
are available at the authors web page, www.hi.is/~jonasson. In addition, the source code is listed in Ap-
penxix A below. There are three functions for evaluating exact log-likelihood, a function for simulating
time series, and a suite of test functions for verifying the correctness of the other functions. The three
log-likelihood evaluation functions are varma_llc for VARMA (vector autoregressive moving aver-
age) models with complete data, varma_llm for VARMA models with missing values and var_ll
for VAR models with or without missing values. All three functions can optionally calculate the gradi-
ent of the log-likelihood, estimates of missing values, and estimates of associated residual or shock se-
ries.
The simulation function is named varma_sim, and it may be used to generate a single VARMA or
VAR series, or several series at a time sharing the same parameters. One of the novelties of
varma_sim is that the initial values are drawn from the appropriate distribution, so that throwing
away the first part of the series to avoid spin-up effects is not needed.
The programs implement the methods described in the companion report [Jonasson and Ferrando 2006]
and they follow very closely the notation used there (including variable names). The companion report
also describes numerical experiments carried out with the Matlab functions.
The development of the software was inspired to an extent by the testing philosophy popularized with
extreme programming: write test programs for everything, preferably before writing the actual algo-
rithms being implemented; see for example [George and Williams 2004]. Thus, the confidence in the
correctness of the coded algorithms is fairly high.
Previously published programs for VARMA and VAR likelihood evaluation are the Fortran programs
of Shea [1989] and Mauricio [1997] and the Matlab programs of Schneider and Neumaier [2001]. At-
4
tention should also be drawn to E by J. Terceiro and others. It is a collection of Matlab functions for
4
state-space estimation of econometric models. E is distributed under the GNU license and available on
the web along with a user manual: www.ucm.es/info/icae/e4. The software and manual have not been
published, but there are some related publications listed on this web page, including [Terceiro 1990].

2. FUNCTION VALUE AND GRADIENT OF LOG-LIKELIHOOD

There are three functions for likelihood evaluation supplied: var_ll implements the savings described in
section 3.3 of the companion report for both the complete data and missing value cases, varma_llc im-
plements the method of section 2.2 and varma_llm the method of section 3.1 in the companion report.
The functions can also find gradients, residual estimates and missing value estimates, c.f. sections 3.2
and 4 of the companion report. When observations are missing, the functions accept the mean-vector as
a proper model parameter (c.f. the introduction to the companion report), but the complete data calls
assume a zero-mean model. To fit a non-zero-mean time-series, the mean-vector of the observations
may be subtracted from each xt at the outset. Help for all three functions is obtained by giving the com-
mand help function at the Matlab prompt (e.g. help var_ll).

2.1 VAR models

A zero-mean VAR model describing a time series of values xt ∈ ℝ , t = 1,…, n, is given by:
r

3
 p
x t = ∑ A j xt − j + ε t (1)
j =1

where the Aj’s are r × r parameter matrices and the εt’s are r-variate N(0, Σ) uncorrelated in time. To
evaluate the exact log-likelihood function associated with (1) when no observations are missing use the
Matlab call:
[ll,ok] = var_ll(X,A,Sig)

where A is an r × rp matrix containing [ A1 … Ap ], Sig is r × r and symmetric containing Σ, and X is an

r × n matrix with xt in its t-th column. Let θ ∈ R nθ with nθ = pr 2 + r (r + 1) 2 denote the vector of parame-
ters, i.e. the elements of A1,…, Ap (column by column) and the lower triangle of Σ. If they describe a
stationary model, ll will return a scalar with the value of the log-likelihood function l (θ ) and the logi-
cal variable ok will return true, but if the model is non-stationary ll will be 0 and ok will be false. To
calculate the 1 × nθ gradient l ′(θ ) in ll or the (maximum likelihood) estimate of the residuals (or
shocks) εt in res use the calls:
[ll,ok,lld] = var_ll(X,A,Sig)
[ll,ok,res] = var_ll(X,A,Sig,'res').

A non-zero-mean VAR model may be written:

p
xt − µ = ∑ A j ( xt − j − µ) + ε t (2)
j =1

If X, A and Sig are as before, mu contains µ and miss is an r × n logical matrix, which is true in locations
of missing values, the Matlab call:
[ll,ok] = var_ll(X,A,Sig,mu,miss)

will return the log-likelihood value in ll (or zero ll and false ok if the model is non-stationary). To
calculate the gradient, residuals, or residuals and maximum likelihood estimates of missing values use
the calls
[ll,ok,lld] = var_ll(X,A,Sig,mu,miss)
[ll,ok,res] = var_ll(X,A,Sig,mu,miss,'res')
[ll,ok,res,xm] = var_ll(X,A,Sig,mu,miss,'res_miss').

(µ is placed at the end of θ and nθ is now pr 2 + r (r + 3) 2).

2.2 Complete data VARMA models

A zero-mean VARMA model for xt ∈ ℝ , t = 1,…, n, is given by
r

p
xt = ∑ A j xt − j + y t (3)
j =1

where y t = εt + Σ j =1 B j εt − j , the Aj’s and the Bj’s are r × r matrices, and the εt’s are r-variate N(0, Σ) un-
q

correlated in time. If X, A and Sig are as in section 2.1 and B contains [ B1 … Bq ], the Matlab call
[ll,ok] = varma_llc(X,A,B,Sig)

will return the likelihood function value in ll and ok will be true unless the model is non-stationary,
then ok will be false. The calls
[ll,ok,lld] = varma_llc(X,A,Sig)
[ll,ok,res] = varma_llc(X,A,Sig,'res')

4
return in addition the 1 × ( p + q)r 2 + r (r + 1) 2 gradient l ′(θ ) in lld or the residual estimates in res,
where θ consists of the parameters: the columns of the Aj’s followed by the columns of the Bj’s fol-
lowed by the columns of the lower triangle of Σ.

2.3 Missing value VARMA models

Let µ be the expected value of xt ∈ ℝ and let other model parameters as well as yt and εt be as in sec-
r

tion 2.2. A non-zero-mean VARMA model for xt, t = 1,…, n, is given by

p
xt − µ = ∑ Aj ( xt − j − µ) + y t (4)
j =1

If X, A, B and Sig are as in section 2.2, mu contains µ and miss is an r × n logical matrix which is true
in locations of missing values then the Matlab call
[ll,ok] = varma_llm(X,A,B,Sig,mu,miss)
will return the likelihood function value in ll and ok will be true unless the model is non-stationary.
The calls
[ll,ok,lld] = varma_llm(X,A,B,Sig,mu,miss)
[ll,ok,res] = varma_llm(X,A,B,Sig,mu,miss,'res')
[ll,ok,res,xm] = varma_llm(X,A,B,Sig,mu,miss,'res_miss')

return in addition the 1 × ( p + q)r 2 + r (r + 3) 2 gradient l ′(θ ) in lld, residual estimates in res, and the
last one returns maximum likelihood estimates of missing values in xm, where µ has been appended to
the θ of section 2.2.

2.4 Model parameters given by a function

Let θ = g (φ ) where φ ∈ R φ and let Jg be the nθ × nφ Jacobian of g as in section 4.3 of the companion re-
n

port. To realize the savings discussed there, use one of the calls
[ll,ok,lld] = var_ll(X,A,Sig,J)
[ll,ok,lld] = var_ll(X,A,Sig,mu,miss,J)
[ll,ok,lld] = varma_llc(X,A,Sig,J)
[ll,ok,lld] = varma_llm(X,A,B,Sig,mu,miss,J)

where J contains Jg. A partial variable change is also possible; c.f. the help text of the functions. To take
an example, assume a distributed lags model,
3
x t = ∑ b j xt − j + ε t
j =1

where the bj’s are fixed constants and the model parameters consist of the r × r matrix C together with
the shock covariance matrix Σ. To evaluate the likelihood and its gradient efficiently the following Mat-
lab code may be used:
A = [b(1)*C b(2)*C b(3)*C];
I = eye(3*r);
JC = [I; I; I];
JSig = eye(r*(r+1)/2);
J = blkdiag(JC, JSig);
[ll,ok,lld] = var_ll(X,A,Sig,J);

3. SIMULATION
The Matlab function varma_sim will generate a random VARMA time series for a specified model. If
A, B and Sig are as above, then the calls

5
 X = varma_sim(A,[],Sig,n)
X = varma_sim(A,B,Sig,n)

generate a single zero-mean n-term series modelled by (1) or (3) in the r × n matrix X. The calls
X = varma_sim(A,[],Sig,n,[],M)
X = varma_sim(A,B,Sig,n,[],M)

will create M such series. When r = 1 X will be n × M and when r > 1 it will be r × n × M. To generate
non-zero-mean series as modelled by (2) or (4) use the calls
X = varma_sim(A,B,Sig,n,mu)
X = varma_sim(A,B,Sig,n,mu,M),

possibly with empty B. The series are started using the procedure described in the second paragraph of
Appendix D in the companion report, and when moving average terms are present the initial shocks are
also drawn as described there.

It is also possible to specify terms to start the series using

X = varma_sim(A,B,Sig,n,mu,M,X0)

where X0 has r rows and at least max( p, q) columns. All the generated series will begin with the last
max( p, q) columns of X0 and the corresponding shocks are drawn as explained above. As before, A, B
and/or mu may be empty.

The shocks used for the generation may be obtained by specifying a second return parameter: [X,eps]
= varma_sim(…). The dimension of eps will be same as that of X.

4. TESTING
The programs in the test suite are of two types: primary tests, for testing the four main functions dis-
cussed above, and secondary tests, that test individual components (subfunctions, helper functions) of
the main functions. To verify the correctness of the main functions it is only necessary to examine and
run the primary tests. The secondary tests were written as an aid in developing the program suite. They
are included for completeness, and as an aid for possible future development and changes. The primary
tests are:
test_varma_llc
test_varma_llc_deriv
test_var_ll
test_var_ll_jac
test_varma_llm
test_varma_jac
test_varma_llm_deriv
test_varma_sim

The correctness of varma_llc is checked against direct likelihood evaluation with equation (2.3) of
the companion report. The value is also compared with that from Algorithm AS311 of Mauricio [1997]
(a Matlab mex-file for that program is included in the test-suite). The function varma_llm is checked
against varma_llc for complete data, and against direct evaluation with equation (2.4) of the compan-
ion report for missing values, and var_ll is simply compared with varma_llm. Gradient calculation
and the Jacobian feature (see section 2.4) are checked by comparing with numerical differentiation. All
tests are carried out for several different test cases with a range of values of p, q and r. Finally the test-
ing of varma_sim is accomplished by comparing data expectations and covariances of generated series
with theoretical ones. All the primary tests may be run via the Matlab script TEST_PRIMARY, and with
TEST_ALL the secondary tests are also run.

6
APPENDIX A. MAIN MATLAB PROGRAMS AND SUBFUNCTIONS
% VARMA_LLC VARMA likelihood and its derivative for complete data % VARMA_LLM VARMA likelihood for missing data
% %
% [LL, OK] = VARMA_LLC(X, A, B, Sig) sets LL to the exact log-likelihood % [LL, OK] = VARMA_LLM(X, A, B, Sig, mu, miss) returns the value of the exact
% function value for n complete observations of the r-variate zero-mean VARMA % log-likelihood function for an r-variate VARMA time series of length n:
% time series: %
% % x(t) - mu = A1·(x(t-1) - mu) + ... + Ap·(x(t-p) - mu) + y(t)
% x(t) = A1·x(t-1) + ... + Ap·x(t-p) + y(t) % where
% where % y(t) = eps(t) + B1·eps(t-1) + ... + Bq·eps(t-q),
% y(t) = eps(t) + B1·eps(t-1) + ... + Bq·eps(t-q), %
% % and x(t), y(t) and eps(t) are r-dimensional with eps(t) normally
% x(t) is r-dimensional and eps(t) is r-variate normal with mean 0 and % distributed, N(0,Sig). Sig and the Ai's and Bi's are r×r matrices, and mu is
% covariance Sig. Sig and the Ai's and Bi's are r×r matrices. On entry A % an r-vector with the mean of x(t). A should be the r × r·p matrix [A1
% should be the r × r·p matrix [A1 A2...Ap], B should be [B1 B2...Bq] and X % A2...Ap], B should be [B1 B2...Bq] and X should have x(t) in its t-th column
% should have x(t) in its t-th column for t = 1,...,n. OK = true indicates % for t = 1,...,n. Set miss(i,t) to true if X(i,t) is missing. OK = true
% success, but is false if the model is non-stationary. % indicates success, but is false if the model is non-stationary.
% %
% [LL, OK, LLD] = VARMA_LLC(X, A, B, Sig) returns the log-likelihood function % [LL, OK, LLD] = VARMA_LLM(X, A, B, Sig, mu, miss) returns the log-likelihood
% value in LL and its gradient in LLD. % function value in LL and its gradient in LLD.
% %
% [LL, OK, EPS] = VARMA_LLC(X, A, B, Sig, 'res') returns the maximum % [LL, OK, EPS] = VARMA_LLM(...,'res') returns the maximum likelihood estimate
% likelihood estimate of the residuals in EPS. % of the residuals in EPS and [LL, OK, EPS, XM] = VARMA_LLM(..., 'res_miss')
% % finds also the maximum likelihood estimate of the missing values in XM.
% [LL, OK, LLD] = VARMA_LLC(X, A, B, Sig, J) may be used to speed up the %
% gradient calculation when A, B and/or Sig depend on a smaller set of % [LL, OK, LLD] = VARMA_LLM(X, A, B, Sig, mu, miss, J) may be used to speed up
% independent variables, and only the gradient w.r.t. this smaller set is % the gradient calculation when A, B and/or Sig depend on a smaller set of
% sought (for instance to fit structural models, distributed lags, and other % independent variables, and only the gradient w.r.t. this smaller set is
% models with constraints on the parameter matrices). J gives the Jacobian of % sought (for instance to fit structural models, distributed lags, and other
% the change of variables and can have r^2·p, r^2·(p+q) or r^2·(p+q) + % models with constraints on the parameter matrices). J gives the Jacobian of
% r·(r+1)/2 rows, for when A, A and B, or A, B and Sig depend on a smaller % the change of variables and can have r^2·p, r^2·(p+q) or r^2·(p+q) +
% set, respectively (the column count of J equals the number of variables in % r·(r+1)/2 rows, for when A, A and B or A, B and Sig depend on a smaller set,
% the smaller set). Thus the possible variable changes are: theta --> A, theta % respectively (the column count of J equals the number of variables in the
% --> [A B], or, theta --> [A B vech(Sig)]. % smaller set). Thus the possible variable changes are theta --> A, theta -->
% % [A B] and theta --> [A B vech(Sig)]. The only possibilities for mu are to
% The so-called "Cholesky decomposition" method is used, see [1] and [2]. % leave it free or to fix it to be zero. Letting mu = [] results in LL
% % containing the likelihood of a zero mean model and in LLD excluding
% [1] K Jonasson and SE Ferrando 2006. Efficient likelihood evaluation for % derivatives w.r.t. mu.
% VARMA processes with missing values. Report VHI-01-2006, Engineering %
% Research Institute, University of Iceland. % The so-called "Cholesky decomposition" method is used, see [1] and [2].
% %
% [2] K Jonasson 2006. Matlab programs for complete and incomplete data exact % [1] K Jonasson and SE Ferrando 2006. Efficient likelihood evaluation for
% VARMA likelihood and its gradient. Report VHI-02-2006, Engineering % VARMA processes with missing values. Report VHI-01-2006, Engineering
% Research Institute, University of Iceland. % Research Institute, University of Iceland.
% %
% Kristján Jónasson, Dept. of Computer Science, University of Iceland, 2006. % [2] K Jonasson 2006. Matlab programs for complete and incomplete data exact
% jonasson@hi.is. % VARMA likelihood and its gradient. Report VHI-02-2006, Engineering
% Research Institute, University of Iceland.
function [ll, ok, out3] = varma_llc(X, A, B, Sig, J_code) %
code = ''; J = []; % Kristján Jónasson, Dept. of Computer Science, University of Iceland, 2006.
if nargin > 4, if ischar(J_code), code = J_code; else J = J_code; end, end % jonasson@hi.is.
FINDRES = isequal(code,'res');
DIFF = nargout == 3 && ~FINDRES; function [ll, ok, varargout] = varma_llm(X, A, B, Sig, mu, miss, J_code)
CHGVAR = nargin>=5 && ~FINDRES; code = ''; J = [];
[p, q, r, n] = get_dimensions(A, B, Sig, X); if nargin>6, if ischar(J_code), code = J_code; else J = J_code; end, end
ko = 0:r:r*n; FINDRES = any(strmatch(code, {'res','res_miss'}, 'exact'));
x = X(:); CHNGVAR = ~isempty(J);
ll = 0; out3 = 0; MEAN = ~isempty(mu);
PLU = vyw_factorize(A); LowTri = struct('LT', true);
vyw_ok = isempty(PLU) || isempty(PLU{1}) || PLU{1}(1)~=0; [p, q, r, n] = get_dimensions(A, B, Sig, X);
ok = vyw_ok && is_stationary(A, Sig, PLU); nPar = r^2*(p+q) + r*(r+1)/2;
if ~ok, [ko,ro,km] = find_missing_info(miss);
if nargout<=1, error('Non-stationary model'); else return, end if MEAN, mubar = repmat(mu, n, 1); else mubar = zeros(n*r, 1); end
end x = X(:);
if nargout <= 2 || FINDRES % ONLY FUNCTION VALUE xo = x(~miss) - mubar(~miss);
[C, G, W] = find_CGW (A, B, Sig); nObs = length(xo);
S = vyw_solve (A, PLU, G); ll = 0; varargout = {0,0};
[Su, Olow] = omega_build (S, G, W, p, n); ll = 0; varargout{1} = 0;
[Lu, Ll, info] = omega_factor (Su, Olow, p, q, ko); assert(info==0); PLU = vyw_factorize(A);
w = lambda_multiply (A, x, false(r, n)); vyw_ok = isempty(PLU) || isempty(PLU{1}) || PLU{1}(1)~=0;
z = omega_forward (Lu, Ll, w, p, q, ko); ok = vyw_ok && is_stationary(A, Sig, PLU);
ld = omega_logdet (Lu, Ll, p, q, ko); if ~ok, if nargout<=1, error('Non-stationary model'); else return, end, end
%
ll = -(r*n*log(2*pi) + z'*z + ld)/2; if nargout<=2 || FINDRES % GRADIENT NOT REQUESTED
[C, G, W] = find_CGW (A, B, Sig);
if FINDRES S = vyw_solve (A, PLU, G);
eps = res_miss(A, C, Lu, Ll, w); Gneg = find_Gneg (A, B, C, n);
out3 = eps; Scol = S_extend (A, G, S, n);
end Acol = find_Acol (A, r);
else % FIND ALSO DERIVATIVES [Su, Olow] = omega_build (S, G, W, p, n);
[CCd,GGd,WWd] = find_CGW_deriv (A, B, Sig); [Suo, Olowo] = omega_remove_miss (Su, Olow, miss);
S = vyw_solve (A, PLU, GGd); [Luo, Llo, info] = omega_ltl (Suo,Olowo,p,q,ko); assert(info==0);
RHS = vyw_deriv_rhs (A, GGd, S); Laom = find_lambda_om (Acol, miss);
[G, Gd] = der2array (GGd); Laomh = profile_back_sub (Luo, Llo, Laom, ko, km, p, q, p);
[W, Wd] = der2array (WWd); clear Laom
if CHGVAR V = find_V (G, Gneg, Scol, miss);
RHS = chng_var (RHS, J); Vhat = profile_back_sub (Luo, Llo, V, ko, km, p, q, q);
Gd = chng_var (Gd, J); clear V
Wd = chng_var (Wd, J); wo = lambda_multiply (A, xo, miss);
end wohat = omega_back_sub (Luo, Llo, wo, p, q, ko);
Sd = vyw_solve (A, PLU, RHS); RLam = profile_ata (Laomh, ko, km, p, p);
[Su, Olow] = omega_build (S, G, W, p, n); P = profile_mult (Laomh, Vhat, ko, km, p, p, q);
[Sud, Olowd] = omega_build_deriv (Sd, Gd, Wd, p, n); Lw = profile_mult (Laomh, wohat, ko, km, p, p, n);
[Lu, Ll,info] = omega_factor (Su, Olow, p, q, ko); assert(info==0); if ~FINDRES
[Lud, Lld] = omega_factor_deriv (Sud, Olowd, Lu, Ll, p, q, ko); clear Laomh
xd = zeros (r*n, 1, r^2*(p+q)+r*(r+1)/2); end
[w, wd] = lambda_multiply (A, x, false(r, n), xd); RV = profile_ata (Vhat, ko, km, p, q);
if CHGVAR, wd = chng_var (wd, J); end Vw = profile_mult (Vhat, wohat, ko, km, p, q, n);
z = omega_forward (Lu, Ll, w, p, q, ko); if ~FINDRES
zd = omega_forward_deriv (Lu, Ll, Lud, Lld, z, wd, p, q, ko); clear Vhat
[logd,logdd] = omega_logdet (Lu, Ll, p, q, ko, Lud, Lld); end
Sm = find_Sm (Scol, miss);
ll = -(r*n*log(2*pi) + z'*z + logd)/2; SmP = Sm*P; clear P
lld = -z'*squeeze(zd) - logdd/2; R = -SmP-SmP' + atba_c (Sm, RLam, Sm + RV); clear Rlam
out3 = lld; LR = chol (R')'; clear R
end K = linsolve (LR, RV - SmP, LowTri); clear SmP
end Q = Sm - RV + K'*K; clear RV
LQ = chol (Q')'; clear Q
u = linsolve (LR, Vw - Sm*Lw, LowTri);
v = linsolve (LQ, Vw - K'*u, LowTri);
%
xSxo = wohat'*wohat - u'*u + v'*v;

7
 logdetO = omega_logdet(Luo, Llo, p, q, ko); % VAR_LL Vector-autoregressive likelihood and optionally its derivative
logdetLR = sum(log(diag(LR))); %
logdetLQ = sum(log(diag(LQ))); % [LL, OK] = VAR_LL(X, A, Sig) sets LL to the exact log-likelihood function
logdetSm = 2*sum(log(diag(chol(Sm)))); % value for n observations of the r-variate zero-mean VAR time series:
logdetSo = logdetO + 2*logdetLR + 2*logdetLQ - 2*logdetSm; %
% % x(t) = A1·x(t-1) + ... + Ap·x(t-p) + eps(t)
ll = -0.5*(nObs*log(2*pi) + logdetSo + xSxo); %
if FINDRES % where x(t) is r-dimensional and eps(t) is r-variate normal with mean 0 and
[eps,xm] = res_miss(A,C,Luo,Llo,wohat,LR,LQ,K,u,v,Laomh,Vhat,Sm,miss); % covariance Sig. Sig and the Ai's are r×r matrices. A should contain the r ×
varargout{1} = eps; % r·p matrix [A1 A2...Ap] and X should have x(t) in its t-th column for t =
if nargout>=4, varargout{2} = xm + mubar(miss); end % 1,...,n. To use this likelihood routine to fit a non-zero-mean time-series,
end % the mean-vector of the observations may be subtracted from each x(t). OK =
else % FIND ALSO GRADIENT % true indicates success, but is false if the model is non-stationary.
if MEAN, xod = xmu_deriv (nPar + r, miss); %
else xod = zeros (nObs, 1, nPar); end % [LL, OK] = VAR_LL(X, A, Sig, mu, miss) is used when there are missing
[wo, wod] = lambda_multiply (A, xo, miss, xod); % values. In that case it is appropriate to let the mean-vector of the series,
[CCd, GGd, WWd] = find_CGW_deriv (A, B, Sig); % mu, be a parameter, and use the model:
[C, Cd] = der2array (CCd); %
[G, Gd] = der2array (GGd); % x(t) - mu = A1·(x(t-1) - mu) + ... + Ap·(x(t-p) - mu) + eps(t)
[W, Wd] = der2array (WWd); %
S = vyw_solve (A, PLU, GGd); % The r×n logical array miss should be true in missing locations. An empty mu
RHS = vyw_deriv_rhs (A, GGd, S); % is equivalent to a zero mu.
Sd = vyw_solve (A, PLU, RHS); %
[Gneg, Gnegd] = find_Gneg (A, B, C, n, Cd); % [LL, OK, EPS] = VAR_LL(..., 'res') returns the maximum likelihood estimate
[Scol, Scold] = S_extend (A, G, S, n, Gd, Sd); % of the residuals in EPS and [LL, OK, EPS, XM] = VAR_LL(...,'res_miss') finds
[Acol, Acold] = find_Acol (A, r, nPar); % in addition the maximum likelihood estimate of the missing values in XM.
if CHNGVAR %
wod = chng_var(wod, J); % [LL, OK, LLD] = VAR_LL(X, A, Sig) and [LL, OK, LLD] = VAR_LL(X, A, Sig, mu,
Gd = chng_var(Gd, J); % miss) return the log-likelihood function value in LL and its gradient in
Wd = chng_var(Wd, J); % LLD. If mu is empty (or not present), LL is as if mu were zero, but LLD is
Sd = chng_var(Sd, J); % without mu-derivatives.
Gnegd = chng_var(Gnegd, J); %
Scold = chng_var(Scold, J); % [LL,OK,LLD] = VAR_LL(...,J) may be used to speed up the gradient calculation
Acold = chng_var(Acold, J); % when A and/or Sig depend on a smaller set of independent variables and only
end % the gradient w.r.t. this smaller set is sought. J gives the Jacobian of the
[Su, Olow] = omega_build (S, G, W, p, n); % variable change, and should have either r^2·p rows or r^2·p + r·(r+1)/2
[Sud, Olowd] = omega_build_deriv (Sd, Gd, Wd, p, n); % rows. In the latter case both A and Sig depend on a smaller set, but in the
[Su,Olow,Sud,Olowd] = omega_remove_miss (Su, Olow, miss, Sud, Olowd); % former case it is only A that does. Thus the possible variable changes are
[Lu,Ll,flg,Lud,Lld] = omega_ltl (Su, Olow, p, q, ko, Sud, Olowd); % theta --> A and theta --> [A vech(Sig)]. See also help text of varma_llc.
clear Su Sud Olow Olowd %
[Laom, Laomd] = find_lambda_om (Acol, miss, Acold); % Lambda_om % The so-called "Cholesky decomposition" method is used, see [1] and [2].
[Laomh, Laomhd] = profile_back_sub (Lu, Ll, Laom, ko, km, p, q, p... %
, Lud, Lld, Laomd); % [1] K Jonasson and SE Ferrando 2006. Efficient likelihood evaluation for
clear Laom Laomd % VARMA processes with missing values. Report VHI-01-2006, Engineering
[V, Vd] = find_V (G, Gneg, Scol, miss, Gd,Gnegd,Scold); % Research Institute, University of Iceland.
[Vhat, Vhatd] = profile_back_sub (Lu, Ll, V, ko, km, p, q, q... %
, Lud, Lld, Vd); % [2] K Jonasson 2006. Matlab programs for complete and incomplete data exact
clear V Vd % VARMA likelihood and its gradient. Report VHI-02-2006, Engineering
[RLam, RLamd] = profile_ata (Laomh, ko, km, p, p, Laomhd); % Research Institute, University of Iceland.
[P, Pd] = profile_mult (Laomh, Vhat, ko, km, p, p, q... %
, Laomhd, Vhatd); % Kristján Jónasson, Dept. of Computer Science, University of Iceland, 2006.
if MEAN % jonasson@hi.is.
Laomhd = add_mu_deriv (r, Laomhd);
[Lud, Lld] = add_mu_deriv (r, Lud, Lld); function [ll, ok, varargout] = var_ll(X, A, Sig, varargin)
end if nargin>3 && ischar(varargin{end}), code = varargin{end}; else code = '';end
% MISS = nargin >=5 && ~iscell(varargin{1}) && islogical(varargin{2});
[woh, wohd] = omega_back_sub (Lu, Ll, wo, p, q, ko, Lud, Lld, wod); FINDRES = any(strmatch(code, {'res','res_miss'}, 'exact'));
[Lw, Lwd] = profile_mult (Laomh, woh, ko,km,p,p,n,Laomhd,wohd); FNDGRAD = nargout == 3 && ~FINDRES; % find gradient
clear Laomh Laomhd CHNGVAR = FNDGRAD && (MISS && nargin>=6 || ~MISS && nargin>=4);
[RV,RVd] = profile_ata (Vhat, ko, km, p, q, Vhatd); LowTri.LT = true;
if MEAN [p, q, r, n] = get_dimensions(A, [], Sig, X); % assert(p>0);
Vhatd = add_mu_deriv (r, Vhatd); nPar = r^2*p + r*(r+1)/2;
end if ~MISS
[Vw, Vwd] = profile_mult (Vhat, woh, ko, km, p,q,n,Vhatd,wohd); if nargin > 3 && islogical(varargin{1}), error('wrong arguments'), end
clear Vhat Vhatd MEAN = false;
[Sm, Smd] = find_Sm (Scol, miss, Scold); mu = zeros(r,1);
[SmP, SmPd] = atb_deriv (Sm, Smd, P, Pd); clear P Pd miss = false(r,n);
R = Sm + RV - SmP - SmP'; if CHNGVAR, J = varargin{1}; end
Rd = Smd + RVd - SmPd - permute(SmPd,[2,1,3]); else
[R, Rd] = atba_c (Sm, RLam, R, Smd... [mu, miss] = deal(varargin{1:2});
, RLamd, Rd); clear RLam RLamd mu=mu(:);
[LR, LRd] = chol_deriv (R, Rd); clear R Rd MEAN = ~isempty(mu);
[K, Kd] = forward_sub_deriv (LR, LRd, RV-SmP... if ~MEAN, mu = zeros(r,1); end
, RVd-SmPd); clear SmP SmPd if CHNGVAR, J = varargin{3}; end
Q = Sm - RV + K'*K; clear RV end
Qd = ata_deriv (K, Kd) + Smd - RVd; clear RVd [ko,ro,km] = find_missing_info(miss);
[LQ, LQd] = chol_deriv (Q, Qd); clear Q Qd mubar = repmat(mu, n, 1);
if MEAN X = X(:);
[Smd,LRd,LQd,Kd] = add_mu_deriv (r, Smd, LRd, LQd, Kd); xo = X(~miss) - mubar(~miss);
end nObs = length(xo);
VwSmLw = Vw - Sm'*Lw; if nObs == n*r, MISS = false; end
VwSmLwd = Vwd - atb_deriv (Sm, Smd, Lw, Lwd); ll = 0;
[u, ud] = forward_sub_deriv (LR, LRd, VwSmLw, VwSmLwd); varargout = {0};
VwKu = Vw - K'*u; PLU = vyw_factorize(A);
VwKud = Vwd - atb_deriv (K, Kd, u, ud); vyw_ok = isempty(PLU) || isempty(PLU{1}) || PLU{1}(1)~=0;
[v, vd] = forward_sub_deriv (LQ, LQd, VwKu, VwKud); ok = vyw_ok && is_stationary(A, Sig, PLU);
% if ~ok, if nargout<=1, error('Non-stationary model'); else return, end, end
xSxo = woh'*woh - u'*u + v'*v; %
xSxod = 2*(woh'*Jac(wohd) - u'*Jac(ud) + v'*Jac(vd)); if ~FNDGRAD || FINDRES % ONLY FUNCTION VALUE
% S = vyw_solve (A, PLU, {Sig});
[ldO, ldOd] = omega_logdet(Lu, Ll, p, q, ko, Lud, Lld); [Lu,LSig,jpat] = omega_ar_factor (S, Sig, miss);
[ldLR, ldLRd] = logdet_L(LR, LRd); wo = lambda_multiply (A, xo, miss);
[ldLQ, ldLQd] = logdet_L(LQ, LQd); wohat = omega_ar_trisolve (Lu, LSig, wo, jpat, ko, 'NoT');
[LSm, LSmd] = chol_deriv(Sm, Smd); logdetO = omega_ar_logdet (Lu, LSig, jpat);
[ldSm, ldSmd] = logdet_L(LSm, LSmd); if MISS
ldSo = ldO + 2*ldLR + 2*ldLQ - 4*ldSm; Gneg = find_Gneg (A, [], {Sig}, n);
ldSod = ldOd + 2*ldLRd + 2*ldLQd - 4*ldSmd; Scol = S_extend (A, {Sig}, S, n);
% Acol = find_Acol (A, r);
ll = -0.5*(nObs*log(2*pi) + ldSo + xSxo); Laom = find_lambda_om (Acol, miss); % Lambda_om
lld = -0.5*(ldSod + xSxod); Laomh = profile_ar_trisolve (Lu, LSig, Laom, jpat, ko, km,p,p,'N');
varargout{1} = lld; clear Laom
end V = find_V ({Sig}, Gneg, Scol, miss);
end Vhat = profile_ar_trisolve (Lu, LSig, V, jpat, ko, km, p, q, 'N');
clear V
function J = Jac(xd) % Change 3-dim derivative to Jacobian matrix Rlam = profile_ata (Laomh, ko, km, p, p);
J = permute(xd, [1,3,2]); P = profile_mult (Laomh, Vhat, ko, km, p, p, q);
end Lw = profile_mult (Laomh, wohat, ko, km, p, p, n);
if ~FINDRES,
clear Laomh
end
RV = profile_ata (Vhat, ko, km, p, q);
Vw = profile_mult (Vhat, wohat, ko, km, p, q, n);
if ~FINDRES
clear Vhat
end
Sm = find_Sm (Scol, miss);
SmP = Sm*P; clear P
R = -SmP-SmP' + atba_c (Sm, Rlam, Sm + RV); clear Rlam
LR = chol (R')'; clear R

8
 K = linsolve (LR, RV - SmP, LowTri); clear SmP ll = -0.5*(nObs*log(2*pi) + ldSo + xSxo);
Q = Sm - RV + K'*K; clear RV lld = -0.5*(ldSod + xSxod);
LQ = chol (Q')'; clear Q varargout{1} = lld;
u = linsolve (LR, Vw - Sm*Lw, LowTri); end
v = linsolve (LQ, Vw - K'*u, LowTri); end
xSxo = wohat'*wohat - u'*u + v'*v;
logdetLR = sum (log(diag(LR))); function J = Jac(xd) % Change 3-dim derivative to Jacobian matrix
logdetLQ = sum (log(diag(LQ))); J = permute(xd, [1,3,2]);
logdetSm = 2*sum (log(diag(chol(Sm)))); end
logdetSo = logdetO + 2*logdetLR + 2*logdetLQ - 2*logdetSm;
else
xSxo = wohat'*wohat; % VARMA_SIM Simulate an ARMA or a VARMA time series model
logdetSo = logdetO; %
LR = []; LQ = []; K = []; u = []; v = []; Laomh = []; Vhat = []; Sm = []; % ARMA MODELS:
end % X = VARMA_SIM(A,B,sigma,n) with scalar sigma generates a zero-mean time
ll = -0.5*(nObs*log(2*pi) + logdetSo + xSxo); % series of length n using the ARMA(p,q) model:
if FINDRES && MISS %
[eps,xm]=res_miss_ar(A,Sig,Lu,LSig,wohat,LR,LQ,K,u,v,Laomh,Vhat,Sm,miss); % x(t) = A1·x(t-1) + ... + Ap·x(t-p) + y(t) (1a)
varargout = {eps, xm + mubar(miss)}; % where:
elseif FINDRES % y(t) = eps(t) + B1·eps(t-1) + ... + Bq·eps(t-q) (1b)
[eps,xm] = res_miss_ar(A,Sig,Lu,LSig,wohat); %
varargout = {eps, xm}; % and eps(t) is N(0,sigma). A should be the row vector [A1,...,Ap] and B
end % should be [B1,...,Bq]. X returns the series as a row vector. To start the
% simulation, x(1),...,x(g),eps(1),...,eps(g) are drawn from N(0,SCE) where
else % FIND ALSO GRADIENT % SCE = [SS CC; CC' EE] is obtained by solving the modified Yule-Walker
if MEAN % equations (see vyw_factorize, vyw_solve and S_build), so there is no need
xod = xmu_deriv(nPar + r, miss); % to throw away the first values to avoid spin-up effects. The model must be
else % stationary.
xod = zeros(nObs, 1, nPar); %
end % X = VARMA_SIM(A,B,sigma,n,mu) returns a time series with mean mu using
[wo, wod] = lambda_multiply (A, xo, miss, xod); % y(t) as above, and x(t) given by:
SigS = mds_set_parmat (p+1, Sig, p+1); %
Sigd = der2array (SigS); % x(t) - mu = A1·(x(t-1) - mu) + ... + Ap·(x(t-p) - mu) + y(t) (2)
[C,G,Cd,Gd] = deal ({Sig}, {Sig}, Sigd, Sigd); %
S = vyw_solve (A, PLU, {Sig}); % In this case x and eps are drawn from N([mu; 0], SCE).
RHS = vyw_deriv_rhs (A, SigS, S); %
Sd = vyw_solve (A, PLU, RHS); % X = VARMA_SIM(A,B,sigma,mu,M) generates M sequences simultaneously and
if MISS % returns the i-th one in the i-th row of X. Use mu=[] to obtain zero-mean.
[Gneg, Gnegd] = find_Gneg (A, [], C, n, Cd); %
[Scol, Scold] = S_extend (A, G, S, n, Gd, Sd); % X = VARMA_SIM(A,B,sigma,n,mu,M,X0) sets x(1)...x(g) to X0(end-p+1:end) -
[Acol, Acold] = find_Acol (A, r, nPar); % mu and draws eps(1)...eps(g) from the conditional distribution of eps(1),
end % ..., eps(g) given x(1),...,x(g)). For this option the model need not be
if CHNGVAR % stationary.
wod = chng_var(wod, J); %
Sigd = chng_var(Sigd, J); % [X,EPS] = VARMA_SIM(...) returns also the shock series, eps(t), in the
Gd = chng_var(Gd, J); % t-th row of EPS.
Sd = chng_var(Sd, J); %
if MISS % VARMA MODELS:
Gnegd = chng_var(Gnegd, J); % X = VARMA_SIM(A,B,Cov,n), where Cov is an r×r matrix uses a VARMA(p,q)
Scold = chng_var(Scold, J); % model given by (1), but now x(t) is r-dimensional, eps(t) is r-variate
Acold = chng_var(Acold, J); % normal with mean 0 and covariance Cov, and Cov and the Ai's and Bi's are
end % r×r matrices. A and B should contain A = [A1 A2...Ap] (an r × r·p matrix)
end % and B = [B1 B2 ... Bq] (an r × r·q matrix). The r×n matrix X returns x(t)
[Lu,LSig,jpat,Lud,LSigd] = omega_ar_factor(S, Sig, miss, Sd, Sigd{1}); % in its t-th column. As in the scalar case the simulated series is spin-up
if MISS % free, the starting values x(1),...,x(g) and eps(1),...,eps(g) being drawn
[Laom, Laomd] = find_lambda_om (Acol, miss, Acold); % Lambda_om % from the correct distribution. The model must be stationary.
[Laomh,Laomhd] = profile_ar_trisolve (Lu, LSig, Laom, jpat, ko, km, p... %
, p, 'N', Lud, LSigd, Laomd); % X = VARMA_SIM(...,mu) uses (2) for x(t) instead of (1a).
clear Laom Laomd %
[V, Vd] = find_V (G, Gneg,Scol,miss,Gd,Gnegd,Scold); % X = VARMA_SIM(...,mu,M) returns M sequences in an r×n×M multidimensional X
[Vhat, Vhatd] = profile_ar_trisolve (Lu, LSig, V, jpat, ko, km, p, q... % (when r > 1) with the i-th sequence in X(1:r, 1:n, i). Use mu = [] to
, 'N', Lud, LSigd, Vd); % obtain zero-mean.
clear V Vd %
end % X = VARMA_SIM(...,mu,M,X0) initializes x(1),...,x(h) with the last h
if MEAN, [Lud,LSigd] = add_mu_deriv (r, Lud, LSigd); end % columns of X0 instead of drawing from N(mu,SS). The model need not be
[woh, wohd] = omega_ar_trisolve (Lu,LSig,wo,jpat,ko,'N',Lud,LSigd,wod); % stationary.
[ldO, ldOd] = omega_ar_logdet (Lu, LSig, jpat, Lud, LSigd); %
if MISS % [X,EPS] = VARMA_SIM(...) returns also the shock series; the i-th eps(t) is
[Rlam, Rlamd] = profile_ata (Laomh, ko, km, p, p, Laomhd); % returned in EPS(:, t, i).
[P, Pd] = profile_mult (Laomh, Vhat,ko,km,p,p,q,Laomhd,Vhatd); %
if MEAN % For both ARMA and VARMA, use VARMA_SIM(A,[],...) for a pure autoregressive
Laomhd = add_mu_deriv (r, Laomhd); % model, and VARMA_SIM([],B,...) for a pure moving average model.
end %
[Lw, Lwd] = profile_mult (Laomh, woh, ko, km,p,p,n,Laomhd,wohd); % The method used is described in [1] and [2].
clear ('Laomh', 'Laomhd') %
[RV, RVd] = profile_ata (Vhat, ko, km, p, q, Vhatd); % [1] K Jonasson and SE Ferrando 2006. Efficient likelihood evaluation for
if MEAN % VARMA processes with missing values. Report VHI-01-2006, Engineering
Vhatd = add_mu_deriv (r, Vhatd); % Research Institute, University of Iceland.
end %
[Vw, Vwd] = profile_mult (Vhat, woh, ko, km, p, q,n,Vhatd,wohd); % [2] K Jonasson 2006. Matlab programs for complete and incomplete data exact
clear ('Vhat', 'Vhatd') % VARMA likelihood and its gradient. Report VHI-02-2006, Engineering
[Sm, Smd] = find_Sm (Scol, miss, Scold); % Research Institute, University of Iceland.
[SmP, SmPd] = atb_deriv (Sm, Smd, P, Pd); clear P Pd %
R = Sm + RV - SmP - SmP'; % Kristján Jónasson, Dept. of Computer Science, University of Iceland, 2006.
Rd = Smd + RVd - SmPd - permute (SmPd,[2,1,3]); % jonasson@hi.is.
[R, Rd] = atba_c (Sm, Rlam, R, Smd...
, Rlamd, Rd); clear RLam RLamd function [x, eps] = varma_sim(A, B, Sig, n, mu, M, x0)
[LR, LRd] = chol_deriv (R, Rd); r = size(Sig, 1);
[K, Kd] = forward_sub_deriv (LR, LRd, RV-SmP... if isempty(A), A = zeros(r,0); end
, RVd-SmPd); clear SmP SmPd if isempty(B), B = zeros(r,0); end
Q = Sm - RV + K'*K; clear RV p = size(A, 2)/r;
Qd = ata_deriv (K, Kd) + Smd - RVd; clear RVd q = size(B, 2)/r;
[LQ, LQd] = chol_deriv (Q, Qd); clear Q Qd Aflp = reshape(flipdim(reshape(A,r,r,p), 3), r, r*p); % [Ap...A2 A1]
if MEAN Bflp = reshape(flipdim(reshape(B,r,r,q), 3), r, r*q); % [Bq...B2 B1]
[Smd, Kd] = add_mu_deriv (r, Smd, Kd); h = max(p,q);
[LRd, LQd] = add_mu_deriv (r, LRd, LQd); if n<h, error('Too short series'); end
end if nargin < 5 || isempty(mu), mu = zeros(r,1); else mu = mu(:); end
VwSmLw = Vw - Sm'*Lw; if nargin < 6 || isempty(M), M=1; end
VwSmLwd = Vwd - atb_deriv (Sm, Smd, Lw, Lwd); if nargin < 7, x0 = []; end
[u, ud] = forward_sub_deriv (LR, LRd, VwSmLw, VwSmLwd); I = 1:r*h;
VwKu = Vw - K'*u; x = zeros(n*r,M);
VwKud = Vwd - atb_deriv (K, Kd, u, ud); eps = zeros(n*r,M);
[v, vd] = forward_sub_deriv (LQ, LQd, VwKu, VwKud); mup = repmat(mu,h,1);
% [C, G, W] = find_CGW(A, B, Sig);
xSxo = woh'*woh - u'*u + v'*v; PLU = vyw_factorize(A);
xSxod = 2*(woh'*Jac(wohd) - u'*Jac(ud) + v'*Jac(vd)); if ~isempty(PLU) && ~isempty(PLU{1}) && PLU{1}(1) == 0,
% error('Non-stationary model: X0 must be specified.'),
[ldLR, ldLRd] = logdet_L (LR, LRd); end
[ldLQ, ldLQd] = logdet_L (LQ, LQd); S = vyw_solve(A, PLU, G);
[LSm, LSmd] = chol_deriv (Sm, Smd); SS = S_build(S, A, G, h);
[ldSm, ldSmd] = logdet_L (LSm, LSmd); CC = CC_build(A, C, h);
ldSo = ldO + 2*ldLR + 2*ldLQ - 4*ldSm; [R,pp] = chol(SS);
ldSod = ldOd + 2*ldLRd + 2*ldLQd - 4*ldSmd; if pp~=0, error('Non-stationary model: X0 must be specified.'), end
else EE = mat2cell(zeros(r*h,r*h), repmat(r,1,h), repmat(r,1,h));
xSxo = woh'*woh; for i=1:h, EE{i,i} = Sig; end
xSxod = 2*(woh'*Jac(wohd)); EE = cell2mat(EE);
ldSo = ldO; if isempty(x0) % Start the sequence from scratch
ldSod = ldOd; x(I,:) = randnm(M, SS)';
end else % Initialize the series with x0 (every generated series starts the same)

9
 assert(length(x0)>=h);
x(I,:) = repmat(reshape(x0(:,end-h+1:end),[],1) - mup, 1, M);
end % ATBA_C Calculate A'·B·A + C and optionally its derivatives
eps(I,:) = randnm(M, EE - CC'*(SS\CC))' + CC'*(SS\x(I,:)); %
eps(r*h+1:end,:) = reshape(randnm((n-h)*M,Sig)', [], M); % shocks % D = ATBA_C(A,B,C) calculates D = A'·B·A + C. B and C are assumed to be
% eps = reshape(randnm(n*M,Sig)', [], M); % for testing % symmetric, only their lower triangle is used (and only the lower triangle of
Ip = r*(h-p)+1:r*h; % D is guaranteed to be correct).
Iq = r*(h-q)+1:r*h; %
I = r*h+1:r*h+r; % [D,Dd] = ATBA_C(A,B,C,Ad,Bd,Cd) calculates also the derivative w.r.t.
for t=h+1:n % Generate the series % parameter theta(i) in Dd(:,:,i), i = 1,...,nPar. Ad(:,:,i), Bd(:,:,i) and
x(I,:) = eps(I,:) + Aflp*x(Ip,:) + Bflp*eps(Iq,:); % Cd(:,:,i) should be the derivatives of A, B and C w.r.t. theta(i).
I = I+r;
Ip = Ip+r; function [D, Dd] = atba_c(A, B, C, Ad, Bd, Cd)
Iq = Iq+r; DIFF = nargin > 3;
end if ~DIFF
if r==1 && M==1 % one ARMA sequence: L = tril(B);
x = reshape(x,1,n) + mu; for i=1:size(B,1), L(i,i) = L(i,i)/2; end
eps = reshape(eps,1,n); ALA = A'*L*A;
elseif r==1 % several ARMA sequences: D = ALA + ALA' + C;
x = reshape(x,n,M) + mu; else
eps = reshape(eps,n,M); [m,n,nPar] = size(Bd);
else % one or more VARMA sequences in r×n×M array: L = tril(B);
x = reshape(x,r,n,M) + repmat(mu,[1,n,M]); Ld = Bd;
eps = reshape(eps,r,n,M); for i=1:m
end L(i,i,:) = L(i,i,:)/2;
end Ld(i,i,:) = Ld(i,i,:)/2;
Ld(1:i-1,i,:) = 0;
function x = randnm(n,Sig,mu); end
% RANDNM Multivariate normal random vectors [LA, LAd] = atb_deriv(L,Ld,A,Ad);
% X = RANDNM(N,SIG,MU) generates N random vectors from an r-variate normal [ALA,ALAd] = atb_deriv(A,Ad,LA,LAd);
% distribution with mean MU and covariance matrix SIG. MU should be an D = ALA + ALA' + C;
% r-vector. SIG is assumed to be positive semidefinite. It is factorized Dd = Cd + ALAd + permute(ALAd,[2,1,3]);
% using [R,p] = chol(SIG) and thus no check is made for that fact. X will end
% be N×r. X = RANDNM(N,S) uses mean 0. end
%
% NOTE: The formula for del was found empirically to be approximately the
% minimum necessary to ensure that Cholesky factorization of Sig+del*I % ATB_C Calculate A'·B + C and optionally its derivatives
% works. The while loop is to further ensure that. %
r=size(Sig,1); % D = ATB_C(A,B,C) calculates D = A'·B + C.
if nargin<3, mu=zeros(1,r); end %
mu = mu(:)'; % [D,Dd] = ATB_C(A,B,C,Ad,Bd,Cd) calculates also the derivative w.r.t.
[R,p] = chol(Sig); % parameter theta(i) in Dd(:,:,i), i = 1,...,nPar. Ad(:,:,i), Bd(:,:,i) and
if p~=0, % Add delta to diagonal of Sig to handle postive semidefinite Sig % Cd(:,:,i) should be the derivatives of A, B and C w.r.t. theta(i).
del = max(1,norm(diag(Sig),inf))*(3+r/50)*eps; % eps is machine epsilon %
kdel = 0; % Dd = ATB_C(A,B,Ad,Bd,Cd) calculates only the derivatives.
while p~=0 %
Sig = Sig + del*eye(r); % When C is zero use ATB_C(A,B), and ATB_C(A,B,Ad,Bd)
kdel = kdel + 1; %
[R,p] = chol(Sig); % METHOD: Dd(:,:,i) = Ad(:,:,i)'*B + A'*Bd(:,:,i) + Cd.
end
assert(kdel<10); function [varargout] = atb_c(A, B, varargin)
end DIFF = nargin > 3;
x = randn(n,r)*R + repmat(mu,n,1); if DIFF
end switch nargin
case 4, [Ad,Bd] = deal(varargin{:}); C = []; Cd = [];
case 5, [Ad,Bd,Cd] = deal(varargin{:}); C = [];
% AAT_DERIV Calculate A'·A and its derivative case 6, [C,Ad,Bd,Cd] = deal(varargin{:});
% otherwise error('illegal call');
% [C, Cd] = AAT_DERIV(A, Ad) calculates C = A·A' and the derivative of C with end
% respect to parameter theta(i) in Cd(:,:,i), i = 1,...,nPar. Ad(:,:,i) should [k,m] = size(A);
% be the derivatives of A w.r.t. theta(i). n = size(B,2);
% nPar = size(Ad,3);
% Use Cd = AAT_DERIV(...) when only the derivative is needed. Dd = reshape(A'*reshape(Bd,k,n*nPar), m, n, nPar) + ...
permute(reshape(B'*reshape(Ad,k,m*nPar), n, m, nPar),[2,1,3]);
function [varargout] = aat_deriv(A, Ad) if ~isempty(Cd), Dd = Dd + Cd; end
[m,n] = size(A); varargout{nargout} = Dd;
nPar = size(Ad,3); end
Cd = zeros(m,m,nPar); if ~DIFF || nargout==2
for i = 1:nPar if nargin==3 || nargin==6, C = varargin{1}; else C = []; end
M = A*Ad(:,:,i)'; Cd(:,:,i) = M + M'; D = A'*B;
end if ~isempty(C), D = D + C; end
if nargout == 1, varargout{1} = D;
varargout{1} = Cd; end
else end
[varargout{1:2}] = deal(A*A', Cd);
end
end % ATB_DERIV Calculate A'·B and its derivative
%
% [C, Cd] = ATB_DERIV(A, Ad, B, Bd) calculates C = A'·B and the derivative of C
% ADD_MU_DER Add derivatives with respect to mu % with respect to parameter theta(i) in Cd(:,:,i), i = 1,...,nPar. Ad(:,:,i)
% % and Bd(:,:,i) should be the derivatives of A and B w.r.t. theta(i).
% [Fd1, Fd2...] = ADD_MU_DER(r, Fd1, Fd2...) modifies the derivatives Fd1, %
% Fd2,... from being w.r.t. theta to being w.r.t. theta and mu, by % Use Cd = ATB_DERIV(A, Ad, B, Bd) when only the derivative is needed.
% increasing the 3rd dimension of each Fdj by r and filling the new elements %
% with zeros. An Fdi may also be a cell array, and then each cell is modified. % METHOD: Cd(:,:,i) = Ad(:,:,i)'*B + A'*Bd(:,:,i).

function [varargout] = add_mu_deriv(r, varargin) function [varargout] = atb_deriv(A, Ad, B, Bd)

for j = 1:length(varargin) [k,m] = size(A);
if iscell(varargin{j}) n = size(B,2);
for i=1:length(varargin{j}) nPar = size(Ad,3);
varargout{j}{i} = add_mu_deriv(r, varargin{j}{i}); BA = reshape(B'*reshape(Ad,k,m*nPar), n, m, nPar);
end BA = permute(BA, [2,1,3]);
else Cd = reshape(A'*reshape(Bd,k,n*nPar), m, n, nPar) + BA;
[m,n,k] = size(varargin{j}); if nargout == 1,
varargout{j} = cat(3, varargin{j}, zeros(m,n,r)); varargout{1} = Cd;
end else
end [varargout{1:2}] = deal(A'*B, Cd);
end end
end

% ATA_DERIV Calculate A'·A and its derivative

% % BACK_SUB_DERIV Derivative of back substitution
% [C, Cd] = ATA_DERIV(A, Ad) calculates C = A'·A and the derivative of C with %
% respect to parameter theta(i) in Cd(:,:,i), i = 1,...,nPar. Ad(:,:,i) should % Xd = BACK_SUB_DERIV(L, Ld, X, Yd) finds the derivative of the solution X to
% be the derivatives of A w.r.t. theta(i). % an upper triangular system L'·X = Y. Ld and Yd should have derivatives of L
% % and Y, and Xd returns the derivative of X.
% Use Cd = ATA_DERIV(A,Ad) when only the derivative is needed. %
% METHOD: From L'·X = Y it follows that Ld(:,:,i)'·X + L'·Xd(:,:,i) = Yd, and
function [varargout] = ata_deriv(A, Ad) % thus Xd may be obtained with back substitution.
[m,n,nPar] = size(Ad);
M = A'*reshape(Ad,m,n*nPar); function Xd = back_sub_deriv(L, Ld, X, Yd)
Cd = reshape(M, n, n, nPar); LTT.LT = true; LTT.TRANSA = true;
Cd = Cd + permute(Cd,[2,1,3]); [n, m] = size(X);
if nargout == 1, nPar = size(Ld,3);
varargout{1} = Cd; LdX = permute(reshape(reshape(Ld, [n,n*nPar])'*X, [n,nPar,m]), [1,3,2]);
else RHS = reshape(Yd - LdX, [n,m*nPar]);
varargout = {A'*A, Cd}; Xd = reshape(linsolve(L,RHS,LTT), [n,m,nPar]);
end end
end

10
% CC_BUILD Calculate covariance of data and shocks for k = min(q,t-1):-1:0
% Y(:,t-k) = Y(:,t-k) + C{k+1}(K,:)'*X(K1);
% CC = CC_BUILD(A,C,n) finds the covariance between the vector x = [x1'...xn']' end
% and the vector [eps'...eps(n)']'. A is [A1...Ap] and C is {C0 C1...Cq}. end
% Used by varma_sim. Y = reshape(Y, n*r, 1);
end
function CC = CC_build(A, C, n)
r = size(C{1},1);
q = length(C) - 1; % DER2ARRAY Convert derivative from cell format to array format
p = size(A,2)/r; %
Ac = makecell(A); % [F,Fd] = DER2ARRAY(FS) extracts the cell arrays F and Fd from the struct
for j=q+1:p % array FS. F{i} is r×r and Fd{i} is r×r×NPAR with Fd{i}(:,:,m) containing the
C{j+1} = zeros(r,r); % derivative of F{i} with respect to the m-th parameter; NPAR = r^2·(p+q) +
for i=1:j, C{j+1} = C{j+1} + Ac{i}*C{j-i+1}; end % r·(r+1)/2.
end %
CC = cell(n,n); % The parameters are ordered by parameter matrix, A1,..., Ap, B1,..., Bq, Sig
for i=1:n % and within each parameter matrix by columns, with the upper triangle of Sig
for j=1:n % removed: A1(1,1), A1(2,1),..., Bq(r,r), Sig(1,1), Sig(r,1), Sig(2,2),...,
if 0<=i-j, CC{i,j} = C{i-j+1}; % Sig(r,r).
else CC{i,j} = zeros(r,r); end %
end % Apart from simple copying, DER2ARRAY adjusts for the fact that Sig is
end % symmetric. Derivatives w.r.t. mu (if present) are set to zero. FS is a
CC = cell2mat(CC); % structure array, the i-th element containing F{i} and its derivatives in a
end % format as described in mds_add_prod.
%
% Fd = der2array(FS) returns only the derivative.
% CHNG_VAR Postmultiply gradient with variable-change Jacobian
% function [F, Fd] = der2array(FS)
% FD = CHNG_VAR(FD, J) changes derivatives FD from being w.r.t. A (and/or B n = length(FS);
% and Sig) to being w.r.t. an alternative set, theta, by postmultiplying the r = size(FS(1).mat, 1);
% leading columns of FD with the Jacobian of the variable change, J. See nPar = (length(FS(1).der)-1)*r^2 + r*(r+1)/2;
% help-text of var_ll and varma_llc. F = cell(1,n);
if nargout>1
function Fd = chng_var(Fd, J) Fd = cell(1,n);
if iscell(Fd) for i = 1:n, [F{i},Fd{i}] = dercopy(FS(i), nPar); end
for i=1:length(Fd), Fd{i} = chng_var(Fd{i}, J); end else
else for i = 1:n, F{i} = dercopy(FS(i), nPar); end
[mJ, nJ] = size(J); end
[a,b,np] = size(Fd); end
Fd1 = reshape(Fd(:,:,1:mJ), a*b, mJ);
Fd2 = reshape(Fd(:,:,mJ+1:end), a*b, np-mJ); function [F, Fd] = dercopy(FS, nPar);
Fd = reshape([Fd1*J Fd2], a, b, nJ + np-mJ); r = size(FS.mat,1);
end nparmat = length(FS.der);
end Fd = zeros(r,r,nPar);
m = 0;
for k=1:nparmat-1 % Copy derivatives w.r.t. A and B matrices
% CHOL_DERIV Derivative of Cholesky factors for c=1:r
% for l=1:r
% Ld = CHOL_DERIV(L, Ad) is used to differentiate Cholesky factorization. If A m = m+1;
% = L*L' is the Cholesky factorization of an n×n matrix A and Ad(:,:,i) Fd(:,:,m) = FS.der{k}{l,c};
% contains derivatives of A with respect to parameter theta(i) for i=1,...,nPar end
% then Ld(:,:,i) will be the derivative of L with respect to theta(i). Only the end
% lower triangle of Ad(:,:,i) is used. end
% for c=1:r % Copy derivative w.r.t. Sigma, and adjust for symmetry
% [L, Ld] = CHOL_DERIV(A, Ad) finds the Cholesky factor of A in L and its m = m+1;
% derivative in Ld. Fd(:,:,m) = FS.der{nparmat}{c,c};
for l=c+1:r
function [varargout] = chol_deriv(varargin) m = m+1;
LT.LT = true; Fd(:,:,m) = FS.der{nparmat}{l,c} + FS.der{nparmat}{c,l};
if nargout == 2 end
[A, Ad] = deal(varargin{:}); end
L = chol(A)'; assert(m==nPar || m==nPar-r);
else if nargout>1
[L, Ad] = deal(varargin{:}); F = FS.mat;
end else
n = length(L); F = Fd;
nPar = size(Ad,3); end
Ld = zeros(n, nPar*n); end
for k=1:n
g = nPar*(k-1);
K = 1:k-1; % FIND_ACOL Create a block column of A-matrices and optionally derivatives
u = L(k,K); %
X = zeros(k-1,nPar); % Acol = FIND_ACOL(A, r) returns the matrix A1
blk = 15; % use blocking to exploit fast multiplication for larger matrices % :
j = 1; % Ap
for i=1:blk:k-1 %
m = min(blk, k-i); % [Acol, Acold] = FIND_ACOL(A, r, nPar) returns also the derivatives of Acol
ie = i+m-1; % w.r.t. the A matrices (taken column by column) and some additional nPar -
je = j+m*nPar-1; % r^2·p parameters (normally the columns of moving average parameter matrices
X(i:ie,:) = reshape(u(1:ie)*Ld(1:ie,j:je), nPar, m)'; % Bj and a covariance matrix Sig).
j = je+1;
end function [Acol, Acold] = find_Acol(A, r, nPar)
% X(:,i) now contains Ldi(K,K)*u' DIFF = nargout > 1;
X1 = reshape(Ad(k,K,:),k-1,nPar); Ac = makecell(A);
X = linsolve(L(K,K), X1-X, LT); p = length(Ac);
Ld(K, g+1:g+nPar) = X; if p==0
sd = reshape(Ad(k,k,:),1,nPar)/2; Acol = zeros(0, r);
Ld(k, g+1:g+nPar) = (sd - u*X)/L(k,k); else
end Acol= cat(1,Ac{:});
Ld = permute(reshape(Ld, n, nPar, n), [3,1,2]); end
if nargout == 2, varargout = {L, Ld}; else varargout = {Ld}; end if DIFF
end Acold = zeros(r*p, r, nPar);
k = 1;
for j = 1:p
% C_MULTIPLY Multiply vector by expanded C-matrix for c = 1:r
% for l = 1:r
% Y = C_MULTIPLY(C, X, n, miss) returns C*X in Y, where C is obtained from the Acold(l+r*(j-1),c,k) = 1;
% n·r×n·r lower block-band matrix: k = k+1;
% end
% C0 C1' C2' ... Cq' end
% C0 C1' C2' ... Cq' end
% ... ... end
% C0 C1' ... Cq' end
% ... :
% C0 C1' % FIND_CGW Determine C, G and W matrices for VARMA log-likelihood
% C0 %
% % [C,G,W] = FIND_CGW(A,B,Sig) calculates the matrices Ci, Gi and Wi for
% by removing columns with indices in miss (which is a subset of {1,2,...,r·n}. % VARMA_LLC (see comments there). On entry A={A1 A2...Ap} and B={B1 B2...Bq}
% C = {C0 C1...Cq} is as returned by find_CGW (Cj is r×r). % are cell arrays of length p and q with the r×r matrices Ai and Bi and Sig is
% also r×r. On exit C, G and W are cell arrays of r×r matrices:
function Y = C_multiply(C, X, n, miss) % C = {C0 C1...Cq} with Cj = cov(x(t),eps(t-j)),
q = length(C) - 1; % G = {G0 G1...Gq} with Gj = cov(y(t),x(t-j)), and
r = size(C{1}, 1); % W = {W0 W1...Wq} with Wj = cov(y(t),y(t-j)).
if nargin < 4, miss = false(r,n); end % These matrices are given by the formulae:
ko = find_missing_info(miss); % Cj = A1·C(j-1) + ... + A(j-1)·C1 + Aj·C0 + Bj·Sig
Y = zeros(r, n); % Gj = Bj·C0' + ... + Bq·C(q-j)'
for t = 1:n % Wj = Bj·Sig·B0' + ... + Bq·Sig·B(q-j)'
K = find(~miss(:,t)); % where C0 = Sig and B0 = -I.
K1 = (ko(t)+1 : ko(t+1))';

11
function [C,G,W] = find_CGW(A,B,Sig) end
[p,q,r] = get_dimensions(A,B,Sig); J = J-r;
I = eye(r); end
A = makecell(A); end
B = [{I} makecell(B)];
C = cell(1,q+1); function Fd = ABGdiff(AB, iParmat, C, Cd)
G = cell(1,q+1); % Differentiate A·C or B·C. iParmat = 1 for A, p·r^2+1 for B
W = cell(1,q+1); % Set iParmat = -1 to differentiate [Ap...A1]·C.
BSig = cell(1,q+1); nPar = size(Cd, 3);
C{1} = Sig; r = size(AB,1);
for j=0:q p = size(AB,2)/r;
BSig{j+1} = B{j+1}*Sig; Fd = reshape(AB*reshape(Cd,r*p,r*nPar),r,r,nPar);
C{j+1} = BSig{j+1}; k = abs(iParmat);
for i=1:min(j,p) I = 0:p-1;
C{j+1} = C{j+1} + A{i}*C{j-i+1}; if iParmat<0, I = fliplr(I); end
end for i = I
end for c=r*i+1:r*i+r
for j = 0:q for l=1:r
G{j+1} = BSig{j+1}; Fd(l,:,k) = Fd(l,:,k) + C(c,:);
W{j+1} = BSig{j+1}; k = k+1;
for i = j+1:q end
G{j+1} = G{j+1} + B{i+1}*C{i-j+1}'; end
W{j+1} = W{j+1} + BSig{i+1}*B{i-j+1}'; end
end end
end
end
% FIND_LAMBDA_OM Find the matrix Lambda_om
%
% FIND_CGW_DERIV Determine G, W and their derivatives % Lamom = FIND_LAMBDA_OM(Acol, miss) returns the matrix consisting of observed
% % rows and missing columns of Lambda i.e. Lam(obs,miss) where Lam is the
% [CCd, GGd, WWd] = find_CGW_deriv(A, B, Sig) returns three struct arrays, CCd % matrix shown in the help of lambda_multiply. Acol should be [A1; ... Ap],
% = [C0...Cq], GGd = [G0...Gq] and WWd = [W0...Wq] where each element is a % provided by find_Acol.
% structure of a matrix and its derivatives w.r.t. all the parameters as %
% described in mds_add_prod. Thus CCd(t).mat contains Ct and CCd(t).der{k}{l,c} % [Lamom,Lamomd] = FIND_LAMBDA_OM(Acol, miss, Acold) finds also the derivative
% contains the derivatives of Ct w.r.t. the (l,k) element of the k-th parameter % (Acold contains the derivatives of Acol).
% matrix (of A1..Ap B1..Bq Sig).
function [Laom,Laomd] = find_lambda_om(Acol, miss, Acold)
function [C, G, W] = find_CGW_deriv(A, B, Sig) DIFF = nargout > 1;
[p, q, r] = get_dimensions(A, B, Sig); [r,n] = size(miss);
A = makecell(A); obs = ~miss;
B = makecell(B); [ko,ro,km,rm] = find_missing_info(miss);
np = p+q+1; N = ko(n+1);
for j=1:q M = km(n+1);
BSig(j) = mds_set_zero(np, r, r); p = size(Acol,1)/r;
BSig(j) = mds_add_prod(BSig(j), B{j}, Sig, p+j, p+q+1); mLaom = findmaxnz(rm, ko, p, n);
C(j) = BSig(j); Laom = zeros(mLaom, M);
for i=1:min(j-1, p) if DIFF
C(j) = mds_add_prod(C(j), A{i}, C(j-i), i); nPar = size(Acold,3);
end Laomd = zeros(mLaom, M, nPar);
if j <= p, C(j) = mds_add_prod(C(j), A{j}, Sig, j, p+q+1); end end
end for t=1:n
C0 = mds_set_parmat(np, Sig, np); if rm(t)~=0
G0 = mds_set_parmat(np, Sig, np); tmax = min(n, t+p);
W0 = mds_set_parmat(np, Sig, np); mY = ko(tmax+1);
for i = 1:q I = obs(:,t+1:tmax);
G0 = mds_add_prod(G0, B{i}, C(i), 'T', p+i); J = miss(:,t);
W0 = mds_add_prod(W0, B{i}, BSig(i), 'T', p+i); K = km(t)+1:km(t+1);
end Laom(ko(t+1 )+1:mY, K) = -Acol(I,J);
for j = 1:q if t<p, Laom(ko(t+1)+1:ko(p+1), K) = 0; end
G(j) = BSig(j); if DIFF
W(j) = BSig(j); Laomd(ko(t+1)+1:mY, K, :) = -Acold(I,J,:);
for i = j+1:q if t<p, Laomd(ko(t+1)+1:ko(p+1), K, :) = 0; end
G(j) = mds_add_prod(G(j), B{i}, C(i-j), 'T', p+i); end
W(j) = mds_add_prod(W(j), B{i}, BSig(i-j), 'T', p+i); end
end end
end end
if q==0, C=[]; G=[]; W=[]; end
C = [C0, C]; function maxnz = findmaxnz(rm, ko, p, n)
G = [G0, G]; % return maximum non-zero row-number of Lambda_om
W = [W0, W]; tmax = find(rm,1,'last'); % last missing time
end trow = min(n, tmax + p); % time of last nonzero row
maxnz = ko(trow + 1); % number of observations until that time
end
% FIND_GNEG Make Gj matrices for negative j
%
% Gcol = find_Gneg(A, B, C, n) returns the column matrix % FIND_MISSING_INFO Information on missing values
% %
% [G(-(n-p-1)); ... ; G(-2) ; G(-1)] % [KO, RO] = FIND_MISSING_INFO(MISS) returns the number of observed values
% % before time t in KM(t) and the number of observed values at time t in RO(t).
% where G(-j) = cov(x(t),y(t+j)). [Gcol,Gcold]=find_Gneg(A,B,C,n,Cd) finds %
% also the derivative of Gcol % [KO, RO, KM, RM] = FIND_MISSING_INFO(MISS) also finds in KM(t) and RM(t) the
% number of missing values before resp. at time t.
function [Gcol, Gcold] = find_Gneg(A, B, C, n, Cd) %
DIFF = nargout > 1; % MISS should be an r by n logical array, true in locations of missing values.
r = size(C{1},1);
p = size(A,2)/r; function [ko, ro, km, rm] = find_missing_info(miss)
q = size(B,2)/r; ro = sum(~miss, 1);
m = n-p+q; ko = [0 cumsum(ro)];
% CHANGE A FROM [A1..Ap] to [Ap..A1] if nargout > 2
A = cell2mat(fliplr(makecell(A))); rm = sum(miss, 1);
if isempty(A), A = zeros(r,0); end km = [0 cumsum(rm)];
if isempty(B), B = zeros(r,0); end end
% FIND Ccol = [C0;...; C(q); C(q+1);...; C(n-p+q)] end
Ccol = cat(1, C{:}, zeros(r*(n-p-1),r));
Gcol = zeros(r*(n-p-1), r);
if DIFF % FIND_SM Make the matrix Sm and its derivative
nPar = size(Cd{1},3); %
Ccold = cat(1, Cd{:}, zeros(r*(n-p-1), r, nPar)); % Sm = find_Sm(Scol, miss) returns Sm. SCol = [S0; S1; ...; S(n-1)].
Gcold = zeros(r*(n-p-1), r, nPar); %
end % [Sm, Smd] = find_Sm(Scol, miss, Scold) also finds the derivatives. Scold
for j = q+1:m-1 % are derivatives of Scol.
J = j*r+1 : (j+1)*r;
K = max(0,j-p)*r+1 : j*r; function [Sm, Smd] = find_Sm(Scol, miss, Scold)
if j<p, AA = A(:,r*(p-j)+1:end); else AA = A; end [r,n] = size(miss);
Ccol(J,:) = AA*Ccol(K,:); DIFF = nargout > 1;
if DIFF, Ccold(J,:,:) = ABGdiff(AA, -1, Ccol(K,:), Ccold(K,:,:)); end %
end % CONSTRUCT OBSERVED INDICES
% TRANSPOSE EACH C{j}: miss = reshape(miss, r, n); %% for safeties sake
Ccol = reshape(permute(reshape(Ccol,r,m,r), [3,2,1]), r*m,r); obs = ~miss;
if DIFF [ko, ro, km, rm] = find_missing_info(miss);
Ccold = reshape(permute(reshape(Ccold, r,m,r,nPar), [3,2,1,4]), r*m,r,nPar); M = km(n+1);
end %
for j=1:n-p-1 % BUILD A COLUMN WITH 2n-1 MATRICES, [S(n-1)';...;S1';S0;S1;...S(n-1)]
J = (n-p-j-1)*r+1 : (n-p-j)*r; SU = reshape(Scol,r,n,r);
I = j*r+1 : (j+1)*r; SU = flipdim(permute(SU, [3,2,1]), 2);
K = (j+1)*r+1 : (j+q+1)*r; SS = [reshape(SU,r*n,r); Scol(r+1:end,:)];
Gcol(J,:) = Ccol(I,:) + B*Ccol(K,:); Sm = zeros(M, M);
if DIFF if DIFF
Gcold(J,:,:) = Ccold(I,:,:) + ABGdiff(B, p*r^2+1, Ccol(K,:),Ccold(K,:,:)); nPar = size(Scold,3);

12
 SUd = reshape(Scold,r,n,r,nPar); function [varargout] = forward_sub_deriv(L, Ld, varargin)
SUd = flipdim(permute(SUd, [3,2,1,4]), 2); LT.LT = true;
SSd = cat(1, reshape(SUd,r*n,r,nPar), Scold(r+1:end,:,:)); if nargout == 2
Smd = zeros(M, M, nPar); [Y, Yd] = deal(varargin{:});
end X = linsolve(L, Y, LT);
% else
% CONSTRUCT Sm [X, Yd] = deal(varargin{:});
Sm = zeros(M, M); end
ib = r*(n-1)+1; ie = r*(2*n-1); [n, m] = size(X);
for t=1:n nPar = size(Ld, 3);
if rm(t)~=0 LdX= reshape(reshape(permute(Ld, [1,3,2]), n*nPar, n)*X, n,nPar, m);
K = km(t)+1 : km(t+1); RHS = reshape(Yd - permute(LdX, [1,3,2]), [n,m*nPar]);
I = ib:ie; Xd = reshape(linsolve(L,RHS,LT), [n,m,nPar]);
J = miss(:,t); if nargout == 2, varargout = {X, Xd}; else varargout = {Xd}; end
Sm(:,K) = SS(I(miss), J); end
if DIFF, Smd(:,K,:) = SSd(I(miss), J, :); end
end % DIMENSIONS Dimensions of VARMA model
ib = ib - r; %
ie = ie - r; % [p,q,r] = GET_DIMENSIONS(A, B, Sig) returns the dimensions of the VARMA
end % model with parameters A = {A1...Ap}, B = {B1...Bp} and Sig (Ai are the AR
end % coefficients, Bi are the MA coefficients and Sig is the covariance matrix of
% the shock series.
%
% FIND_V Find V-matrix (and its derivative) for missing value likelihood % [p,q,r,n] = GET_DIMENSIONS(A, B, Sig, X) returns also the length of the
% % time series (the number of rows in X) in n.
% V = FIND_V(G, Gcol, Scol, miss) finds the matrix V as the observed rows
% and missing columns of the matrix Lam·S which is given by: function [p, q, r, n] = get_dimensions(A, B, Sig, X)
% r = size(Sig,1);
% S0 S1'... Sn-1' S0 S1'... Sn-1' if iscell(A), p = length(A); else p = size(A,2)/r; end
% S1 S0 : S1 S0 : if iscell(B), q = length(B); else q = size(B,2)/r; end
% : ... S1' : ... S1' if nargin>3
% Sp-1... S0 Sp-1... S0 assert(size(X,1)==r);
% ---------------------- or ------------------- n = size(X,2);
% Gq ... Gp-n+1 Gp ... Gp-n+1 end
% Gq : : : end
% .. Gq
% Gq...G0 .. % IS_STATIONARY Check VARMA model for stationarity
% Gq... G0 %
% % IS_STATIONARY(A, Sig) returns true if the VARMA-process x(t) = A1·x(t-1) +
% (depending on whether p or q is bigger). G, Gneg and Scol should be % ... + Ap·x(t-p) + y(t) is stationary, false otherwise. A is the r × r·p
% {G0...Gq} (from find_CGW), [G(p-n+1);...;G(-1)] (from find_Gneg) and % matrix [A1...Ap]. The time series y(t) is given by a moving average process,
% [S0;...; S(n-1)] (from vyw_solve and S_extend). miss(i,t) is false if X(i,t) % y(t) = B1·y(t-1) + ... + Bq·y(t-q) + eps(t), where eps(t) is N(0,Sig).
% is missing. % Whether x is stationary or not does not depend on B, so it need not be a
% % parameter.
% [V, Vd] = FIND_V(G, Gneg, Scol, miss, Gd, Gnegd, Scold) finds also the %
% derivative of V. Gd and Scold should be (cell) arrays with the derivatives % IS_STATIONARY(A, Sig, PLU) uses PLU from a previous call to vyw_factorize.
% of G and Scol. %
% The test is based on solving the modified Yule-Walker equations for the AR
function [V,Vd] = find_V(G, Gneg, Scol, miss, Gd, Gnegd, Scold) % process x(t)=A1·x(t-1)+...+Ap·x(t-p)+eps(t) with eps(t) ~ N(0,I), but the
DIFF = nargout > 1; % solution to these equations gives a postive definite Su matrix (see
[r,n] = size(miss); % omega_build) if and only if all the roots of the polynomial fi(b) = I - A1·b
obs = ~miss; % - A2·b^2 - ... - Ap·B^p are outside the unit circle, which characterizes a
[ko,ro,km,rm] = find_missing_info(miss); % stationary VARMA process.
N = ko(n+1);
M = km(n+1); function stat = is_stationary(A, Sig, PLU)
p = n - size(Gneg,1)/r - 1; [R,pp] = chol(Sig);
q = length(G) - 1; if pp>0, stat=false; return, end;
mV = findmaxnz(rm, ko, p, q, n); if isempty(A), stat=true; return, end
V = zeros(mV, M); r = size(A,1);
if DIFF p = length(A)/r;
nPar = size(Scold,3); if nargin==2
Vd = zeros(mV, M, nPar); PLU = vyw_factorize(A);
end if ~isempty(PLU) && ~isempty(PLU{1}) && PLU{1}(1) == 0
% Make SS = [S(n-1)';...; S2'; S1'; S0;... S(p-1)]: stat=false;
SU = reshape(Scol,r,n,r); return
SU = flipdim(permute(SU, [3,2,1]), 2); end
SS = [reshape(SU,r*n,r); Scol(r+1:r*p,:)]; end
GG = cat(1,Gneg,G{:}); I = eye(r);
if DIFF S = vyw_solve(A, PLU, {I});
SUd = reshape(Scold,r,n,r,nPar); Su = omega_build(S, {I}, {I}, p, p);
SUd = flipdim(permute(SUd, [3,2,1,4]), 2); [R, pp] = chol(Su');
SSd = cat(1, reshape(SUd,r*n,r,nPar), Scold(r+1:r*p,:,:)); stat = pp==0;
GGd = cat(1,Gnegd,Gd{:}); end
end
for t=1:n % LAMBDA_BUILD Build the matrix Lambda
if rm(t)~=0 %
LamS = [ % LAMBDA = LAMBDA_BUILD(A, r, n) builds the n·r × n·r matrix:
SS((n-t)*r+1 : (n-t+p)*r, miss(:,t)); %
GG((n-t)*r+1 : end, miss(:,t))]; % Lambda = I
tmax = min(n,max(p,t+q)); % I
mY = ko(tmax+1); % I
K = km(t)+1:km(t+1); % I
V(1:mY, K) = LamS(obs(:,1:tmax), :); % -Ap ... -A2 -A1 I
if DIFF % -Ap -A2 -A1 I
LamSd = cat(1, ... % ... .. ..
SSd((n-t)*r+1 : (n-t+p)*r, miss(:,t), :), ... % -Ap ... -A1 I
GGd((n-t)*r+1 : end, miss(:,t), :)); %
Vd(1:mY, K, :) = LamSd(obs(:,1:tmax), :, :); % where each Ai is r×r and A is a block row vector of the Ai, A = [A1...Ap].
end %
end % [LAM, LAMD] = LAMBDA_BUILD(A, r, n, nPar) builds also the derivative of
end % Lambda.
end %
% This function is useful for testing purposes.
function maxnz = findmaxnz(rm, ko, p, q, n)
% return maximum non-zero row-number of V function [Lam, Lamd] = lambda_build(A, r, n, nPar)
tmax = find(rm,1,'last'); % last missing time A = makecell(A);
if isempty(tmax), tmax=0; end; % in case nothing is missing p = length(A);
trow = min(n, max(p, tmax+q)); % time of last nonzero row Lam = mat2cell(eye(r*n), repmat(r,n,1), repmat(r,n,1));
maxnz = ko(trow + 1); % number of observations until that time for i=p+1:n
end for j=1:p
Lam{i,i-j} = -A{j};
% FORWARD_SUB_DERIV Derivative of forward substitution end
% end
% [X, Xd] = FORWARD_SUB_DERIV(L, Ld, Y, Yd) solves a lower triangular system Lam = cell2mat(Lam);
% L·X=Y and returns the derivative of X in Xd. if nargout > 1
% assert(nPar >= r*r*p); % at least the elements of A
% Xd = FORWARD_SUB_DERIV(L, Ld, X, Yd) finds the derivative if X is given. Lamd = zeros(r, n, r, n, r, r, p);
% for i=p+1:n
% The parameters are: for j=1:p
% L n×n A lower triangular matrix for l=1:r
% Ld n×n×nPar Ld (:,:,i) contains derivatives of L with respect to a for c=1:r
% parameter theta(i), i=1...,nPar Lamd(l,i,c,i-j,l,c,j) = -1;
% X n×m Solution to L·X = Y end
% Yd n×m×nPar Derivatives of Y. May also be n×nPar when m=1 end
% Xd n×m×nPar Returns derivatives of X. Is n×nPar if Yd is n×nPar. end
% end
% METHOD: Lamd = cat(3, reshape(Lamd, r*n, r*n, r*r*p), zeros(r*n,r*n,nPar-r*r*p));
% From L·X = Y it follows that Ld(:,:,i)·X + L·Xd(:,:,i) = Yd, and thus Xd end
% may be obtained with forward substitution. end

13
% LAMBDA_MULTIPLY Multiply with Lambda
%
% Y = LAMBDA_MULTIPLY(A, X, false(r,n)) returns Lam·X in Y, where Lam is an n·r % LOGDET_L Logarithm of determinant of a lower triangular matrix
% × n·r lower block-band matrix given by: %
% % ld = LOGDET_L(L) returns log(det(L)) and [ld, ldd] = LOGDET_L(L, Ld) finds in
% Lam = I % addition the derivative of ld with respect to several parameters. If L is
% I % n×n, Ld should be n×n×nPar and contain the derivative of L (nPar is the
% I % number of parameters)
% I
% -Ap ... -A2 -A1 I function [ld, ldd] = logdet_L(L, Ld);
% -Ap -A2 -A1 I ld = sum(log(diag(L)));
% ... .. .. if nargin>1
% -Ap ... -A1 I nPar = size(Ld,3);
% ldd = zeros(1,nPar);
% and X is an (n·r)-element column vector. A should be [A1...Ap]. for j=1:nPar
% ldd(j) = sum(diag(Ld(:,:,j))./diag(L));
% Y = LAMBDA_MULTIPLY(A, X, miss) where miss is r × n and true in missing end
% locations multiplies with Lam(obs,obs) where obs are the observed indices in end
% {1,2,...,r·n}. X should be an N-vector, where N is the number of observed end
% values. This call is used for multiplying with the matrix Lam_o.
%
% [Y,Yd] = LAMBDA_MULTIPLY(A,X,miss,Xd) returns Lam·X in Y and the derivatives % MAKECELL Change to cell array
% of it in Yd, when Xd contains the derivatives of X w.r.t. all the parameters. %
% Yd is an N × 1 × nPar 3-dimensional array. % C = MAKECELL(C) changes C from [C1 C2...Cm] to {C1 C2...Cm} where each Ci is
% square. If C is empty the empty cell array, {}, is returned.
function [Y, Yd] = lambda_multiply(A, X, miss, Xd)
DIFF = nargout > 1; function Cout = makecell(C)
[r, n] = size(miss); if isempty(C), Cout = {}; return, end
MISS = any(miss(:)); r = size(C,1);
p = size(A,2)/r; m = size(C,2)/r;
Ac = makecell(A); Cout = cell(1, m);
if MISS for i=1:m
obs = ~miss; Cout{i} = C(:, (i-1)*r+1:i*r);
Xf = zeros(r*n,1); end
Xf(obs) = X; end
X = reshape(Xf, r, n);
[ko, ro, km] = find_missing_info(miss);
N = ko(n+1); % MDS_ADD_PROD Add product to matrix-function and update derivative
else %
X = reshape(X, r, n); % FSNEW = MDS_ADD_PROD(FS,X,Y,kX,kY) calculates F + X·Y and its derivatives
end % with respect to all the parameter matrices of a VARMA model, [A1...Ap B1...Bq
Y = X; % Sig]. X should be the kX-th parameter matrix and Y the kY-th one. FS is a
if DIFF % structure with the following fields:
nPar = size(Xd,3); % mat The r×r matrix F
if MISS % der A (p+q+1)-element cell-array with the derivatives of F w.r.t. to the
Xfd = zeros(n*r, nPar); % parameter matrices; der{k} is an r×r cell matrix with derivatives
Xfd(obs,:) = Xd; % with respect to the k-th parameter matrix, P, and der{k}{l,c} is an
else % r×r matrix with the derivative of F with respect to P(l,c).
Xfd = Xd; % spcod Vector of sparsity codes. The sparsity code has one of the values:
end % 'z' FS.der{k} is all zeros
I = any(Xfd,1); % nonzeros in nPar direction % 'e' FS.der{k}{i,j}(i,j) is the only nonzero in FS.der{k}{i,j}
nI = sum(I); % 'r' The nonzeros of FS.der{k}{i,j} are all in its i-th row
Xfd = reshape(Xfd, r, n, nPar); % 'c' The nonzeros of FS.der{k}{i,j} are all in its j-th column
Zd = permute(Xfd(:,:,I),[1,3,2]); % 'x' The nonzeros of FS.der{k}{i,j} are all in its i-th column
Xd = reshape(Zd, r*nI, n); % 'f' All cells in FS.der{k} are full matrices
L = p*nPar+1 : n*nPar; % F + X·Y and its derivative are returned in FSNEW which is a structure as FS.
end %
L = p+1:n; % FSNEW = MDS_ADD_PROD(FS,X,Y,'T',kX,kY) calculates F + X·Y' and its
for k = 1:p % derivatives. FS, X and Y are as described above.
K = p+1-k:n-k; %
Y(:, L) = Y(:, L) - Ac{k}*X(:, K); % FSNEW = MDS_ADD_PROD(FS,X,A,kX) calculates F + X·A and its derivaties for a
if (DIFF) % constant matrix A. In this case A need not have r columns.
YdIL = reshape(Zd(:,:,L), r, nI*(n-p)); %
YdIL = YdIL - Ac{k}*reshape(Xd(:,K), r, nI*(n-p)); % FSNEW = MDS_ADD_PROD(FS,X,GS,kX) calculates F + X·G and its derivatives. FS
Zd(:,:,L) = reshape(YdIL, r, nI, n-p); % and GS are structures as described above and X is the kX-th parameter matrix.
end % G need not have r columns (and if it does not it must have sparsity code 'z',
end % 'r' or 'f', and now der{k} is r×size(G,2)).
Y = Y(:); %
if MISS, Y = Y(obs); end % FSNEW = MDS_ADD_PROD(FS,X,GS,'T',kX) calculates F + X·G' and its derivatives.
if (DIFF) % FS, GS and X are as above. G need not have r rows, and if it does not, must
Yd = zeros(r, n, nPar); % have sparsity code 'f' or 'z', and now der{k} is r×size(G,1).
Yd(:,:,I) = permute(reshape(Zd, r, nI, n), [1,3,2]);
for k=1:p function FSNEW = mds_add_prod(FS,X,Y,varargin)
j0 = r^2*(k-1); N = length(FS.der);
for j = 1:r if ischar(varargin{1}), transp='T'; varargin(1) = []; else transp=''; end
J = j0 + (j:r:r^2); lJ = length(J); kX = varargin{1};
Yd(j, p+1:n, J) = Yd(j, p+1:n, J) - reshape(X(:,p-k+1:n-k)', 1, n-p,lJ); if isstruct(Y)
end GS = Y;
end % global cover
Yd = reshape(Yd, n*r, 1, nPar); % for k=1:N
if MISS, Yd = Yd(obs, 1, :); end % i = 2 + (k==kX) + 2*isequal(transp,'T');
end % j = 2 + strfind('zercxf',GS.spcod(k));
end % cover(i,j)='x';
% end
% LAMBDA_T_MULTIPLY Multiply vector with transpose of Lambda cod = ['FpXG' transp];
% elseif nargin >= 5
% Y = LAMBDA_T_MULTIPLY(A, X, r, n) returns Lam'·X in Y, where Lam is an cod = ['FpXY' transp];
% n·r×n·r lower block-band matrix given by: kY = varargin{2};
% assert(kX~=kY);
% Lam = I else
% I cod = 'FpXA';
% I kY = 0;
% I end
% -Ap ... -A2 -A1 I r = size(X,1);
% -Ap -A2 -A1 I der = FS.der;
% ... .. .. spcod = FS.spcod;
% -Ap ... -A1 I % fprintf('Cod=%s kX=%d\n', cod, kX);
% switch cod
% and X is a vector with compatible dimensions. A should be {A1...Ap}. case 'FpXA'
% F = FS.mat + X*Y;
% Y = LAMBDA_T_MULTIPLY(A, X, r, n, miss) multiplies with Lam(obs,obs)' where for c = 1:r
% obs are the observed indices in {1,2,...,r·n}. This call is used for for l = 1:r
% multiplying with the matrix Lam_o'. der{kX}{l,c}(l,:) = der{kX}{l,c}(l,:) + Y(c,:);
function Y = lambda_T_multiply(A, X, r, n, miss) end
if nargin < 5, miss = false(r,n); end end
A = makecell(A); spcod(kX) = setsp(spcod(kX),'r');
p = length(A); case 'FpXY'
ko = find_missing_info(miss); F = FS.mat + X*Y;
Y = X; for c = 1:r
for t=p+1:n for l = 1:r
K = find(~miss(:,t)); der{kX}{l,c}(l,:) = der{kX}{l,c}(l,:) + Y(c,:);
K1 = ko(t)+1 : ko(t+1); der{kY}{l,c}(:,c) = der{kY}{l,c}(:,c) + X(:,l);
for k = 1:p end
L = find(~miss(:,t-k)); end
L1 = ko(t-k)+1 : ko(t-k+1); spcod(kX) = setsp(spcod(kX),'r');
Y(L1) = Y(L1) - A{k}(K,L)'*X(K1); spcod(kY) = setsp(spcod(kY),'c');
end case 'FpXYT'
end F = FS.mat + X*Y';
end for c = 1:r

14
 for l = 1:r
der{kX}{l,c}(l,:) = der{kX}{l,c}(l,:) + Y(:,c)';
der{kY}{l,c}(:,l) = der{kY}{l,c}(:,l) + X(:,c); % OMEGA_AR_FACTOR Cholesky factorization of Omega in pure autoregressive case
end %
end % [Lu, LSig, jpat] = OMEGA_AR_FACTOR(S, Sig, miss) determines the Cholesky
spcod(kX) = setsp(spcod(kX),'r'); % factorization of Omega(obs,obs) in the pure autregressive case. Omega is then
spcod(kY) = setsp(spcod(kY),'x'); %
case 'FpXG' % S0 S1'... Sp-1'|
F = FS.mat + X*GS.mat; % S1 S0 : |
Gd = GS.der; % S2 S1 S0 | r·p
for k = 1:N % : ... S1'|
% fprintf('GS.spcod(%d)=%s\n',k,GS.spcod(k)) % Sp-1 S0 |
for c = 1:r % --------------------+------------
for l = 1:r % |Sig
switch GS.spcod(k) % | Sig r·(n-p)
case 'f', der{k}{l,c} = der{k}{l,c} + X*Gd{k}{l,c}; % | ...
case 'r', der{k}{l,c} = der{k}{l,c} + X(:,l)*Gd{k}{l,c}(l,:); % | Sig
case 'c', der{k}{l,c}(:,c) = der{k}{l,c}(:,c) + X*Gd{k}{l,c}(:,c); %
case 'x', der{k}{l,c}(:,l) = der{k}{l,c}(:,l) + X*Gd{k}{l,c}(:,l); % Lu returns the Cholesky factor of the upper left partition and LSig{j}
case 'e', der{k}{l,c}(:,c) = der{k}{l,c}(:,c) + X(:,l); % returns the Cholesky factor of Sig(obs,obs) where obs are the indices of the
case 'z' % do nothing % observed values for the j-th missing pattern occurring after time p.
end % jpat(i) returns the number of the missing pattern at time p+j. Thus
end % LSig{jpat(i)} is the Cholesky factor of the i-th block in the lower right
end % partition of Omega(obs,obs). S should be {S0...S(p-1)} and miss should be
switch GS.spcod(k) % r×n, true in missing locations.
case {'f','r'}, spcod(k) = setsp(spcod(k),'f'); %
case {'c','e'}, spcod(k) = setsp(spcod(k),'c'); % [Lu, LSig, jpat, Lud, LSigd] = OMEGA_AR_FACTOR(S, Sig, miss, Sd, Sigd)
case {'x'}, spcod(k) = setsp(spcod(k),'x'); % finds also the derivatives of Lu and LSig.
end %
end % OMEGA_AR_FACTOR(S, Sig, n) and OMEGA_AR_FACTOR(S, Sig, n, Sd, Sigd) may be
for c = 1:r % used instead when there are no missing values.
for l = 1:r
der{kX}{l,c}(l,:) = der{kX}{l,c}(l,:) + GS.mat(c,:); function [Lu,LSig,jpat,Lud,LSigd] = omega_ar_factor(S,Sig,miss,Sd,Sigd)
end DIFF = nargout > 3; if DIFF, assert(nargin==5); end
end MISS = length(miss) > 1 && any(miss(:));
spcod(kX) = setsp(spcod(kX),'r'); p = length(S) - 1;
case 'FpXGT' Su = omega_build(S, {Sig}, {Sig}, p, p);
F = FS.mat + X*GS.mat'; r = size(Sig,1);
Gd = GS.der; if MISS
for k = 1:N n = size(miss, 2);
% fprintf('GS.spcod(%d)=%s\n',k,GS.spcod(k)) miss1 = miss(:,1:p);
for c = 1:r miss2 = miss(:,p+1:end);
for l = 1:r else
switch GS.spcod(k) if length(miss)==1, n = miss; else n = size(miss,2); end
case 'f', der{k}{l,c} = der{k}{l,c} + X*Gd{k}{l,c}'; end
case 'c', der{k}{l,c} = der{k}{l,c} + X(:,c)*Gd{k}{l,c}(:,c)'; if DIFF
case 'x', der{k}{l,c} = der{k}{l,c} + X(:,l)*Gd{k}{l,c}(:,l)'; Sud = omega_build_deriv(Sd, {Sigd}, {Sigd}, p, p);
case 'r', der{k}{l,c}(:,l) = der{k}{l,c}(:,l)+X*Gd{k}{l,c}(l,:)'; if MISS,
case 'e', der{k}{l,c}(:,l) = der{k}{l,c}(:,l)+X(:,c); [Su, Olow, Sud] = omega_remove_miss(Su, [], miss1, Sud, []);
case 'z' % do nothing end
end elseif MISS
end Su = omega_remove_miss(Su, [], miss1);
end end
switch GS.spcod(k) Lu = chol(Su')';
case {'r','e'}, spcod(k) = setsp(spcod(k),'x'); if DIFF, Lud = chol_deriv(Lu, Sud); end
case {'f','c','x'}, spcod(k) = setsp(spcod(k),'f'); if MISS
case {'z'}, % do nothing [pat, ipat, jpat] = unique(miss2', 'rows'); % the unique missing patterns
end else
end pat = false(1, r);
for c = 1:r jpat = ones(1, n - p);
for l = 1:r end
der{kX}{l,c}(l,:) = der{kX}{l,c}(l,:) + GS.mat(:,c)'; npat = size(pat,1);
end for i=1:npat
end obs = ~pat(i,:);
spcod(kX) = setsp(spcod(kX),'r'); LSig{i} = chol(Sig(obs,obs))';
end if DIFF
FSNEW.mat = F; LSigd{i} = chol_deriv(LSig{i}, Sigd(obs, obs, :));
FSNEW.der = der; end
FSNEW.spcod = spcod; end
end end

function spcod = setsp(spcod, s)

% If s~=spcod and either both are in rcx or both are in ex then spcod:='f'. % OMEGA_AR_LOGDET Calculate log of determinant of Omega in pure AR case
% Otherwise if s is >spcod in the ordering z<e<r<c<f then spcod:=s %
if spcod ~= s && ... % LD = OMEGA_AR_LOGDET(Luo, LSig, jpat) finds log(det(Omega)) in the pure
(any('xcr'==spcod) && any('xcr'==s) || any('xe'==spcod) && any('xe'==s)) % autoregressive case. Luo is Cholesky factor of Su(obs,obs) and LSig{jpat(i)}
spcod = 'f'; % is the Cholesky factor of the i-th diagonal block in the lower right part of
elseif strfind('zexcrf',s) > strfind('zexcrf',spcod) % Omega.
spcod = s; %
end % [LD, LDD] = OMEGA_AR_LOGDET(Luo, LSig, jpat, Luod, LSigd) finds also the
end % gradient of LD.

function [ld, ldd] = omega_ar_logdet(Luo, LSig, jpat, Luod, LSigd)

% MDS_SET_PARMAT Set matrix and derivative to a parameter matrix and derivative npat = length(LSig);
% count = histc(jpat,1:npat); % count of each pattern
% FS = MDS_SET_PARMAT(nparmat,X,kX) sets FS to a structure appropriate for if nargout == 1 % ONLY FUNCTION VALUES
% mds_add_prod. FS.mat is set to X which should be the kX-th paramter matrix ld = 2*sum(log(diag(Luo)));
% (out of the sequence A1...Ap B1...Bq Sig), and nparmat=p+q+1. FS.der is set for j = 1:npat
% to the derivative of X with respect to all the parameter matrices ld = ld + 2*count(j)*sum(log(diag(LSig{j})));
% FS.spcod(kX) is set to 'e' and FS.spcod(j) is set to 'z' for j~=kX. end
else % FIND ALSO DERIVATIVES
function FS = mds_set_parmat(nparmat,X,kX) [ld, ldd] = logdet_L(Luo, Luod);
r = size(X,1); ld = 2*ld; ldd = 2*ldd;
FS = mds_set_zero(nparmat,r,r); for j = 1:npat
FS.mat = X; [ldt, lddt] = logdet_L(LSig{j}, LSigd{j});
for c=1:r ld = ld + 2*count(j)*ldt;
for l=1:r ldd = ldd + 2*count(j)*lddt;
FS.der{kX}{l,c}(l,c) = 1; end
end end
end end
FS.spcod(kX) = 'e';
end
% OMEGA_AR_TRISOLVE Triangular solve with Cholesky factor of Omega, pure AR case
%
% MDS_SET_ZERO Set matrix and its derivative to zero % X = OMEGA_AR_TRISOLVE(Lu, LSig, Y, JPAT, ko, 'NoT') solves L·X = Y where L
% % is the Cholesky factor of Omega(obs,obs) in a pure auto-regressive case
% FS = MDS_SET_ZERO(p+q+1,r,n) sets FS to a structure appropriate for % (q=0) stored in Lu and LSig, which are returned by omega_ar_factor. ko(t) is
% mds_add_prod containing an r×n matrix and its derivative both equal to 0; p, % the number of observed values before time t.
% q and r are the dimensions of the VARMA model being applied. %
% X = OMEGA_AR_TRISOLVE(Lu, LSig, Y, JPAT, ko, 'T') solves L'·X = Y.
function FS = mds_set_zero(npar,r,n) %
FS.mat = zeros(r,n); % [X, Xd] = OMEGA_AR_TRISOLVE(Lu, LSig, Y, JPAT, ko, code, Lud, LSigd, Yd)
FS.der = cell(1,npar); % finds also the deriative of X. Lud, LSigd and Yd are the derivatives of Lu,
der_k = mat2cell(zeros(r^2,n*r),repmat(r,r,1),repmat(n,r,1)); % LSig and Y, and code begins with 'N' or 'T'
for k = 1:npar %
FS.der{k} = der_k; % As in omega_back_sub, Y may have fewer than N rows (N=number of observed
end % values), and then the rest of the rows are taken to be zero. It must end
FS.spcod = repmat('z',1,npar); % on a block boundary.
end

15
function [Y,Yd] = omega_ar_trisolve(Lu, LSig, Y, jpat, ko, code, Lud, LSigd, Yd) % Here is the structure of Omega for q<p:
% Note: Xd and Xd are stored in Y and Yd %
OPTS.LT = true; % S0 S1'... Sp-1'| transpose
if code(1)=='T', OPTS.TRANSA = true; end % S1 S0 : | of lower
n = length(ko)-1; % S2 S1 S0 | left part r·p
p = n - length(jpat); % : ... S1'|
mY = size(Y,1); % Sp-1 S0 |
tmax = find(ko == mY, 1, 'first') - 1; assert(~isempty(tmax)); % --------------------+-----------------------
e = size(Lu,1); % Gq ... G2 G1 | W0 transpose
[mY,nY] = size(Y); % Gq G2 | W1 W0 of lower
Y(1:e, :) = linsolve(Lu, Y(1:e, :), OPTS); % .. G3 | W2 W1 W0 part
for t=p+1:tmax % .. : | : : ..
K = ko(t)+1 : min(mY, ko(t+1)); % Gq | Wq-1 .. r·(n-p)
Y(K,:) = linsolve(LSig{jpat(t-p)}, Y(K,:), OPTS); % | Wq ..
end % | Wq ..
if nargout > 1, % | ... ..
nPar = size(Yd, 3); % | Wq ..... W0
Lud = reshape(permute(Lud, [1,3,2]), e*nPar, e); %
Yd(1:e,:,:) = Yd(1:e,:,:)-permute(reshape(Lud*Y(1:e,:),e,nPar,nY), [1,3,2]); % For q >= p the picture is:
if isempty(Yd), Xd = Yd; return; end %
for t=p+1 : tmax % S0 S1'... Sp-1' Gp'... Gq-1'| transpose
K = ko(t)+1 : min(mY, ko(t+1)); % S1 S0 : : : | of lower
ro = ko(t+1) - ko(t); % S2 .. : : : | left part
Lj = reshape(permute(LSigd{jpat(t-p)}, [1,3,2]), ro*nPar, ro); % : .. S1' G2' : |
if code(1)=='T' % Sp-1 S1 S0 G1'... Gq-p'| r·h
Yd(K,:,:) = Yd(K,:,:) - permute(reshape(Lj'*Y(K,:),ro,nPar,nY),[1,3,2]); % Gp G2 G1 W0 ...Wq-p-1'|
else % : : : .. : |
Yd(K,:,:) = Yd(K,:,:) - permute(reshape(Lj*Y(K,:),ro,nPar,nY),[1,3,2]); % : : : .. : |
end % Gq-1... Gq-p Wq-p-1... W0 |
end % -------------------------------+-------------------------
Yd(1:e, :) = linsolve(Lu, Yd(1:e, :), OPTS); % Gq .. Gq-p+1 Wq-p ... W1 | W0 transpose
for t=p+1:tmax % .. : : W2 | W1 W0 of lower
K = ko(t)+1 : min(mY,ko(t+1)); % Gq : : | : .. part
Yd(K,:) = linsolve(LSig{jpat(t-p)}, Yd(K,:), OPTS); % Wq : | : .. r·(n-h)
end % .. : | : ..
Yd = reshape(Yd, mY, nY, nPar); % Wq | : ..
end % | Wq ..
end % | .. ..
% | Wq ... W0
%
% OMEGA_BACK_SUB Back substitution with Cholesky factor of Omega or Omega_o % The Olow matrix is r·(n-p)×r·(q+1) and the following example for q = 4, n-p =
% % 5 shows its structure:
% X = OMEGA_BACK_SUB(Lu, Ll, Y, p, q, ko) solves L_o'·X = Y where p and q are %
% the dimensions of the ARMA model and Lo is the Cholesky factor of Omega_o, % G5 G4 G3 W2 W1 W0
% stored in Lu and Ll (as returned by omega_factor). ko(t) is the number of % G5 G4 W3 W2 W1 W0
% observed values before time t. % G5 W4 W3 W2 W1 W0 r·(n-p)
% % W5 W4 W3 W2 W1 W0
% Y may have fewer rows than L_o (which has N rows, N being the total number of % W5 W4 W3 W2 W1 W0
% observations) and then L_o(1:k,1:k)·X = Y is solved, where k is the row count % r·(q+1)
% of Y. This will save computations. There is the restriction that Y must end
function [Su, Olow] = omega_build(S,G,W,p,n)
q = length(G) - 1;
% on a block boundary (i.e. k must be equal to ko(j) for some j), and that it h = max(p,q);
% must have at least ko(p+1) rows. S = cell2mat(fliplr(S(1:p)));
% G = cell2mat(fliplr(G));
% [X,Xd] = OMEGA_BACK_SUB(Lu, Ll, Y, p, q, ko, Lud, Lld, Yd) finds in addition W = cell2mat(fliplr(W));
% the derivative of X. Again Y (and Yd) may have less than N rows. r = size(G,1);
Su = zeros(h,h);
function [X,Xd] = omega_back_sub(Lu, Ll, Y, p, q, ko, Lud, Lld, Yd) Olow = zeros(r*(n-h), r*(q+1));
DIFF = nargout>1; K = 1:r;
N = ko(end); for t = 1:p
k = size(Y,1); Su(K,1:t*r) = S(:,end-t*r+1:end);
tmax = find(ko == k, 1, 'first') - 1; K = K + r;
assert(~isempty(tmax)); end
X = Y; J = (q-p)*r+1 : q*r;
if DIFF for t = p+1:h
nPar = size(Lld,3); Su(K,1:p*r) = G(:,J);
Xd = Yd; Su(K,p*r+1:t*r) = W(:,end-(t-p)*r+1:end);
end J = J - r;
if isempty(X) || tmax<0, return, end K = K + r;
e = size(Lu,1); end
LTT.LT = true; K = 1:r;
LTT.TRANSA = true; je = min(p,q)*r;
h = max(p,q); for t = 1:n-h
assert(e == ko(h+1)); Olow(K,1:je) = G(:,1:je);
for t = tmax:-1:h+1 Olow(K,je+1:end) = W(:,je+1:end);
J = ko(t-q)+1:ko(t); if je > 0, je = je-r; end
K = ko(t)+1:ko(t+1); K = K + r;
JL = J - ko(t-q); end
KL = K - ko(t-q); end
X(K,:) = linsolve(Ll(K-e,KL), X(K,:), LTT);
X(J,:) = X(J,:) - Ll(K-e,JL)'*X(K,:); % OMEGA_BUILD_DERIV Build derivatives of matrix Omega
if DIFF %
Xd(K,:,:) = back_sub_deriv(Ll(K-e,KL), Lld(K-e,KL,:), X(K,:), Xd(K,:,:)); % [Sud,Olowd] = OMEGA_BUILD_DERIV(Sd,Gd,Wd,n) returns an h·r×h·r×nPar array Sud
Xd(J,:,:)=Xd(J,:,:)-atb_deriv(Ll(K-e,JL),Lld(K-e,JL,:),X(K,:),Xd(K,:,:)); % and an (n-h)r×(q+1)r×nPar array Olowd with the derivatives of Su and Olow
end % with respect to all parameters. Sd, Gd and Wd are the derivatives of the
end % Si, Gi and Wi matrices.
if e>0
if tmax<h, e=ko(tmax+1); Lu=Lu(1:e,1:e); end function [Sud, Olowd] = omega_build_deriv(Sd,Gd,Wd,p,n)
X(1:e,:) = linsolve(Lu, X(1:e,:), LTT); q = length(Gd) - 1;
if DIFF h = max(p,q);
if tmax < h, Lud=Lud(1:e,1:e,:); end r = size(Gd{1},1);
Xd(1:e,:,:) = back_sub_deriv(Lu, Lud, X(1:e,:), Xd(1:e,:,:)); Sd = cell2mat(fliplr(Sd(1:p)));
end Gd = cell2mat(fliplr(Gd));
end Wd = cell2mat(fliplr(Wd));
end nPar = size(Gd,3);
Sud = zeros(r*h, r*h, nPar);
Olowd = zeros(r*(n-h), r*(q+1), nPar);
% OMEGA_BUILD Build band covariance matrix for a VARMA model K = 1:r;
% for t = 1:p
% [Su,Olow] = OMEGA_BUILD(S,G,W,p,n) finds cov. matrix Omega of w=(w1'...wn')': Sud(K, 1:t*r, :) = Sd(:, end-t*r+1:end, :);
% K = K + r;
% Su h·r×h·r, upper left part of Omega, where h = max(p,q) end
% Olow (n-h)r×(q+1)r, block-diagonals of lower part of Omega J = (q-p)*r+1 : q*r;
% S r×(p+1)·r, = {S0 S1...Sp}, Si = cov(x(t),x(t-i)) for t = p+1:h
% G r×(q+1)r, = {G0...Gq}, Gi = cov(y(t),y(t-i)) Sud(K, 1:p*r, :) = Gd(:, J, :);
% W r×(q+1)r, = {W0...Wq}, Wi = cov(y(t),x(t-i)) Sud(K, p*r+1:t*r, :) = Wd(:, end-(t-p)*r+1:end, :);
% n length of series J = J - r;
% p number of autoregressive terms K = K + r;
% q number of moving average terms end
% r dimension of xt K = 1:r;
% je = min(p,q)*r;
for t = 1:n-h
Olowd(K, 1:je, :) = Gd(:, 1:je, :);
Olowd(K, je+1:end, :) = Wd(:, je+1:end, :);
if je > 0, je = je-r; end
K = K + r;
end
end

16
% OMEGA_FACTOR Cholesky factorization of Omega % OMEGA_FORWARD_DERIV Derivative of solution to L·X = Y or Lo·X = Y
% %
% [Lu,Ll,info] = omega_factor(Su,Olow,p,q,ko) calculates the Cholesky % Xd = OMEGA_FORWARD_DERIV(Lu, Ll, Lud, Lld, X, Yd, p, q, ko) calculates the
% factorization Omega = L·L' of Omega which is stored in two parts, a full % derivative of the solution X to L·X = Y where L is the Cholesky factor of
% upper left partition, Su, and a block-band lower partition, Olow, as returned % Omega, stored in Lu and Ll (as returned by omega_factor), Lud and Lld are the
% by omega_build. Omega is symmetric, only the lower triangle of Su is % derivatives of Lu and Ll (calculated with omega_factor_deriv) and Yd is the
% populated, and Olow only stores diagonal and subdiagonal blocks. On exit, L = % derivative of Y. ko(t) is the number of observed values before time t (for
% [L1; L2] with L1 = [Lu 0] and L2 is stored in block-band-storage in Ll. Info % the complete data case it is 0:r:r*n).
% is 0 on success, otherwise the loop index resulting in a negative number %
% square root. P and q are the dimensions of the problem and ko is a vector % For the missing value case, Lu, Ll, Lud and Lld store the matrix Lo and its
% with ko(t) = number of observed values before time t. % derivatives.
% %
% In the complete data case ko should be 0:r:n*r. For missing values, Su and % As in OMEGA_FORWARD the calculation may be speeded when Y (and X) have a zero
% Olow are the upper left and lower partitions of Omega_o = Omega with missing % block with ko(tmin) rows at top. If Y = [O; Y1], X = [O; X1] and Y1d is the
% rows and columns removed. In this case Lu and Ll return L_o, the Cholesky % derivative of Y1, use the call X1d = OMEGA_FORWARD_DERIV(Lu, Ll, Lud, Lld,
% factor of Omega_o. % X1, Y1d, p, q, ko, tmin). Also, to use only the leading tmax × tmax blocks of
% L and its derivatives, call: OMEGA_FORWARD_DERIV(..., tmin, tmax).
function [Lu,Ll,info] = omega_factor(Su,Olow,p,q,ko) %
n = length(ko)-1; % METHOD: Differentiating through L·X = Y with respect to a parameter one
h = max(p,q); % obtains L·Xd = Yd - Ld·X, so Xd may be obtained with forward substitution (by
ro = diff(ko); % calling omega_forward).
Ll = zeros(size(Olow));
[Lu,info] = chol(Su'); % upper left partition function Xd = omega_forward_deriv(Lu, Ll, Lud, Lld, X, Yd, p, q, ko,tmin,tmax,T)
if info>0, return; end if nargin<10, tmin=1; end
Lu = Lu'; if nargin<11, tmax=length(ko)-1; end
e = ko(h+1); % order of Su n = length(ko) - 1;
for t = h+1 : n % loop over block-lines in Olow h = max(p,q);
K = ko(t)+1 : ko(t+1); ro = diff(ko);
KL = K - ko(t-q); e = ko(h+1); % order of Lu
JL = 1 : ko(t)-ko(t-q); Xd = Yd;
tmin = t-q; tmax = t-1; nPar = size(Xd,3);
Ll(K-e, JL) = omega_forward(Lu, Ll, Olow(K-e,JL)', p, q, ko, tmin, tmax)'; for i=1:nPar
[Ltt, info] = chol(Olow(K-e,KL) - Ll(K-e,JL)*Ll(K-e,JL)'); % Find Yd-Ld·X and store in Xd:
if info>0, info = info + ko(t); return; end j = ko(tmin) + 1;
Ll(K-e, KL) = Ltt'; m = ko(h+1) - j + 1;
end Xd(1:m,:,i) = Xd(1:m,:,i) - tril(Lud(j:end,j:end,i))*X(1:m,:);
end for t=max(h+1,tmin):tmax
t1 = max(t-q,tmin);
J = ko(t1)+1 : ko(t);
% OMEGA_FACTOR_DERIV Derivative of Cholesky factorization of Omega K = ko(t)+1 : ko(t+1);
% JX = J - j + 1;
% [Lud,Lld] = OMEGA_FACTOR_DERIV(Sud, Olowd, Lu, Ll, p, q, ko) calculates the KX = K - j + 1;
% derivative of the Cholesky factorization of Omega. On entry Sud should be an JL = J - ko(t-q);
% r·h×r·h×nPar array with the derivatives of Su and Olowd should be an KL = K - ko(t-q);
% r·(n-h)×r·(q+1) array with the derivatives of Olow where Su and Olow are the Xd(KX,:,i) = Xd(KX,:,i)-Lld(K-e,JL,i)*X(JX,:)-tril(Lld(K-e,KL,i))*X(KX,:);
% upper left and lower partitions of Omega as returned by omega_build. end
% Moreover, Lu and Ll should be as returned by omega_factor and p and q are the Xd(:,:,i) = omega_forward(Lu, Ll, Xd(:,:,i), p, q, ko, tmin, tmax);
% dimensions of the model. ko is an (n+1)-vector with ko(t) = number of end
% observed values before time t. On exit Lud and Lld return the derivatives of end
% Lu and Ll.
%
% In the missing value case, Sud and Olowd are from omega_remove_miss. % OMEGA_LOGDET Calculate log of determinant of Omega
%
function [Lud, Lld] = omega_factor_deriv(Sud, Olowd, Lu, Ll, p, q, ko) % LD = OMEGA_LOGDET(Lu, Ll, p, q, ko) returns log(det(Omega)) from the Cholesky
n = length(ko) - 1; % factorization of Omega given by omega_factor.
h = max(p, q); %
Lud = chol_deriv(Lu, Sud); % Derivative of upper left partition: % [LD, LDD] = OMEGA_LOGDET(Lu, Ll, p, q, ko, Lud, Lld) returns also the
% % derivative of log(det(Omega)) in LDD. Lud and Lld are from
% Find derivative of lower part: % omega_factor_deriv.
e = size(Lu, 1); %
Lld = zeros(size(Olowd)); % In the missing value case, Lu, Ll, Lud and Lld store the matrix L_o and its
for t = h+1 : n % loop over block-lines in Olow % derivative.
K = ko(t)+1 : ko(t+1);
KL = K - ko(t-q); function [ld,ldd] = omega_logdet(Lu, Ll, p, q, ko, Lud, Lld)
JL = 1 : ko(t)-ko(t-q); n = length(ko) - 1;
U = Ll(K-e, JL); if nargin <=5 % no derivatives
Yd = permute(Olowd(K-e, JL, :), [2,1,3]); nPar = 0;
Ud = omega_forward_deriv(Lu, Ll, Lud, Lld, U', Yd, p, q, ko, t-q, t-1); ld = 2*sum(log(diag(Lu)));
Ud = permute(Ud, [2,1,3]); else
Lld(K-e, JL, :) = Ud; nPar = size(Lld,3);
UUdT = aat_deriv(U, Ud); [ld, ldd] = logdet_L(Lu, Lud);
Lld(K-e, KL, :) = chol_deriv(Ll(K-e, KL), Olowd(K-e, KL, :) - UUdT); ld = 2*ld; ldd = 2*ldd;
end end
end e = size(Lu,1);
h = max(p,q);
ro = diff(ko);
% OMEGA_FORWARD Forward substitution with Cholesky factor of Omega or Omega_o for t = h+1:n
% J = ko(t)-e + (1:ro(t));
% X = OMEGA_FORWARD(Lu, Ll, Y, p, q, ko) solves L·X = Y where p, q and r are K = ko(t)-ko(t-q) + (1:ro(t));
% the dimensions of the ARMA model and L is the Cholesky factor of Omega, if nPar==0
% stored in Lu and Ll (as returned by omega_factor). ko(t) is the number of ld = ld + 2*logdet_L(Ll(J,K));
% observed values before time t. In the missing value case, Lu and Ll store the else
% matrix L_o (see omega_factor). [ldt, lddt] = logdet_L(Ll(J,K), Lld(J,K,:));
% ld = ld + 2*ldt;
% When Y = [O; Y1] where O is a zero matrix with ko(tmin) rows, the calculation ldd = ldd + 2*lddt;
% may be speeded using the call X1 = OMEGA_FORWARD(Lu, Ll, Y1, p, q, ko, tmin). end
% The X1 returned will contain rows ko(tmin)+1...ko(n+1) of X (the first end
% ko(tmin) rows being known to be zero). Moreover, to use only the first tmax end
% block rows of L, use X1 = OMEGA_FORWARD(Lu, Ll, Y1, p, q, ko, tmin, tmax),
% with Y1 containing block-rows tmin to tmax. This call syntax is used in
% omega_factor. % OMEGA_LTL LTL-factorization of Omega
%
function X = omega_forward(Lu, Ll, Y, p, q, ko, tmin, tmax) % [LU,LL,INFO] = OMEGA_LTL(SU,OLOW,P,Q,KO) calculates the LTL-factorization
LT.LT = true; % Omega = L'·L of Omega which is stored in two parts, a full upper left
if nargin<7, tmin=1; end % partition, SU, and a block-band lower partition, OLOW, as returned by
if nargin<8, tmax=length(ko)-1; end % omega_build. Omega is symmetric, only the lower triangle of SU is
h = max(p,q); % populated, and OLOW only stores diagonal and subdiagonal blocks. On exit, L =
ro = diff(ko); % [L1; L2] with L1 = [LU 0] and L2 is stored in block-band-storage in LL. INFO
e = ko(h+1); % order of Lu % is 0 on success. P and Q are the dimensions of the problem and KO is a vector
j = ko(tmin) + 1; % with KO(t) = number of observed values before time t.
m = ko(h+1) - j + 1; %
X = Y; % In the complete data case KO should be 0:r:n*r. For missing values, SU and
if m>0, X(1:m,:) = linsolve(Lu(j:end,j:end), Y(1:m,:), LT); end % OLOW are the upper left and lower partitions of Omega_o = Omega with missing
for t=max(h+1,tmin):tmax % rows and columns removed. In this case LU and LL return L_o, the LTL-factor
t1 = max(t-q,tmin); % of Omega_o.
J = ko(t1)+1 : ko(t); %
K = ko(t)+1 : ko(t+1); % [LU,LL,INFO,LUD,LLD] = OMEGA_LTL(SU,OLOW,P,Q,KO,SUD,OLOWD) finds in addition
JX = J - j + 1; % the derivatives of LU and LL.
KX = K - j + 1; %
JL = J - ko(t-q); % METHOD SKETCH: For a full block matrix A, determine k-th block row for k = n,
KL = K - ko(t-q); % n-1,..., 1 using the partitioning of A:
X(KX,:) = X(KX,:) - Ll(K-e,JL)*X(JX,:); %
X(KX,:) = linsolve(Ll(K-e,KL), X(KX,:), LT); % |L1' X M' | |L1 | |A1 U B'|
end % | Lkk' Y' | · |X' Lkk | = |U' Akk V'|
end % | L2'| |M Y L2| |B V A2|
%
% This implies that Lkk'·Lkk = Akk - Y'·Y (so Lkk is the LTL-factor of Akk -
% Y'·Y) and also that X' = Lkk'\(U' - Y'·M). When A = Omega, LL is given by
% this procedure, with suitable ammendments drawing on the band structure of

17
% OLOW. To find LU use the partitioning of A: K = K+1;
% end
% |LU' M' | . |LU | = |SU B'| Olowo = Olowo(obs2, :); % remove rows
% | L2'| |M L2| |B A2| if DIFF
% Olowod = Olowod(obs2, :, :);
% which implies that LU'·LU = SU - M'·M. [varargout{1:2}] = deal(Suod, Olowod);
end
function [Lu,A,info,Lud,Ad] = omega_ltl(Su,Olow,p,q,ko,Sud,Olowd) end
DIFF = nargin>5;
LTT.LT = true; LTT.TRANSA = true;
n = length(ko)-1; % PROFILE_AR_TRISOLVE Pure AR Omega triangular solve for profile-sparse matrix
h = max(p,q); %
e = ko(h+1); % order of Su % X = PROFILE_AR_TRISOLVE(Lu, LSig, Y, JPAT, ko, km, p, g, 'N') solves Lo·X=Y
Lu = Su; % exploiting known sparsity in Y, which is such that Y = Yf(obs,miss) and the
A = Olow; % (i,j)-block of Yf is known to be zero for i > max(p,j+g). This function is
if DIFF % used to form Vhat (with g=0) and Laomh (with g=p) in the pure autoregressive
nPar = size(Sud,3); % case. Lu, LSig and JPAT come from omega_ar_factor and give the ltl-factor of
Lud = Sud; % Omega_o, ko(t) is the number of observed and km(t) the number of missing
Ad = Olowd; % values before time t. X = PROFILE_AR_TRISOLVE(...,'T') solves Lo'·X=Y.
end %
for k=n:-1:h+1 % [X,Xd] = PROFILE_AR_TRISOLVE(...,Lud,Lld,Yd) finds also the derivative of X.
K = (ko(k)+1 : ko(k+1)) - e; %
c = ko(k-q); % NOTE: The routine is implemented using a blocked algorithm for speed
for i = min(n,k+q):-1:k
b = ko(i-q); function [X,Xd] = profile_ar_trisolve(Lu,LSig,Y,JPAT,ko,km,p,g,cod,Lud,LSigd,Yd)
JD = K + e - c; DIFF = nargout > 1;
JU = (b+1 : ko(k)) - c; n = length(ko) - 1;
J = K + e - b; M = km(n+1);
if i>k blk = ceil(25*n/(M+1));
I = (ko(i)+1 : ko(i+1)) - e; X = Y;
JUD = (b+1 : ko(k+1)) - c; if DIFF, Xd = Yd; end
JMY = 1 : ko(k+1)-b; mY = size(Y,1);
A(K,JUD) = A(K,JUD) - A(I,J)'*A(I,JMY); for t1=1:blk:n
if (DIFF) t = min(n,t1+blk-1);
JM = 1 : ko(k)-b; if km(t+1) > km(t1)
Ad(K,JD,:) = Ad(K,JD,:) - ata_deriv(A(I,J), Ad(I,J,:)); tmax = min(n,max(p,t+g));
Ad(K,JU,:)=Ad(K,JU,:)-atb_deriv(A(I,J),Ad(I,J,:),A(I,JM),Ad(I,JM,:)); I = 1 : min(mY, ko(tmax+1));
end K = km(t1)+1:km(t+1);
else if ~DIFF
[A(K,JD), info] = ltl(A(K,JD)); X(I,K) = omega_ar_trisolve(Lu, LSig, Y(I,K), JPAT, ko, cod);
if (info~=0) return; end else
A(K,JU) = linsolve(A(K,JD), A(K,JU), LTT); [X(I,K),Xd(I,K,:)] = ...
if (DIFF) omega_ar_trisolve(Lu, LSig, Y(I,K), JPAT, ko,cod,Lud,LSigd,Yd(I,K,:));
Ad(K,JD,:) = ltl_deriv(A(K,JD), Ad(K,JD,:)); end
Ad(K,JU,:) = back_sub_deriv(A(K,JD),Ad(K,JD,:),A(K,JU),Ad(K,JU,:)); end
end end
end end
end
end
for i = h+1:min(n,h+q) % PROFILE_ATA Multiply A transposed with A where A is a profile-sparse matrix
b = ko(i-q); %
M = b+1:e; % C = PROFILE_ATA(A, ko, km, p, g) sets C to A'·A making use of known sparsity
I = (ko(i)+1 : ko(i+1)) - e; % in A which is such that A = Af(obs,miss) and the (i,j)-block of Af is known
JM = 1 : ko(k)-b; % to be zero for i > max(p,j+g). This function is used to form Vhat'·Vhat
Lu(M,M) = Lu(M,M) - A(I,JM)'*A(I,JM); % (with g=q) and Laomh'·Laomh (with g=p) efficiently. The vectors ko and km
if (DIFF) Lud(M,M,:) = Lud(M,M,:) - ata_deriv(A(I,JM), Ad(I,JM,:)); end % specify the sparsity structure, ko(t) is the number of observed and km(t)
end % the number of missing values before time t.
[Lu, info] = ltl(Lu); %
if (DIFF) Lud = ltl_deriv(Lu, Lud); end % [C,Cd] = PROFILE_ATA(A, ko, km, p, g, Ad) finds also the derivative of C.
end
function [C,Cd] = profile_ata(A, ko, km, p, g, Ad);
function [L,info] = ltl(A) n = length(ko)-1;
% Calculates LTL-factorization of A. Only uses lower triangle of A. Returns DIFF = nargout>1;
% info=0 on success. mA = size(A,1);
n = size(A,1); M = size(A,2);
[L,info] = chol(A(n:-1:1,n:-1:1)); C = zeros(M,M);
if info~=0, L=A; return; end if DIFF, nPar = size(Ad,3); Cd = zeros(M,M,nPar); end
L = L(n:-1:1,n:-1:1); blk = ceil(25*n/(M+1));
end for t1=1:blk:n
t = min(n,t1+blk-1);
function Ld = ltl_deriv(L, Ad) if km(t+1)>km(t1)
% Returns the derivative of the LTL-factorization of A in Ld. L should be the tmax = min(n,max(p,t+g));
% LTL-factor of A and Ad the derivative of A. I = 1 : min(mA, ko(tmax+1));
n = size(L,1); K = km(t1)+1:km(t+1);
Ld = chol_deriv(L(n:-1:1, n:-1:1)', permute(Ad(n:-1:1, n:-1:1, :), [2,1,3])); L = 1 : km(t+1);
Ld = permute(Ld(n:-1:1, n:-1:1, :), [2,1,3]); J = 1 : km(t1);
end C(K,L) = C(K,L) + A(I,K)'*A(I,L);
if DIFF
if ~isempty(J)
% OMEGA_REMOVE_MISS Remove rows and columns of missing values from Omega Cd(K,J,:) = Cd(K,J,:) + atb_deriv(A(I,K),Ad(I,K,:), A(I,J),Ad(I,J,:));
% end
% [Suo, Olowo] = OMEGA_REMOVE_MISS(Su, Olow, miss) removes rows and columns Cd(K,K,:) = Cd(K,K,:) + ata_deriv(A(I,K), Ad(I,K,:));
% given by the vector miss(:) from Omega, keeping only rows and columns end
% corresponding to observed values. Omega is stored in two parts, Su and Olow end
% as returned by omega_build, and the updated Omega is similarly returned in end
% two parts, a full upper left partition Suo and a sparse block-band lower part % Make upper triangular part:
% Olowo (which will in general have variable sized blocks). The r×n logical C = tril(C)+tril(C,-1)';
% matrix miss should be true in missing locations, thus specifiying the if DIFF
% structure of Olowo. for k=1:size(Cd,3), Cd(:,:,k) = tril(Cd(:,:,k))+tril(Cd(:,:,k),-1)'; end
% end
% [Suo,Olowo,Suod,Olowod] = OMEGA_REMOVE_MISS(Su,Olow,miss,Sud,Olowd) also end
% updates the derivative of Omega, stored in the three dimensional arrays Sud
% and Olowd as calculated by omega_build_deriv.
% PROFILE_BACK_SUB Omega back substitution for a profile-sparse matrix
function [Suo, Olowo, varargout] = omega_remove_miss(Su, Olow, miss, Sud, Olowd) %
[r, n] = size(miss); % X = PROFILE_BACK_SUB(Lu, Ll, Y, ko, km, p, q, g) solves Lo'·X = Y exploiting
ko = find_missing_info(miss); % known sparsity in Y, which is such that Y = Yf(obs,miss) and the (i,j)-block
DIFF = nargout > 2; % of Yf is known to be zero for i > max(p,j+g). This function is used to form
h = size(Su,1)/r; % Vhat (with g=q) and Laomh (with g=p). Lo is the ltl-factor of Omega_o (from
q = size(Olow,2)/r - 1; % omega_ltl), stored in Lu and Ll, and ko(t) is the number of observed and
observed = ~miss; % km(t) the number of missing values before time t..
obs1 = observed(:,1:h); %
obs2 = observed(:,h+1:end); % [X,Xd] = PROFILE_BACK_SUB(Lu, Ll, Y, ko, km, p, q, g, Lud, Lld, Yd) finds
Suo = Su(obs1, obs1); % copy Su-partit. (also OK because of how Matlab works) % also the derivative of X.
if DIFF
Suod = Sud(obs1, obs1, :); % NOTE: The routine is implemented using a blocked algorithm for speed
Olowod = zeros(size(Olowd));
end function [X,Xd] = profile_back_sub(Lu, Ll, Y, ko, km, p, q, g, Lud, Lld, Yd)
% REMOVE COLUMNS FROM LOWER PART(S): DIFF = nargout > 1;
ro = diff(ko); n = length(ko) - 1;
J = 1 : r; M = km(n+1);
K = h+1-q : h+1; blk = ceil(25*n/(M+1));
Olowo = zeros(size(Olow)); X = Y;
for t = 1:n-h if DIFF, Xd = Yd; end
obsk = observed(:,K); mY = size(Y,1);
m = sum(ro(K)); for t1=1:blk:n
Olowo(J, 1:m) = Olow(J, obsk(:)); t = min(n,t1+blk-1);
if DIFF, Olowod(J, 1:m, :) = Olowd(J, obsk(:), :); end if km(t+1) > km(t1)
J = J+r; tmax = min(n,max(p,t+g));

18
 I = 1 : min(mY, ko(tmax+1)); LTT.LT = true; LTT.TRANSA = true;
K = km(t1)+1:km(t+1); v1 = linsolve(LQ, v, LTT);
if ~DIFF v2 = linsolve(LR, u + K*v1, LTT);
X(I,K) = omega_back_sub(Lu, Ll, Y(I,K), p, q, ko); % Following is because Vhat and Laomh are not stored in full if there are
else % no missing values at the end of the time series.
[X(I,K),Xd(I,K,:)] = ... k1 = size(Vhat,1);
omega_back_sub(Lu, Ll, Y(I,K), p, q, ko, Lud, Lld,Yd(I,K,:)); k2 = size(Laomh,1);
end y = wohat;
end y(1:k1) = y(1:k1) + Vhat*(v1 - v2);
end y(1:k2) = y(1:k2) + Laomh*(Sm*v2);
end y = omega_forward(Luo, Llo, y, p, q, ko);
y = lambda_T_multiply(A, y, r, n, miss);
eps = C_multiply(C, y, n, miss);
% PROFILE_MULT Multiply with a profile-sparse matrix xm = Sm*v2;
% end
% C = PROFILE_MULT(A, B, ko, km, p, g, h) sets C to A'*B making use of known eps = reshape(eps, r, n);
% sparsity in A and B which is such that A = Af(obs,miss) and the (i,j)-block end
% of Af is known to be zero for i > max(p,j+g) and B = Bf(obs,miss) with the
% (i,j)-block of Bf zero for i > max(p,j+h). When B is full set h = n. This
% function is used to multiply efficiently with Vhat' (g=q) and Laomh' (g=p). % RES_MISS_AR Estimate residuals and missing values for AR-model
% The matrix P = Laomh'·Vhat is formed with g=p, h=q. ko(t) is the number of %
% observed and km(t) the number of missing values before time t. % [EPS] = RES_MISS_AR(A,Sig,Lu,LSig,z) returns the maximum likelihood estimate
% % of the residuals of the VAR model:
% [C,Cd] = PROFILE_MULT(A,B, ko, km, p, g, h, Ad,Bd) finds also the derivative %
% of C. % x(t) - mu = A1·(x(t-1) - mu) + ... + Ap·(x(t-p) - mu) + eps(t)
%
function [C,Cd] = profile_mult(A, B, ko, km, p, g, h, Ad, Bd); % where eps(t) is N(mu, Sig), mu is an r-vector, Sig is r×r and the A's are
n = length(ko)-1; % the coefficients of the model (p>0). The input parameters are the variables
DIFF = nargout>1; % with the same names in var_llc.
[k1,l] = size(A); [k2,M] = size(B); %
m = min(k1, k2); % [EPS] = RES_MISS_AR(A,Sig,Lu,LSig,woh,LR,LQ,K,u,v,Laomh,Vhat,Sm,miss) is
C = zeros(l,M); % used for the missing value case. [EPS, XM] = FIND_AR_RES_MISS(...) returns
if DIFF, nPar = size(Ad,3); Cd = zeros(l,M,nPar); end % also the maximum likelihood estimate of missing values.
blk = ceil(25*n/(M+1));
if h<n function [eps,xm] = res_miss_ar(A,Sig,Lu,LSig,z,LR,LQ,K,u,v,Laomh,Vhat,Sm,miss);
tA1 = 1; LTT.LT = true; LTT.TRANSA = true;
tB1 = 1; C = find_CGW(A, [], Sig);
while tA1<=n && tB1<=n if nargin == 5
tA = min(n,tA1 + blk - 1); r = size(Sig, 1);
tB = min(n,tB1 + blk - 1); p = length(A)/r;
kA = min(m, ko(min(n, max(p,tA+g)) + 1)); n = length(z)/r;
kB = min(m, ko(min(n, max(p,tB+h)) + 1)); jpat = ones(1,n-p);
if kA < kB y = omega_ar_trisolve(Lu, LSig, z, jpat, 0:r:n*r, 'T');
if km(tA+1) > km(tA1) y = lambda_T_multiply(A, y, r, n);
I = km(tA1)+1:km(tA+1); miss = zeros(r, n);
J = 1:kA; if nargout>1, xm = []; end
L = km(tB1)+1:km(n+1); else
C(I,L) = C(I,L) + A(J,I)'*B(J,L); [r, n] = size(miss);
if DIFF p = size(A,2)/r;
Cd(I,L,:) = Cd(I,L,:)+atb_deriv(A(J,I),Ad(J,I,:),B(J,L),Bd(J,L,:)); ko = find_missing_info(miss);
end [pat, ipat, jpat] = unique(miss(:, p+1:end)', 'rows');
end ko = find_missing_info(miss);
tA1 = tA1+blk; v1 = linsolve(LQ, v, LTT);
else v2 = linsolve(LR, u + K*v1, LTT);
if km(tB+1) > km(tB1) % Following is because Vhat and Laomh are not stored in full if there are
I = km(tB1)+1:km(tB+1); % no missing values at the end of the time series.
J = 1:kB; k1 = size(Vhat,1);
L = km(tA1)+1:km(n+1); k2 = size(Laomh,1);
C(L,I) = C(L,I) + A(J,L)'*B(J,I); y = z;
if DIFF y(1:k1) = y(1:k1) + Vhat*(v1 - v2);
Cd(L,I,:) = Cd(L,I,:)+atb_deriv(A(J,L),Ad(J,L,:),B(J,I),Bd(J,I,:)); y(1:k2) = y(1:k2) + Laomh*(Sm*v2);
end y = omega_ar_trisolve(Lu, LSig, y, jpat, ko, 'T');
end y = lambda_T_multiply(A, y, r, n, miss);
tB1 = tB1+blk; if nargout>1, xm = Sm*v2; end
end end
end eps = C_multiply(C, y, n, miss);
else eps = reshape(eps, r, n);
for t1=1:blk:n end
t = min(n,t1+blk-1);
if km(t+1) > km(t1)
tmax = min(n,max(p,t+g)); % S_BUILD Calculate covariance of x
I = 1 : min(m, ko(tmax+1)); %
K = km(t1)+1:km(t+1); % SS = S_BUILD(S,A,G,n) finds the covariance matrix of the vector x =
C(K,:) = C(K,:) + A(I,K)'*B(I,:); % [x1'...xn']' of all the values of a VARMA time series,
if DIFF %
XX = atb_deriv(A(I,K),Ad(I,K,:), B(I,:),Bd(I,:,:)); % x(t) = A1·x(t-1) + ... + Ap·x(t-p) + y(t)
Cd(K,:,:) = Cd(K,:,:) + XX; % where
end % y(t) = eps(t) + B1·eps(t-1) + ... + Bq·eps(t-q),
end %
end % and x(t), y(t) and eps(t) are r-dimensional with eps(t) N(0,Sig). S =
end % {S0,S1...Sp}, with Sj = Cov(x(t),x(t-j)) and G = {G0 G1...Gq} with Gj =
end % Cov(y(t),x(t-j)) should be previously calculated with find_CGW. A should
% contain [A1...Ap]. SS is the matrix:
%
% RES_MISS Estimate residuals and missing values % S0 S1' S2'...Sn-1'
% % S1 S0 S1'...Sn-2'
% EPS = RES_MISS(A, C, Lu, Ll, w) returns the maximum likelihood % S2 :
% estimate of the residuals of the VARMA model: % : :
% % Sn-1 ...... S1 S0
% x(t) - mu = A1·(x(t-1) - mu) + ... + Ap·(x(t-p) - mu) + y(t) %
% with % and the Sj are found with the recurrence relation:
% y(t) = eps(t) + B1·eps(t-1) + ... + Bq·eps(t-q), %
% % Sj = A1*S(j-1) + A2*S(j-2) + ........ + Ap*S(j-p) + Gj
% where eps(t) is N(mu, Sig), mu is an r-vector, Sig is r×r and the A's and %
% B's are the coefficients of the model. Lu, Ll, C and w are as in varma_llc. % with Gj = 0 for j > q.
%
% [EPS, XM] = RES_MISS(A,C,Luo,Llo,wohat,LR,LQ,K,u,v,Laomh,Vhat,Sm,miss) function SS = S_build(S, A, G, n)
% is used for the missing value case. The paramteres are as in varma_llm, and r = size(G{1},1);
% in addition to the EPS estimate, the maximum likelihood estimate of the p = length(S) - 1;
% missing values is returned in XM. q = length(G) - 1;
S = [S cell(1,n-p-1)];
function [eps,xm] = res_miss(varargin); A = makecell(A);
if nargin==5 for j=p+1:n-1
[A, C, Lu, Ll, w] = deal(varargin{:}); if j <= q, S{j+1} = G{j+1}; else S{j+1} = zeros(r,r); end
r = size(C{1}, 1); for i=1:p
p = length(A)/r; S{j+1} = S{j+1} + A{i}*S{j+1-i};
q = length(C) - 1; end
n = length(w)/r; end
what = omega_forward(Lu, Ll, w, p, q, 0:r:r*n); St = cell(1,n-1);
y = omega_back_sub(Lu, Ll, what, p, q, 0:r:r*n); Sc = cell(n-1,1);
y = lambda_T_multiply(A, y, r, n); S0 = S{1};
eps = C_multiply(C, y, n); for i=1:n-1
xm = []; St{i} = S{i+1}';
else Sc{i} = S{i+1};
[A, C, Luo, Llo, wohat, LR,LQ,K,u,v,Laomh,Vhat,Sm,miss] = deal(varargin{:}); end
[r, n] = size(miss); SS = cell(n,n);
p = length(A)/r; for i=1:n
q = length(C) - 1; SS(i,i) = {S0};
ko = find_missing_info(miss); SS(i,i+1:end) = St(1:n-i);

19
 SS(i+1:end,i) = Sc(1:n-i); if p==0, PLU={}; return, end % NOTHING TO DO (PURE MOVING AVERAGE)
end r = size(A{1},1);
SS = cell2mat(SS); F0r = cell(1,p-1);
end F0c = cell(p-1,1);
F = cell(p-1,p-1);
I = eye(r);
% S_EXTEND Extend S0...Sp from Yule-Walker equations to S0...S(n-1) % SET F TO IDENTITY MATRICES, F0c TO ZERO AND INITIALIZE F0r AND F00
% for i=1:p-1
% Scol = S_EXTEND(A, G, S, n) when A = [A0...Ap], G = {G0,...,Gq} and S = for j=1:p-1, F{i,j} = zeros(r^2,r^2); end
% {S0,...,Sp} calculates S(p+1)...S(n-1) and returns them together with F{i,i} = eye(r^2);
% S0,..., Sp in a block column vector Scol = [S0; S1;...; S(n-1)]. Thus Scol F0c{i} = zeros(r^2,r^2);
% will be n·r × r. F0r{i} = -kron(A{i},I) - kron(A{p},A{p-i});
% end
% [Scol, Scold] = S_EXTEND(A, G, S, n, Gd, Sd) returns also the n·r × r × nPar F00 = eye(r^2) - kron(A{p},A{p});
% derivatives of Scol. % MAIN LOOP. LOOP OVER SUBMATRICES (K,J), BLOCK-ROWS (i) AND BLOCK-COLUMNS (j)
K = 1:r;
function [Scol,Scold] = S_extend(A, G, S, n, Gd, Sd) J = 1:r:r^2;
DIFF = nargout > 1; for k = 1:r
r = size(G{1}, 1); for i=1:p-1
p = size(A,2)/r; for j=1:i-1 % SUBTRACT AHAT MATRICES FROM F
q = length(G)-1; F{i,i-j}(K,K) = F{i,i-j}(K,K) - A{j};
A = cell2mat(fliplr(makecell(A))); % change A from [A1..Ap] to [Ap..A1] end
if isempty(A), A = zeros(r,0); end for j=i+1:p % SUBTRACT ATILDA MATRICES FROM F
Scol = cat(1, S{:}, zeros(r*(n-p-1),r)); F{i,j-i}(K,J) = F{i,j-i}(K,J) - A{j};
if DIFF end
nPar = size(Sd{1},3); F0c{i}(K,K) = F0c{i}(K,K) - A{i}; % SUBTRACT AHATi FROM COLUMN 0
Scold = cat(1, Sd{:}, zeros(r*(n-p-1),r,nPar)); F0c{i}(K,J) = F0c{i}(K,J) - A{i}; % SUBTRACT ATILDAi FROM COLUMN 0
end end
pp = p+1; % SUBTRACT ATILDAp·AHATp FROM F00 & ADD TO DIAGONAL OF F00
K = r+1:r*pp; K1 = 1:r;
I = r*pp+1:r*(pp+1); for k1 = 1:r
for j = p+1:n-1 F00(K,K1) = F00(K,K1) - A{p}(:,k1)*A{p}(k,:);
if j <= q K1 = K1+r;
Scol(I,:) = G{j+1}; end
if DIFF, Scold(I,:,:) = Gd{j+1}; end F00(K(k),K(k)) = F00(K(k),K(k)) + 1;
end K = K+r;
Scol(I,:) = Scol(I,:) + A*Scol(K,:); J = J+1;
if DIFF, Scold(I,:,:) = Scold(I,:,:) + AGdiff(A, Scol(K,:), Scold(K,:,:)); end
end % CHANGE F0r, F0c, AND F FROM CELL-ARRAYS TO ORDINARY MATRICES
K = K+r; F0r = cell2mat(F0r);
I = I+r; F0c = cell2mat(F0c);
end if p <= 1, F0r = zeros(r^2,0); F0c = zeros(0,r^2); end
end F = cell2mat(F);
% REMOVE ROWS/COLUMNS CORRESPONDING TO UPPER-TRIANGLE FROM ROW 0/COLUMN 0
function Fd = AGdiff(A, G, Gd) J = [];
nPar = size(Gd, 3); for k=1:r, J = [J ((k-1)*r+k):k*r]; end
r = size(A,1); F = [
p = size(A,2)/r; F00(J,J) F0r(J,:)
Fd = reshape(A*reshape(Gd,r*p,r*nPar),r,r,nPar); F0c(:,J) F ];
k = 1; [L,U,P] = lu(F); [perm,dum] = find(P');
for i = p-1:-1:0 PLU = {perm,L,U};
for c=r*i+1:r*i+r if ~all(diag(U)), perm(1)=0; end
for l=1:r end
Fd(l,:,k) = Fd(l,:,k) + G(c,:);
k = k+1;
end % VYW_SOLVE Solve modified vector Yule-Walker equations
end %
end % S = VYW_SOLVE(A,PLU,G) returns a cell matrix S = {S0,S1,...,Sp} where Sj =
end % cov(x(t),x(t-j)). PLU is from vyw_factorize and G = {G0,G1,...,Gq} should
% have been found by find_CGW. The Sj are obtained by solving the modified
% vector Yule-Walker equations shown in vyw_factorize (with Gj=0 for q<j<=p in
% VYW_DERIV_RHS Rhs of equations for derivative of vector-Yule-Walker solution % case p>q).
% %
% RHS = VYW_DERIV_RHS(A,GGd,S) calculates the right-hand-side of equations % S = VYW_SOLVE(A,PLU,Y) where Y{i} is a three dimensional array, solves, for
% that determine the derivative of the Sj matrices calculated by VYW. A is a % each j, the system with right hand side [Y{1}(:,:,j), ... Y{p+1}(:,:,j)] and
% cell array of the AR parmeter matrices, GGd is an mds struct-array of Gi % returns the result in S{1}(:,:,j),..., S{p+1}(:,:,j). This call is used when
% derivatives (calculated with find_CGW_deriv), and S should be previously % calculating derivatives of S.
% calculated with vyw_solve. The i-th right-hand-side matrix is returned in
% the der components of the r×r mds structure RHS{i}, i=1,...,p+1. function S = vyw_solve(A,PLU,Y)
A = makecell(A);
function RHS = vyw_deriv_rhs(A, GGd, S) p = length(A);
r = size(GGd(1).mat,1); nY = length(Y); % nY is q+1 for original eqns and p+1 for derivative eqns
p = size(A,2)/r; if isstruct(Y), Y = {Y.mat}; end
q = length(GGd) - 1; if p==0, S={Y{1}}; return, end % NOTHING TO DO (PURE MOVING AVERAGE)
A = makecell(A); r = size(Y{1},1);
RHS = cell(1,p+1); nrhs = size(Y{1},3);
for j=0:p g = cell(p-1,nrhs);
if j<=q, R = GGd(j+1); end I = eye(r);
if j> q, R = mds_set_zero(p+q+1,r,r); end % INITIALIZE g
for i = 1:j for i=1:p-1
R = mds_add_prod(R, A{i}, S{j-i+1}, i); if i < nY
end g{i} = reshape(Y{i+1},r^2,nrhs);
for i = j+1:p else
R = mds_add_prod(R, A{i}, S{i-j+1}', i); g{i} = zeros(r^2,nrhs);
end end
RHS(j+1:j+1) = der2array(R); end
end for k = 1:nrhs
end if nY > p, Y{1}(:,:,k) = Y{1}(:,:,k) + A{p}*Y{p+1}(:,:,k)'; end
g0(:,k) = reshape(Y{1}(:,:,k)',r^2,1);
end
% VYW_FACTORIZE Set up and LU-factorize modified vector Yule-Walker equations % REMOVE ROWS/COLUMNS CORRESPONDING TO UPPER-TRIANGLE FROM ROW 0/COLUMN 0
% J = [];
% PLU = VYW_FACTORIZE(A) returns the LU-factorization of a matrix F of order for i=1:r, J = [J ((i-1)*r+i):i*r]; end
% r^2·p-r(r-1)/2 which may be used to solve the modified Yule-Walker equations: g = [g0(J,:); cell2mat(g)];
% % SOLVE
% S0 - A1*S1' + A2*S2' + A3*S3' + ... + Ap*Sp' = G0 (1.0) [perm,L,U] = deal(PLU{:});
% S1 - A1*S0 + A2*S1' + A3*S2' + ... + Ap*S(p-1)' = G1 (1.1) LT.LT = true;
% S2 - A1*S1 + A2*S0 + A3*S1' + ... + Ap*S(p-2)' = G2 (1.2) (1) UT.UT = true;
% S3 - A1*S2 + A2*S1 + A3*S0 + ... + Ap*S(p-3)' = G3 (1.3) s = linsolve(U,linsolve(L,g(perm,:),LT),UT); % a faster s = F\g;
% ... % COPY s TO S0 AND FURTHER TO S{1}
% Sp - A1*S(p-1) + A2*S(p-2) + ........ + Ap*S0 = Gp (1.p) S0 = zeros(r,r,nrhs);
% j = 1;
% where Gj = cov(y(t),x(t-j)) and Sj = cov(x(t),x(t-j)). The system is obtained for i=1:r
% by substituting Sp given by (1.p) into (1.0) and then making modifications to j1 = j+r-i;
% take into account that S0 is symmetric. The result is a linear system S0(i:r,i,:) = s(j:j1,:);
% j = j1+1;
% F·s = g end
% for k=1:nrhs
% where s and g are column vectors, g obtained from the Gj and s containing the S0(:,:,k) = S0(:,:,k) + S0(:,:,k)';
% elements of S0, S1,..., S(p-1) in column order with upper triangle elements end
% of S0 removed (i.e. s' = [S0(1,1),..., S0(1,r), S0(2,2),..., S0(r,r), S{1} = S0;
% S1(1,1),..., S1(1,r), S1(2,1),..., S(p-1)(r,r)]) % COPY REST OF s TO S{2},..., S{p}
% s = reshape(s(j:end,:),r,r,p-1,nrhs);
% PLU returns a cell array {perm, L, U} where perm specifies a permutation for i=1:p-1
% matrix P such that P·L·U = F. On entry A should be [A1...Ap]. S{i+1} = reshape(s(:,:,i,:),r,r,nrhs);
end
function PLU = vyw_factorize(A) % FINALLY DETERMINE Sp IN S{p+1}
A = makecell(A); if nY > p
p = length(A); S{p+1} = Y{p+1};

20
 else
S{p+1} = zeros(r,r,nrhs); function xmud = xmu_deriv(nPar, miss)
end [r, n] = size(miss);
for k=1:nrhs [ko, ro] = find_missing_info(miss);
for i=1:p obs = ~miss;
S{p+1}(:,:,k) = S{p+1}(:,:,k) + A{i}*S{p-i+1}(:,:,k); nobs = sum(obs(:));
end xmud = zeros(nobs, 1, nPar);
end for t=1:n
end J = find(~miss(:,t));
for i=1:ro(t)
% XMU_DERIV Derivatives of observed components of x minus mu xmud(ko(t)+i, 1, nPar-r+J(i)) = -1;
% end
% XMUD = XMU_DERIV(NPAR, MISS) returns derivatives of the observed components end
% of x - repmat(mu,n,1) w.r.t. all NPAR parameters, A1...Ap, B1...Bq, Sig, mu end
% (with matrices taken column by column using only the lower triangle of Sig).
% NPAR should be (p+q+1)·r^2 + r·(r+3)/2 and MISS(i,t) should be true if the
% value X(i,t) is missing. Only the derivatives w.r.t. mu are nonzero.

APPENDIX B. TEST PROGRAMS

% TEST_PRIMARY Script to run primary tests of Vauto package: % and gradients in cell arrays F = {f1...fn} and G = {g1...gn} where fi and gi
% are the function value and Jacobian of the i-th function. With this syntax
disp 'TESTING PRIMARY VAUTO FUNCTIONS VAR_LL, VARMA_LLC, VARMA_LLM AND % DMAX is a vector of the discrepancies.
VARMA_SIM'
test_varma_llc function [dmax,g,gnum] = diff_test(fun, x, varargin)
test_varma_llc_deriv [f,g] = fun(x, varargin{:});
test_var_ll if ~iscell(f), f={f}; g={g}; end
test_var_ll_jac Nf = length(f);
test_varma_llm n = length(x);
test_varma_jac eps = norm(x)*1e-6;
test_varma_llm_deriv xp = x;
test_varma_sim quiet xm = x;
disp('ALL PRIMARY TESTS SUCCEEDED') for i=1:n
xp(i) = x(i)+eps;
xm(i) = x(i)-eps;
% TEST_ALL Script to run all tests for Vauto package: fp = fun(xp, varargin{:});
fm = fun(xm, varargin{:});
disp 'RUNNING PRIMARY AND SECONDARY TESTS FOR VAUTO PACKAGE' if ~iscell(fp), fp = {fp}; fm = {fm}; end
% TEST ROUTINES NEEDED FOR COMPLETE-DATA LOG-LIKELIHOOD: for j=1:Nf
test_der2array_mds_set gnum{j}(:,i) = (fp{j} - fm{j})/(2*eps);
test_find_CGW end
test_vyw quiet xp(i) = x(i);
test_omega_building xm(i) = x(i);
test_omega_forward end
test_omega quiet for j=1:Nf
test_varma_llc assert(isequal(size(g{j}), size(gnum{j})));
test_parmatprod quiet assert(all(isfinite(g{j}(:))));
% Derivative routines: assert(all(isfinite(gnum{j}(:))));
test_find_CGW_deriv gmax = max(max(abs(g{j})));
test_vyw_deriv quiet if isempty(gnum{j}) && isempty(g{j})
test_trisolve_deriv dmax(j) = 0;
test_chol_deriv quiet elseif gmax<1e-13
test_omega_deriv quiet dmax(j) = max(max(abs(gnum{j}-g{j})));
test_varma_llc_deriv else
% dmax(j) = max(max(abs(gnum{j}-g{j})))/gmax;
% TEST MISSING VALUE ROUTINES: end
test_lambda_multiply quiet end
test_find_Sm end
test_find_lambda_om
test_find_Vhat
test_product_deriv % FIND_CGWS
test_omega_building miss %
test_atba_c % [C,G,W,S] = find_CGWS(A, B, Sig) finds the matrices Cj, Gj, Wj and Sj in
test_profile % cell arrays {C1...Cq}, {G1...Gq}, {W1...Wq} and {S1...Sp}.
test_profile_ar %
test_C_multiply % [C,G,W,S,Cd,Gd,Wd,Sd] = find_CGWS(A, B, Sig) returns also derivatives
test_omega_ar %
test_var_ll % The routine is used by the testing suite. It calls find_CGW (and
test_var_ll_jac % find_CGW_deriv) and the vyw-functions. See further comments in these.
test_varma_llm
test_varma_jac function [C,G,W,S,Cd,Gd,Wd,Sd] = find_CGWS(A, B, Sig);
test_varma_llm_deriv if nargout <=4
% [C,G,W] = find_CGW(A, B, Sig);
% MISC TESTS LUvyw = vyw_factorize(A);
test_varma_sim quiet S = vyw_solve(A, LUvyw, G);
% else
disp('ALL TESTS SUCCEEDED') [CCd, GGd, WWd] = find_CGW_deriv(A, B, Sig);
[C, Cd] = der2array(CCd);
[G, Gd] = der2array(GGd);
% ALMOSTEQUAL Check if two arrays are almost equal [W, Wd] = der2array(WWd);
% LUvyw = vyw_factorize(A);
% ALMOSTEQUAL(X, Y) returns true if X and Y are arrays of the same size and S = vyw_solve(A, LUvyw, G);
% with the same elements except for differences that may be attributed to RHS = vyw_deriv_rhs(A, GGd, S);
% rounding errors. The maximum allowed relative difference is 5·10^(-14) and Sd = vyw_solve(A, LUvyw, RHS);
% the maximum allowed absolute difference is also 5·10^(-14). The relative end
% difference is measured relative to the element in X or Y with largest end
% absolute value.

function close = almosteqal(x,y) % GETFLAGS Get flags from varargin

if iscell(x) %
if ~iscell(y), close=false; return, end % [ARGS, FLG1, FLG2...] = GETFLAGS(ARGS, 'FLAG1', 'FLAG2'...) sets FLG1 to true
x = cell2mat(x); % if FLAG1 is a string member of the cell array ARGS, otherwise to false, and
y = cell2mat(y); % similarly for FLAG2, FLAG3 etc. On exit all the FLAGS that are present have
end % been removed from ARGS.
M = max([1; abs(x(:)); abs(y(:))]); %
close = isequal(size(x),size(y)) && all(abs(x(:)-y(:)) < 5e-14*M); % To obtain flags with arguments (as in "plot(x,y,'linewidth',3)") use
end % [ARGS,FLG,VALUE] = GETFLAGS(ARGS, 'FLAG'). If FLAG is present in ARGS, FLG is
% set to true and the next argument is returned in VALUE. Both FLAG and VALUE
% are removed from ARGS. If FLAG is last in ARGS or not there, VALUE becomes
% DIFF_TEST Check analytic derivative against numerical derivative % empty. To process several flag-value pairs use multiple calls to GETFLAGS.
% %
% DMAX = DIFF_TEST(@FUN, X, PAR1, PAR2,...) checks the derivative of FUN at X. % EXAMPLE:
% FUN should be a function with header [F, G] = FUN(X, PAR1, PAR2,...) where % function somefunc(varargin)
% X is an m-vector, F is an n-dimensional function value and G is an n×m % [varargin, aflag, bflag] = getflags(varargin, '-a', '-b')
% matrix with the Jacobian of FUN at X. DMAX returns the maximum relative % [varargin, cflag, cval] = getflags(varargin, '-c')
% difference between G(i,j) and a numerical derivative calculated with the %
% central difference formula. % Assume now that somefunc is called with: somefunc('ABC','-b',2,'-c',5).
% % After the call to getflags, varargin will be {'ABC', 2}, aflag is false,
% [DMAX,G,GNUM] = DIFF_TEST(...) returns also the G from FUN and the numerical % bflag is true, cflag is true and cval is 5. Similarly, after the call:
% Jacobian. % somefunc ABC -b 2 -c 5
% % varargin will be {'ABC', '2'}, cval will be '5' and the other variables
% For efficiency it is possible to test a FUN that returns several functions % will be as before.

21
function [args,varargout] = getflags(args,varargin) nparmat = (nPar + r*(r-1)/2)/(r*r);
assert(nargout==nargin || nargout == 3 && nargin ==2); FS = mds_set_zero(nparmat,r,r);
n = length(args); FS.mat = F;
IC = false(n,1); FS.spcod(nparmat) = 'f';
for i=1:n, IC(i) = ischar(args{i}); end; m = 0;
Keep = true(n,1); for k = 1:nparmat-1
Jc = find(IC); for c=1:r
for k=1:length(varargin) for l=1:r
Iq = strmatch(varargin{k},args(IC),'exact'); m = m+1;
Keep(Jc(Iq)) = false; FS.der{k}{l,c} = Fd(:,:,m);
varargout{k} = ~isempty(Iq); end
if nargout > nargin end
if ~isempty(Iq) && length(args) > Jc(Iq), FS.spcod(k) = 'f';
varargout{2} = args{Jc(Iq)+1}; end
Keep(Jc(Iq)+1) = false; for c=1:r % Copy derivative w.r.t. Sigma, and adjust for symmetry
else m = m+1;
varargout{2} = []; FS.der{nparmat}{c,c} = Fd(:,:,m);
end for l=c+1:r
end m = m+1;
end FS.der{nparmat}{l,c} = Fd(:,:,m)/2;
args = args(Keep); FS.der{nparmat}{c,l} = Fd(:,:,m)/2;
end end
function [ll,lld] = loglik(theta, X, p, q, r, miss) end
[A, B, Sig, mu] = theta2mat(theta, p, q, r); end
if nargin<6, miss = isnan(X); end
if nargout==1
[ll, ok] = varma_llm(X, A, B, Sig, mu, miss); % TESTCASE Create testcases for VARMA likelihood calculation
else %
[ll, ok, lld] = varma_llm(X, A, B, Sig, mu, miss); % [A,B,Sig] = TESTCASE(NAME) returns a named test case in matrices A =
end % [A1...Ap], B = [B1...Bq] and Sig suitable for testing various components of
assert(ok); % the calculation of exact VARMA likelihood. TESTCASE SUMMARY prints a list of
end % the named testcases (with possible values for NAME)
%
% [A,B,Sig,p,q,r] = TESTCASE(NAME) returns also the dimensions of the case.
% MAKEMISSING Create missing value test cases %
% % [A,B,Sig,name] = TESTCASE(i) returns the i-th named testcase. To get also
% X = MAKEMISSING(X,i) creates a testcase for missing value functions by % dimensions use [A,B,Sig,p,q,r,name] = TESTCASE(i).
% assigning a subset of X the value nan. The chosen subset is determined by i %
% as follows: % n = TESTCASE('NUMBER') returns the number of different named testcases.
% i=-1 X(1,1) only is missing %
% -2 X(end,end) only is missing % [A,B,Sig,name] = TESTCASE('ALL') returns all the named testcases in three
% -3 first four values of X(:) missing % cell arrays; the i-th one is returned in A{i}, B{i} and Sig{i}. In addition
% -4 last four values of X(:) missing % name{i} returns the name of the i-th case.
% -5 or 'miss-5a' 5% missing, all in first quarter %
% -6 or 'miss-5b' same as i=5 % [A,B,Sig] = TESTCASE(p,q,r) returns an unnamed testcase with dimensions
% -7 or 'miss-25' 25% missing, single runs at start % p,q,r.
% >=0 i% of the values (chosen at random) missing (i<90)
% function [A,B,Sig,varargout] = testcase(varargin)
% (with r even, the miss-25 option makes half of the series have the first name = '';
% half of the values missing, with r odd (r+1)/2 series have a little less if nargin == 1
% than half of the values at the beginning missing) type = varargin{1};
else
function X = makemissing(X, i) p = varargin{1};
% Drop some values from X (make them missing by replacing with nan), with a q = varargin{2};
% pattern according to i. r = varargin{3};
switch i type = 'unnamed';
case 0, % nothing missing end
case -1, X(1) = nan; testcases = {
case -2, X(end) = nan; 'tinyAR'
case -3, X(1:4) = nan; 'tinyMA'
case -4, X(end-3:end) = nan; 'tinyARMA'
case {-5,'miss-5a'} 'smallAR'
L = length(X(:)); 'smallMA'
k = ceil(5*L/100); 'smallARMA1'
X(randset(0.25*L, k)) = nan; 'smallARMA2'
case {-6,'miss-5b'} 'mediumAR'
X = makemissing(X,5); 'mediumARMA1'
case {-7,'miss-25'} 'mediumARMA2'
[r,n] = size(X); 'mediumMA'
R = ceil(r/2); 'largeAR'
k = round(n*0.25*r/R); %'pivotfailure'
X(1:R, 1:k) = nan; };
otherwise N = length(testcases);
assert(i<=90) if ~ischar(type), name = testcases{type};
L = length(X(:)); elseif ~isequal(type,'all'), name=type; end
k = ceil(i*L/100); switch type
X(randset(L, k)) = nan; case 'number'
end A = N; % count of named cases
end return
case 'summary'
function S = randset(n, k) fprintf('No. Name p q r Stationary?\n');
% S set to a k-element random set of numbers in {1,2,...,n} for i = 1:N
S = unique(floor(n*rand(2*k,1))+1); fmt = '%-2i %-12s %2d %2d %2d %6s\n';
while length(S) < k, S = union(S, floor(n*rand(k,1))+1); end [A,B,Sig,name] = testcase(i);
J = randperm(length(S(:))); [r,p,q] = get_dimensions(A,B,Sig);
S = S(J(1:k)); if is_stationary(A, Sig) stat = 'yes'; else stat = 'no'; end
end fprintf(fmt, i, name, r, p, q, stat);
end
clear A B Sig name
% MAT2THETA Convert parameter matrices to a vector return
% case 'all'
% THETA = MAT2THETA(A,B,Sig) changes the parameter matrices of a VARMA model, A for i=1:N
% = {A1...Ap}, B = {B1...Bq} and Sig to a vector of all the parameters. It is a name{i} = testcases{i};
% utility used by functions that compare analytical derivatives with numerical [A{i}, B{i}, Sig{i}, p{i}, q{i}, r{i}] = testcase(name{i});
% ones. A matrix input A = [A1 ... Ap] and B = [B1 ... Bq] is also permitted. end
% if any([4,7] == nargout), varargout{nargout-3} = name; end
% THETA = MAT2THETA(A,B,Sig,MU) adds MU at the end THETA. if nargout > 4, [varargout{1:3}] = deal(p,q,r); end
return
function theta = mat2theta(A,B,Sig,mu) case 'unnamed'
r = size(Sig,1); A = rand(r,r*p)/(p*r)*2;
if iscell(A), A = cell2mat(A); end B = rand(r,r*q)/(q*r)*2;
if iscell(B), B = cell2mat(B); end Sig=hilb(r)+0.2*eye(r);
s = []; while ~is_stationary(A,Sig), A=A/2; end
for i=1:r, s = [s; Sig(i:r,i)]; end case {1,'tinyAR'}
theta = [A(:); B(:); s]; A = 0.5;
if nargin==4, theta = [theta; mu]; B = [];
end Sig = 0.8;
case {2,'tinyMA'}
A = [];
% MDS_SET_PARMAT Set matrix and derivative to given values B = 0.5;
% Sig = 0.8;
% FS = MDS_SET(F,Fd) sets FS to a structure appropriate for mds_add_prod. case {3,'tinyARMA'}
% FS.mat is set to F, FS.der{k} is set to Fd(:,:,r^2·(k-1)+1:r^2·k) and A = 0.4;
% FS.spcod(k) is set to 'f' for all k. The operation is the inverse of B = 0.4;
% der2array. Sig = 0.8;
case {4,'smallAR'}
function FS = mds_set(F,Fd) %A = [[0.3 0.1; 0.4 0.2],[0.1 0.1;0.1 0.2]];
r = size(F,1); A = [0.1 0.1; 0.1 0.1];
nPar = size(Fd,3); B = [];

22
 Sig = [2 1;1 3]; % TEST_CHOL_DERIV AND FORWARD_SUB_DERIV
case {5,'smallMA'} %
A = []; % TEST_CHOL_DERIV asserts that the analytical derivatives calculated with
B = [0.3 0.1 0.1 0.1; 0.3 0.1 0.1 0.1]; % chol_deriv and forward_sub_deriv agree with central difference
Sig = [2 1;1 2]; % approximations. Use TEST_CHOL_DERIV QUIET to suppress output.
case {6,'smallARMA1'}
A = [0.3 0.1; 0.4 0.2]; function test_chol_deriv(varargin);
B = [0.2 0.3; 0.2 0.3]; [args,quiet] = getflags(varargin,'quiet');
Sig = [2 1;1 2]; out = double(~quiet);
case {7,'smallARMA2'} fprintf('TESTING CHOL_DERIV AND FORWARD_SUB_DERIV...');
B = [[0.4 0.1; 0.2 0.1], [0.2 0.1; 0.3 0.1]]; %
A = [0.2 0.3; 0.2 0.3]; % FIRST TEST TWO-OUTPUT FEATURES:
Sig = [2 1;1 2]; n = 4;
case {8,'mediumAR'} A = gallery('lehmer',n);
A = [ Ad = cat(3,hilb(n),ones(n));
0.35 0.15 0.15 0.11 0.14 0.07; [L, Ld] = chol_deriv(A, Ad);
0.25 0.15 0.05 0.12 0.15 0.08; L = tril(L);
0.15 0.05 0.01 0.13 0.16 0.09]; assert(almostequal(L, chol(A)'));
B = []; Ld1 = chol_deriv(L, Ad);
Sig=[2.0 0.5 0.0; assert(almostequal(Ld, Ld1));
0.5 2.0 0.5; Ad2 = cat(3,tril(hilb(n)),tril(ones(n)));
0.0 0.5 1.0]; Ld2 = chol_deriv(L,Ad2);
case {9,'mediumARMA1'} for i=1:2, Ld(:,:,i)=tril(Ld(:,:,i)); end
A = [ for i=1:2, Ld2(:,:,i)=tril(Ld2(:,:,i)); end
0.15 0.10 0.05 0.11 0.14 0.17 0.01 0.04 0.06; assert(almostequal(Ld, Ld2));
0.16 0.11 0.06 0.12 0.15 0.18 0.02 0.05 0.08; %
0.17 0.12 0.07 0.13 0.16 0.19 0.03 0.06 0.09]; Y = rand(n,2);
B = fliplr(A); Yd = rand(n,2,2);
Sig=[2.0 0.5 0.0; [X, Xd] = forward_sub_deriv(L, Ld, Y, Yd);
0.5 2.0 0.5; assert(almostequal(L*X, Y));
0.0 0.5 1.0]; Xd1 = forward_sub_deriv(L, Ld, X, Yd);
case {10,'mediumARMA2'} assert(almostequal(Xd, Xd1));
B = [ %
0.15 0.10 0.05 0.11 0.14 0.17 0.01 0.04 0.06; % NOW COMPARE ANALYTIC AND NUMERICAL DERIVATIVES:
0.16 0.11 0.06 0.12 0.15 0.18 0.02 0.05 0.08; theta = [0.2 0.3 0.1];
0.17 0.12 0.07 0.13 0.16 0.19 0.03 0.06 0.09]; d(1) = diff_test(@fun_fact, theta, 1);
A = fliplr(B); d(2) = diff_test(@fun_fact, theta, n);
Sig=[2.0 0.5 0.0; d(3) = diff_test(@fun_forward, theta, 1, 1);
0.5 2.0 0.5; d(4) = diff_test(@fun_forward, theta, 1, 4);
0.0 0.5 1.0]; d(5) = diff_test(@fun_forward, theta, n, 1);
case {11,'mediumMA'} d(6) = diff_test(@fun_forward, theta, n, 4);
A = []; assert(max(d) < 1e-8)
B = [ disp(' OK')
0.35 0.25 0.15 0.11 0.14 0.17; end
0.25 0.15 0.05 0.12 0.15 0.18;
0.15 0.05 0.01 0.13 0.16 0.19]; function [f,g] = fun_fact(theta, n)
Sig=[2.0 0.5 0.0; B1 = gallery('lehmer',n); %...to find some use for these gallery functions
0.5 2.0 0.5; B2 = repmat(1,n,n);
0.0 0.5 1.0]; B3 = wilkinson(n);
case {12,'largeAR'} C = gallery('minij',n);
r=7; p=5; rand('state',6); p = 1:n;
A = 1.8*rand(r,r*p)/r/p; A = C + theta(1)*B1 + theta(2)*B2 + theta(3)*B3;
Sig=hilb(r)+0.2*eye(r); Ad = cat(3,B1,B2,B3);
B = []; L = chol(A)';
case {'pivotfailure'} % create almost singular vyw equations Ld = chol_deriv(L, Ad);
A = [ f = vech(L);
-0.6250 0.2400 0.2500 0.1250 0.6250 0.3750 g = vech(Ld);
0 0 0.1250 0.5000 0.2500 0.1250 end
0.2500 0 0.1430 0 0.2500 0.2500];
A(1,1) = -0.62999813363195223; %% as singular as can be made function [f,g] = fun_forward(theta, n, m)
B=[]; B1 = gallery('lehmer',n); %...to find some use for these gallery functions
Sig = eye(3); B2 = repmat(1,n,n);
otherwise B3 = wilkinson(n);
error('no such testcase') C = gallery('minij',n);
end p = 1:n;
if any([4,7] == nargout), varargout{nargout-3} = name; end A = C + theta(1)*B1 + theta(2)*B2 + theta(3)*B3;
if nargout > 4 Ad = cat(3,B1,B2,B3);
r = size(Sig,1); L = chol(A)';
p = size(A,2)/r; Ld = chol_deriv(L, Ad);
q = size(B,2)/r; %
[varargout{1:3}] = deal(p, q, r);end Y1 = ones(n,m);
assert(isequal(Sig,Sig')); Y2 = repmat(1:m,n,1);
assert(min(eig(Sig))>0); Y3 = repmat((1:n)',1,m);
end Y = theta(1)*Y1 + theta(2)*Y2 + theta(3)*Y3;
Yd = cat(3,Y1,Y2,Y3);
%
% TEST_ATBA_C Test function atba_c X = L\Y;
% Xd = forward_sub_deriv(L, Ld, X, Yd);
% TEST_ATBA_C checks that ATBA_C works correctly. f = X(:);
g = reshape(Xd,n*m,3);
function test_atba_c end
fprintf('TESTING ATBA_C...');
A1 = rand(4,3);
A2 = rand(4,3); % TEST_C_MULTIPLY Tests function C_multiply
B1 = hilb(4); % TEST_LAMBDA_MULTIPLY checks that C_multiply returns the right value.
B2 = pascal(4);
C1 = gallery('lehmer',3); function test_C_multiply
C2 = gallery('minij',3); fprintf('TESTING C_MULTIPLY... ');
x = [0.74; 0.33]; rand('seed',1);
d = diff_test(@fun, x, A1, A2, B1, B2, C1, C2); r = 3;
assert(all(d<1e-8)); n = 4;
A1 = zeros(0,3); A2 = A1; C0 = [1 2 3; 2 3 4; 3 4 5];
d = diff_test(@fun, x, A1, A2, B1, B2, C1, C2); C1 = rand(r);
%d C2 = rand(r);
disp('OK'); O = zeros(r);
end C = {C0, C1, C2};
X = rand(n*r,1);
function [f,g] = fun(x,A1,A2,B1,B2,C1,C2) miss = false(r,n);
[n,m] = size(A1); CC = [C0 C1' C2' O ;
A = x(1)*A1 + x(2)*A2; Ad = cat(3, A1, A2); O C0 C1' C2';
B = x(1)*B1 + x(2)*B2; Bd = cat(3, B1, B2); O O C0 C1';
C = x(1)*C1 + x(2)*C2; Cd = cat(3, C1, C2); O O O C0];
B = B(1:n,1:n); Y = C_multiply(C, X, n, miss);
Bd = Bd(1:n,1:n,:); assert(almostequal(Y, CC*X));
C = C(1:m,1:m); %
Cd = Cd(1:m,1:m,:); % TRY MISSING VALUES
D = atba_c(A, B, C); miss([1,2,4,8]) = true;
[f,h] = atba_c(A, B, C, Ad, Bd, Cd); obs = find(~miss);
f = vech(f); Y = C_multiply(C, X(obs), n, miss);
g = vech(h); assert(almostequal(Y, CC(:,obs)*X(obs)));
assertEqual(tril(D), tril(A'*B*A + C)); fprintf('OK\n');
assertEqual(f, vech(D)); end
end

function assertEqual(A,B)
assert(isequal(size(A),size(B)) && all(abs(A(:)-B(:)) < 1e-14));
end

23
% TEST_DER2ARRAY_MDS_SET Check functions der2array and mds_set % TEST_FIND_CGW_DERIV Test find_CGW_deriv
%
function test_der2array_mds_set % TEST_FIND_CGW_DERIV prints out "OK" if the analytical derivative found with
fprintf('TESTING DER2ARRAY AND MDS_SET.... '); % find_CGW_deriv agree with numerical differentiation.
test(0,1);
test(1,1); function test_find_CGW_deriv
test(0,2); fprintf('TESTING FIND_CGW_DERIV... ');
test(1,2); d = 0;
test(2,2); for p=0:2:2
disp('OK') for q=0:2:2
end for r=1:2
d = max(d,test(p,q,r));
function test(ppq,r); end
nPar = (ppq)*r^2 + r*(r+1)/2; end
F = rand(r,r); end
Fd = rand(r,r,nPar); assert(d<1e-8);
% PARTIAL TEST OF MDS_SET fprintf('max-diff=%.1e, OK\n', d)
FS = mds_set(F,Fd); end
assert(almostequal(FS.mat,F));
assert(almostequal(FS.der{1}{1,1},Fd(:,:,1))); function d = test(p,q,r);
if ppq>0, assert(almostequal(FS.der{2}{1,1},Fd(:,:,r^2+1))); end nPar = r^2*(p+q)+r*(r+1)/2;
if ppq>0 && r>1 x = rand(nPar,1);
assert(almostequal(FS.der{1}{2,1},reshape(Fd(:,:,2),r,r))); d = diff_test(@fun,x,p,q,r);
assert(almostequal(FS.der{1}{1,2},reshape(Fd(:,:,r+1),r,r))); end
end
% TEST DER2ARRAY FOR ONE-ELEMENT INPUT function [f,g] = fun(x,p,q,r)
[F1,Fd1] = der2array(FS); [A,B,Sig] = theta2mat(x,p,q,r);
Fd2 = der2array(FS); if nargout==1
assert(almostequal(F,F1{1})); [C,G,W] = find_CGW(A,B,Sig);
assert(almostequal(Fd,Fd1{1}) && almostequal(Fd,Fd2{1})); else
% TEST DER2ARRAY FOR TWO-ELEMENT INPUT [CCd,GGd,WWd] = find_CGW_deriv(A, B, Sig);
G = rand(r,r); [C,Cd] = der2array(CCd);
Gd = rand(r,r,nPar); [G,Gd] = der2array(GGd);
GS = mds_set(G,Gd); [W,Wd] = der2array(WWd);
[F1,Fd1] = der2array([FS GS]); g = [vec(cell2mat(Cd)); vec(cell2mat(Gd)); vec(cell2mat(Wd))];
assert(almostequal(F,F1{1})); end
assert(almostequal(G,F1{2})); f = [vec(cell2mat(C)); vec(cell2mat(G)); vec(cell2mat(W))];
assert(almostequal(Fd,Fd1{1})); end
assert(almostequal(Gd,Fd1{2}));
end
% TEST_FIND_LAMBDA_OM Check that lambda_om is calculated correctly
%
% TEST_FIND_CGW Test find_CGW which finds lagged cross-covariance matrices % TEST_FIND_LAMBDA_OM checks that functions find_Acol and find_lambda_om
% % return the correct values. Runs testcases 1-9
% TEST_FIND_CGW prints out "OK" if find_CGW works OK for three test cases with %
% three different values of (p,q,r) (find_CGW finds the (lagged) % TEST_FIND_LAMBDA_OM(k) tries only testcase k
% cross-covariance matrices of (xt,epst), (yt,xt) and (yt,yt)). It may also be
% useful for debugging. TEST_FIND_CGW also does a partial test the function function test_find_laom(caseno)
% find_CGW_deriv, to be exact it tests that the function value returned by it fprintf('TESTING FIND_ACOL AND FIND_LAMBDA_OM... ');
% is the same as find_CGW returns. Further tests are carried out by randn('state',1); rand('state',1);
% test_vyw_deriv. if nargin<1, cases=1:9; else cases=caseno; end
for tcase = cases
function test_find_CGW [A, B, Sig, p, q, r, name] = testcase(tcase);
fprintf('TESTING FIND_CGW... ') n = p+q+3;
A1 = [1 1; 1 1]; Acol = find_Acol(A, r);
A2 = [2 2; 1 1]; Lam = lambda_build(A, r, n);
B1 = [1 1; 2 2]; %
B2 = [2 2; 2 2]; % Nothing missing:
B3 = [3 3; 2 2]; miss = false(r,n);
Sig = [3 1; 1 3]; Laom = find_lambda_om(Acol, miss);
r = size(A1,1); assert(isempty(Laom));
I = eye(r); %
% % Two cases with missing cases (fraction missing given by PCT):
% check p=2, q=3: PCT = [0.10 0.25];
[C,G,W] = find_CGW([A1,A2],[B1,B2,B3],Sig); for k = 1:2
[CCd,GGd,WWd] = find_CGW_deriv([A1,A2],[B1,B2,B3],Sig); miss(unique(floor(r*n*rand(round(r*n*PCT(k)),1)) + 1)) = true;
assert(almostequal({CCd.mat}, C) && almostequal({GGd.mat}, G) &&... obs = ~miss;
almostequal({WWd.mat},W)); Laom = find_lambda_om(Acol, miss);
% LaomOK = Lam(obs,miss);
Wb = W; mLaom = size(Laom, 1);
C0 = C{1}; C = C(2:end); assert(equal(Laom, LaomOK(1:mLaom, :)));
G0 = G{1}; G = G(2:end); assert(all(all(LaomOK(mLaom+1:end,:)==0)))
W0 = W{1}; W = W(2:end); end
assert(length(G) == 3); %
assert(length(W) == 3); % Compare derivatives with numerical ones:
assert(almostequal(G0, Sig + B1*C{1}' + B2*C{2}' + B3*C{3}')); [d,g,gnum] = diff_test(@laomfun, mat2theta(A, B, Sig), p, q, r, miss);
assert(almostequal(G{1},B1*Sig + B2*C{1}' + B3*C{2}' )); % g=g{1},gnum=gnum{1}
assert(almostequal(G{2},B2*Sig + B3*C{1}' )); assert(d<1e-8);
assert(almostequal(G{3},B3*Sig )); end
B0 = I; disp('OK');
assert(almostequal(W0 , B0*Sig*B0' + B1*Sig*B1' + B2*Sig*B2' + B3*Sig*B3')); end
assert(almostequal(W{1}, B1*Sig*B0' + B2*Sig*B1' + B3*Sig*B2' ));
assert(almostequal(W{2}, B2*Sig*B0' + B3*Sig*B1' )); function [f,g] = laomfun(theta, p, q, r, miss);
assert(almostequal(W{3}, B3*Sig*B0' )); [A, B, Sig] = theta2mat(theta,p,q,r);
% nPar = r^2*(p+q) + r*(r+1)/2;
% check p = 0, q=3 if nargout == 1
[C,G,W] = find_CGW([],[B1,B2,B3],Sig); Acol = find_Acol(A, r);
[CCd,GGd,WWd] = find_CGW_deriv([],[B1,B2,B3],Sig); Laom = find_lambda_om(Acol, miss);
assert(almostequal({CCd.mat}, C) && almostequal({GGd.mat}, G) &&... else
almostequal({WWd.mat},W)); [Acol, Acold] = find_Acol(A, r, nPar);
% [Laom, Laomd] = find_lambda_om(Acol, miss, Acold);
C0 = C{1}; C = C(2:end); g = reshape(Laomd,[],nPar);
G0 = G{1}; G = G(2:end); end
assert(almostequal(C0,Sig)); f = Laom(:);
assert(length(G) == 3); end
assert(almostequal(W,Wb));
assert(almostequal(G0 , Sig + B1*C{1}' + B2*C{2}' + B3*C{3}')); function e = equal(x, y)
assert(almostequal(G{1},B1*Sig + B2*C{1}' + B3*C{2}' )); e = isequal(size(x), size(y)) && isequal(x(:), y(:));
assert(almostequal(G{2},B2*Sig + B3*C{1}' )); end
assert(almostequal(G{3},B3*Sig ));
%
% check p=2, q=0: % TEST_FIND_SM Check that find_Sm works correctly.
[C,G,W] = find_CGW([A1,A2],[],Sig); %
[CCd,GGd,WWd] = find_CGW_deriv([A1,A2],[],Sig); % TEST_FIND_SM asserts that find_Sm returns the right thing for testcases 1 to
assert(almostequal({CCd.mat}, C) && almostequal({GGd.mat}, G) &&... % 7 (see testcase.m)
almostequal({WWd.mat},W)); %
% % TEST_FIND_SM(k) checks only testcase k
C0 = C{1}; C = C(2:end);
G0 = G{1}; G = G(2:end); function test_somsm_build(caseno)
W0 = W{1}; W = W(2:end); fprintf('TESTING FIND_SM...');
assert(isempty(G) && isempty(W)); if nargin<1, cases=1:7; else cases=caseno; end
assert(almostequal(C0,Sig)); for tcase = cases
assert(almostequal(G0,Sig)); [A, B, Sig, p, q, r, name] = testcase(tcase);
assert(almostequal(W0,Sig)); n = p+q+3;
% [C, G, W] = find_CGW(A, B, Sig);
disp('OK') LUvyw = vyw_factorize(A);
end S = vyw_solve(A, LUvyw, G);
Scol = S_extend(A, G, S, n);
SS = S_build(S, A, G, n);

24
 nPar = r^2*(p+q) + r*(r+1)/2; function [f,g] = vhatfun(theta,p,q,r,n,miss);
% [A, B, Sig] = theta2mat(theta,p,q,r);
% Call the function once (useful for debug): nPar = r^2*(p+q) + r*(r+1)/2;
miss = false(r, n); miss([1 3]) = true; [ko, ro, km] = find_missing_info(miss);
[f,g] = sommfun(mat2theta(A, B, Sig), p, q, r, n, miss); if nargout == 1
% [C,G,W,S] = find_CGWS(A, B, Sig);
% First try with nothing missing: Gneg = find_Gneg(A, B, C, n);
Sm = find_Sm(Scol, false(r,n)); Scol = S_extend(A, G, S, n);
assert(isempty(Sm)); [Su,Olow] = omega_build(S, G, W, p, n);
% [Suo, Olowo] = omega_remove_miss(Su, Olow, miss);
% Check sizes of derivatives: [Luo, Llo, info] = omega_ltl(Suo, Olowo, p, q, ko);
[CCd, GGd, WWd] = find_CGW_deriv(A, B, Sig); V = find_V(G, Gneg, Scol, miss);
RHS = vyw_deriv_rhs(A, GGd, S); Vhat = profile_back_sub(Luo, Llo, V, ko, km, p, q, q);
Sd = vyw_solve(A, LUvyw, RHS); else
[G, Gd] = der2array(GGd); [C,G,W,S,Cd,Gd,Wd,Sd] = find_CGWS(A, B, Sig);
[Scol,Scold] = S_extend(A, G, S, n, Gd, Sd); [Scol,Scold] = S_extend(A, G, S, n, Gd, Sd);
miss = false(r,n); miss([1,3,4]) = true; [Gneg, Gnegd] = find_Gneg(A, B, C, n, Cd);
[Sm, Smd] = find_Sm(Scol, miss, Scold); [Su,Olow] = omega_build(S, G, W, p, n);
assert(isequal(size(Smd), [3, 3, nPar])); [Sud, Olowd] = omega_build_deriv(Sd, Gd, Wd, p, n);
% [Suo, Olowo,Suod,Olowod] = omega_remove_miss(Su, Olow, miss, Sud,Olowd);
% Now two cases with ca. 25% missing: [Luo, Llo, info,Luod,Llod] = omega_ltl(Suo, Olowo, p, q, ko, Suod, Olowod);
for k = 1:2 [V, Vd] = find_V(G, Gneg, Scol, miss, Gd, Gnegd, Scold);
miss = false(r,n); [Vhat,Vhatd] = profile_back_sub(Luo,Llo, V, ko, km, p, q, q, Luod,Llod, Vd);
miss(unique(floor(r*n*rand(round(r*n/4),1)) + 1)) = true; g = reshape(Vhatd,[],nPar);
Sm = find_Sm(Scol, miss); end
assert(almostequal(Sm, SS(miss,miss))); f = Vhat(:);
end end
%
% Finally compare derivatives with numerical ones:
[d,g,gnum] = diff_test(@sommfun, mat2theta(A, B, Sig), p, q, r, n, miss); % TEST_LAMBDA_MULTIPLY Tests function lambda_multiply
assert(d<1e-8); %
end % TEST_LAMBDA_MULTIPLY checks that lambda_multiply returns the right value for
disp('OK'); % a few test-cases. Further checks of lambda_multiply are made by
end % test_omega_deriv and test_var_ll_jac.

function [f,g] = sommfun(theta, p, q, r, n, miss); function test_lambda_multiply(varargin)

[A, B, Sig] = theta2mat(theta,p,q,r); [args, qui] = getflags(varargin,'quiet'); out = double(~qui);
nPar = r^2*(p+q) + r*(r+1)/2; fprintf('TESTING LAMBDA_MULTIPLY... '); fprintf(out,'\n');
if nargout == 1 r = 1;
[C, G, W] = find_CGW(A, B, Sig); n = 4;
PLU = vyw_factorize(A); x = (1:n)';
S = vyw_solve(A, PLU, G); A = 1;
Scol = S_extend(A, G, S, n); Lam = [1 0 0 0;-A 1 0 0;0 -A 1 0;0 0 -A 1];
Sm = find_Sm(Scol, miss); W = lambda_multiply([A], x, false(r, n));
f = Sm(:); assert(almostequal(W, Lam*x));
else %
nmiss = sum(miss(:)); % TRY A MISSING VALUE:
nobs = n*r - nmiss; obs = [1,2,4]; miss = true(r, n); miss(obs) = false;
[C, G, W] = find_CGW(A, B, Sig); Wo = lambda_multiply(A, x(obs), miss);
[CCd, GGd, WWd] = find_CGW_deriv(A, B, Sig); assert(almostequal(Wo, Lam(obs,obs)*x(obs)));
PLU = vyw_factorize(A); x = (1:12)';
S = vyw_solve(A, PLU, G); n = 6; r = 2;
RHS = vyw_deriv_rhs(A, GGd, S); O = zeros(2);
Sd = vyw_solve(A, PLU, RHS); A1 = [0 1;1 1];
[G, Gd] = der2array(GGd); A2 = [2 3;2 2];
[Scol,Scold] = S_extend(A, G, S, n, Gd, Sd); A3 = [4 5;3 3];
[Sm, Smd] = find_Sm(Scol, miss, Scold); Lam = lambda_build([A1 A2 A3], r, n);
f = Sm(:); w = lambda_multiply([A1 A2 A3], x, false(r, n));
g = reshape(Smd, nmiss^2, []); assert(almostequal(w, Lam*x));
end %
end % LAMBDA_T_MULTIPLY:
W = lambda_T_multiply([A1 A2 A3], x, r, n);
assert(almostequal(W, Lam'*x));
% TEST_FIND_VHAT Check that Vhat is calculated correctly %
% % MORE MISSING VALUES:
% TEST_FIND_VHAT checks that the functions find_V and profile_back_sub return obs = [1 5 6 9 10 12];
% the correct Vhat matrix as well as its derivative. Runs testcases 1-7 miss = true(r,n); miss(obs) = false;
% Wo = lambda_multiply([A1 A2 A3], x(obs), miss);
% TEST_FIND_VHAT(k) tries only testcase k assert(almostequal(Wo, Lam(obs,obs)*x(obs)));
%
function test_find_vhat(caseno) % AND FOR LAMBDA_T_MULTIPLY:
fprintf('TESTING FIND_V AND PROFILE_BACK_SUB...'); Wo = lambda_T_multiply([A1 A2 A3], x(obs), r, n, miss);
randn('state',1); rand('state',1); assert(almostequal(Wo, Lam(obs,obs)'*x(obs)));
if nargin<1, cases=1:7; else cases=caseno; end obs = [1 3 4 5 8 10 11 12];
for tcase = cases miss = true(r,n);
[A, B, Sig, p, q, r, name] = testcase(tcase); miss(obs) = false;
n = p+q+3; Wo = lambda_T_multiply([A1 A2 A3], x(obs), r, n, miss);
[C, G, W, S] = find_CGWS(A, B, Sig); assert(almostequal(Wo, Lam(obs,obs)'*x(obs)));
Scol = S_extend(A, G, S, n); %
Gneg = find_Gneg(A, B, C, n); % NOW TEST WITH EMPTY A:
[Su,Olow] = omega_build(S, G, W, p, n); w1 = lambda_multiply([], x, false(r, n));
SS = S_build(S, A, G, n); assert(almostequal(w1, x));
Lam = lambda_build(A, r, n); %
% % DERIVATIVE WITH EMPTY A:
% First try with nothing missing: xd = rand(12,1,8);
ko = 0:r:r*n; [w2,wd] = lambda_multiply([], x, false(r, n), xd);
km = zeros(1,n+1); assert(almostequal(w1, w2));
miss = false(r,n); assert(almostequal(wd, xd));
[Lu, Ll, info] = omega_factor(Su, Olow, p, q, ko); %
V = find_V(G, Gneg, Scol, miss); % TEST DERIVATIVE FOR COMPLETE DATA AND ZERO xd
Vhat = profile_back_sub(Lu, Ll, V, ko, km, p, q, q); xd = zeros(12,1,8);
assert(isempty(Vhat)); [w, dd] = lambda_multiply([A1 A2], x, false(r, n), xd);
% dd1 = [
% Now some cases with missing values: -3 0 -4 0 -1 0 -2 0
for k = [-4:-1 20 40] 0 -3 0 -4 0 -1 0 -2
X = makemissing(ones(r,n), k); -5 0 -6 0 -3 0 -4 0
miss = isnan(X); 0 -5 0 -6 0 -3 0 -4
obs = ~miss; -7 0 -8 0 -5 0 -6 0
[ko,ro,km] = find_missing_info(miss); 0 -7 0 -8 0 -5 0 -6
[Suo, Olowo] = omega_remove_miss(Su, Olow, miss); -9 0 -10 0 -7 0 -8 0
[Luo, Llo, info] = omega_ltl(Suo, Olowo, p, q, ko); 0 -9 0 -10 0 -7 0 -8];
V = find_V(G, Gneg, Scol, miss); dd = squeeze(dd(5:12,1,:));
Vhat = profile_back_sub(Luo, Llo, V, ko, km, p, q, q); assert(almostequal(dd1, dd));
% %
V1 = Lam(obs,obs)*SS(obs,miss) + Lam(obs,miss)*SS(miss,miss); % DERIVATIVE FOR COMPLETE DATA AND NONZERO xd
Vhat1 = omega_back_sub(Luo, Llo, V1, p, q, ko); A = [A1 A2 A3];
Vx = zeros(size(Vhat1)); p = 3;
Vx(1:size(Vhat,1), :) = Vhat; nPar1 = 0;
assert(almostequal(Vx, Vhat1)); nPar = r^2*p + nPar1;
% xd = rand(length(x), 1, nPar);
% Finally compare derivatives with numerical ones: [w, wd] = lambda_multiply(A, x, false(r, n), xd);
if k>20 || tcase<4 % Just to save time [Lam, Lamd] = lambda_build(A, r, n, nPar);
[d,g,gnum] = diff_test(@vhatfun, mat2theta(A, B, Sig), p, q, r, n,miss); for k=1:nPar
assert(d<1e-8); assert(almostequal(wd(:,:,k), Lam*xd(:,:,k) + Lamd(:,:,k)*x));
end end
end %
end % NUMERICAL DERIVATIVE COMPARISON
disp('OK'); theta = [A(:); ones(nPar1,1)];
end [d,g,gnum] = diff_test(@wodfun, theta, xd, r, p, n, false(r,n));

25
 fprintf(out, 'Derivative test (1): %.1e\n', d); d1 = abs((ld - log(det(Omega)))/ld);
assert(d<1e-8); assert(d1 < 5e-15);
%
% AND MISSING VALUES WITH DERIVATIVE fprintf(out, ' max diff=%.1e', max(d));
x = x(obs); else
xd = xd(obs, :, :); fprintf(out, ' Omega not positive definite');
[d,g,gnum] = diff_test(@wodfun, theta, xd, r, p, n, miss); end
fprintf(out, 'Derivative test (2): %.1e\n', d); end
assert(d<1e-8); if length(cases)==1, disp(' OK'); return, end
% %
disp('OK'); % TEST NON-POSITIVE-DEFINITE FAILURE
end [A, B, Sig, p, q, r, name] = testcase('mediumARMA1');
n = 12;
function [f,g] = wodfun(theta, xd, r, p, n, miss) [C, G, W, S] = find_CGWS(A, B, Sig);
N = size(xd,1); [Su, Olow] = omega_build(S, G, W, p, n);
nPar = size(xd, 3); Olow(end,end) = -1;
x = reshape(xd, N, nPar)*theta; [Lu, Ll, info] = omega_factor(Su, Olow, p, q, 0:r:r*n);
A = reshape(theta(1:r^2*p),r,r*p); assert(info == n*r);
[w, wd] = lambda_multiply(A, x, miss, xd); Su(3,3) = -1;
f = w(:); [Lu, Ll, info] = omega_factor(Su, Olow, p, q, 0:r:r*n);
g = reshape(wd, N, nPar); assert(info == 3);
end fprintf(out, '\n');
disp(' OK');
end
% TEST_OMEGA Test omega_factor, omega_forward omega_back_sub and omega_logdet
%
% TEST_OMEGA checks that the functions omega_factor, omega_ltl, omega_forward, % TEST_OMEGA_AR Test omega functions for pure autoregressive case:
% omega_back_sub and omega_logdet work correctly for the test-cases that %
% "testcase" provides and no missing values. The missing value case is tested % TEST_OMEGA_AR checks the functions omega_ar_factor, omega_ar_trisolve and
% by test_omega_ar and test_varma_llm (and test_var_ll). % omega_ar_logdet, both function value and derivative value, by comparing to
% % the values returned by omega_factor, omega_forward and omega_logdet.
% TEST_OMEGA(n) runs only test case # n
function test_omega_ar(varargin)
function test_omega(varargin) [varargin,quiet] = getflags(varargin,'quiet');
[varargin, quiet] = getflags(varargin,'quiet'); out = double(~quiet);
out = double(~quiet); fprintf('TESTING OMEGA_AR_FACTOR, OMEGA_AR_TRISOLVE AND OMEGA_AR_LOGDET...');
fprintf('TESTING OMEGA_FACTOR, OMEGA_FORWARD AND OMEGA_LOGDET...'); randn('state',1); rand('state',1);
ncase = testcase('number'); p = 2; r = 3; n = 6;
if ~isempty(varargin) cases = varargin{1}; else cases = 1:ncase; end [A,B,Sig,p,q,r]=testcase(p,0,r);
for j = cases X = varma_sim(A, B, Sig, n);
[A, B, Sig, p, q, r, name] = testcase(j); X = makemissing(X, 12);
fprintf(out, '\n case %-11s r=%d p=%d q=%d ...', name, r, p, q); miss = isnan(X);
[C, G, W, S] = find_CGWS(A, B, Sig); miss1 = miss(:,1:p);
n = p+q+3; miss2 = miss(:,p+1:end);
[Su, Olow] = omega_build(S, G, W, p, n); PLU = vyw_factorize(A);
% S = vyw_solve(A, PLU, {Sig});
% TEST OMEGA_FACTOR AND OMEGA_LTL (by building a full Omega mat & comparing) %
h = max(p,q); Su = omega_build(S, {Sig}, {Sig}, p, p); % Test omega_build on finding Su only
[Lu, Ll, info] = omega_factor(Su, Olow, p, q, 0:r:r*n); assert(info==0) SS = S_build(S, A, {Sig}, p);
[Lu1, Ll1, info1] = omega_ltl(Su, Olow, p, q, 0:r:r*n); assert(info1==0) assert(almostequal(tril(Su),tril(SS)))
rh = r*h; %
clear LowLeft LowLeftf LowLeftf1 Omega Omegaf Omegaf1 % Test omega_ar_trisolve, complete data, not transp. (i.e. forward substition)
Omega(1:rh, 1:rh) = Su; ko = 0:r:r*n;
I = 1:r; jpat = ones(1, n-p);
J = 1:r*(q+1); [Su,Olow] = omega_build(S, {Sig}, {Sig}, p, n);
for i=1:n-h [Lu,Ll,info] = omega_factor(Su,Olow,p,q,ko);
LowLeft(I,J) = Olow(I,:); Y = rand(n*r,2);
LowLeftf(I,J) = Ll(I,:); Z = omega_forward (Lu, Ll, Y, p, q, ko);
LowLeftf1(I,J) = Ll1(I,:); LSig = {chol(Sig')'};
I = I + r; Zar = omega_ar_trisolve(Lu, LSig, Y, jpat, ko, 'NoT');
J = J + r; assert(almostequal(Z,Zar));
end %
width = min(size(LowLeft,2), r*n); % Test omega_ar_trisolve with derivatives, complete data
Omega(rh+1:r*n,r*n-width+1:r*n) = LowLeft(:,end-width+1:end); nPar = r^2*p + r*(r+1)/2;
Omega = tril(Omega) + tril(Omega,-1)'; Yd = rand(n*r,2,nPar);
[R ,cholinfo] = chol(Omega); SigS = mds_set_parmat(p+1, Sig, p+1);
R1 = flipud(fliplr(chol(flipud(fliplr(Omega)))))'; RHS = vyw_deriv_rhs(A, SigS, S);
if cholinfo==0 Sd = vyw_solve(A, PLU, RHS);
assert(cholinfo==0); Sigd = der2array(SigS);
Omegaf (1:rh, 1:rh) = Lu ; LSigd = chol_deriv(LSig{1}, Sigd{1});
Omegaf1(1:rh, 1:rh) = Lu1; [Sud, Olowd] = omega_build_deriv(Sd, Sigd, Sigd, p, n);
Omegaf (rh+1:r*n,r*n-width+1:r*n) = LowLeftf (:,end-width+1:end); [Lud, Lld] = omega_factor_deriv(Sud, Olowd, Lu, Ll, p, q, ko);
Omegaf1(rh+1:r*n,r*n-width+1:r*n) = LowLeftf1(:,end-width+1:end); Zd = omega_forward_deriv(Lu, Ll, Lud, Lld, Z, Yd, p, q, ko);
Omegaf = tril(Omegaf ) + tril(Omegaf ,-1)'; [Zar, Zard] = omega_ar_trisolve(Lu, LSig, Y, jpat, ko, 'NoT', Lud,{LSigd},Yd);
Omegaf1 = tril(Omegaf1) + tril(Omegaf1,-1)'; assert(almostequal(Zd,Zard));
d1 = norm(tril(R' -Omegaf )); %
d2 = norm(tril(R1'-Omegaf1)); % Check omega_ar_factor Lu:
assert(d1 < 1e-14); [Luo, LSig, jpat] = omega_ar_factor(S, Sig, miss);
assert(d2 < 1e-14); Luo = tril(Luo);
d = [d1 d2]; obs = ~miss1(:);
% Suo = Su(obs,obs);
% TEST OMEGA_FORWARD: assert(almostequal(tril(Luo*Luo'), tril(Suo)));
rand('state',0); %
Y = rand(r*n,3); % Check omega_ar_factor LSig:
X = omega_forward(Lu, Ll, Y, p, q, 0:r:r*n); npat = length(LSig);
d1 = norm(R'*X - Y); for t=p+1:n
assert(d1 < 1e-14); j = jpat(t-p);
d(end+1) = d1; obs = ~miss(:,t);
% assert(almostequal(LSig{j}*LSig{j}', Sig(obs,obs)));
% TEST OMEGA_BACK_SUB (BOTH EMPTY Y, FULL Y AND ENDING WITH ZEROS): end
X = omega_back_sub(Lu, Ll, zeros(r*n,0), p, q, 0:r:r*n); %
assert(isequal(size(X), [r*n,0])); % Check omega_ar_factor derivative:
X = omega_back_sub(Lu, Ll, zeros(0,3), p, q, 0:r:r*n); theta = mat2theta(A, B, Sig);
assert(isequal(size(X), [0,3])); dmax = diff_test(@fun, theta, p, r, miss);
Y1 = Y(1:r*(n-2), :); Y1z = [Y1; zeros(r*2,3)]; assert(dmax<1e-9);
Y2 = Y(1:r*2, :); Y2z = [Y2; zeros(r*(n-2),3)]; %
X = omega_back_sub(Lu, Ll, Y, p, q, 0:r:r*n); % Test omega_ar_trisolve with missing values
X1 = omega_back_sub(Lu, Ll, Y1, p, q, 0:r:r*n); [ko, ro, km] = find_missing_info(miss);
X2 = omega_back_sub(Lu, Ll, Y2, p, q, 0:r:r*n); nobs = ko(n+1);
d1 = norm(R*X - Y); Y = rand(nobs,2);
X1z = [X1; zeros(r*2,3)]; [Suo,Olowo,Suod,Olowod] = omega_remove_miss (Su, Olow, miss, Sud, Olowd);
X2z = [X2; zeros(r*(n-2),3)]; [Luo, Llo, info] = omega_factor (Suo,Olowo,p,q,ko); assert(info==0);
d2 = norm(R*X1z - Y1z); Z = omega_forward (Luo, Llo, Y, p, q, ko);
d3 = norm(R*X2z - Y2z); Zar = omega_ar_trisolve(Luo, LSig, Y, jpat, ko, 'NoT');
assert(d1 < 1e-14); assert(almostequal(Z,Zar));
assert(d2 < 1e-14); %
assert(d3 < 1e-14); % Test omega_ar_trisolve with derivatives
d = [d d1 d2 d3]; Yd = rand(nobs,2,nPar);
% [Luod, Llod] = omega_factor_deriv (Suod, Olowod, Luo, Llo, p, q, ko);
% TEST OMEGA_FORWARD WHEN Y STARTS WITH ZEROES Zd = omega_forward_deriv(Luo, Llo, Luod, Llod, Z, Yd, p, q, ko);
Y = [rand(r*(n-1),3)]; [Luo,LSig,jpat,Luod,LSigd] = omega_ar_factor(S,Sig,miss,Sd,Sigd{1});
X = omega_forward(Lu, Ll, Y, p, q, 0:r:r*n, 2); [Zar,Zard] = omega_ar_trisolve(Luo, LSig, Y, jpat, ko, 'NoT', Luod, LSigd,Yd);
O = zeros(r,3); assert(almostequal(Zd,Zard));
d1 = R'*[O; X] - [O; Y]; %
assert(norm(d1)<1e-14); % Check omega_ar_logdet
d(end+1) = norm(d1); [ld, ldd] = omega_logdet(Luo, Llo, p, q, ko, Luod, Llod);
% ldx = omega_ar_logdet(Luo, LSig, jpat);
% TEST OMEGA_LOGDET: [ldar, lddar] = omega_ar_logdet(Luo, LSig, jpat, Luod, LSigd);
ld = omega_logdet(Lu, Ll, p, q, 0:r:r*n); assert(almostequal(ld,ldx) && almostequal(ld,ldar) && almostequal(ldd,lddar));

26
 disp(' OK'); end
end disp('OK')
end
function [f,g] = fun(theta,p,r,miss)
nPar = r^2*p + r*(r+1)/2;
[A, B, Sig] = theta2mat(theta, p, 0, r); % TEST_OMEGA_DERIV Test complete data derivatives of Omega and its factors
PLU = vyw_factorize(A); %
S = vyw_solve(A, PLU, {Sig}); % TEST_OMEGA_DERIV checks the functions omega_build_deriv, omega_factor_deriv,
[Lu, LSig, jpat] = omega_ar_factor(S, Sig, miss); % omega_forward_deriv in the complete data case. In addition the derivative
f = vech(Lu); % features of lambda_multiply, omega_ltl, omega_back_sub and omega_logdet are
for i=1:length(LSig), f = [f; vech(LSig{i})]; end % tested, as well as the derivative of z'z (also only for complete data). Thus
if nargout > 1 % all the ingredients in the derivative of the log-likelihood function are
SigS = mds_set_parmat(p+1, Sig, p+1); % tested, thus providing a double-check in addition to the checks of
RHS = vyw_deriv_rhs(A, SigS, S); % test_varma_llc_deriv and test_varma_llm_deriv.
Sd = vyw_solve(A, PLU, RHS); %
Sigd = der2array(SigS); % By default a set of mostly r=2 testcases are generated and used. Specify
[Lu,LSig,jpat,Lud,LSigd] = omega_ar_factor(S,Sig,miss,Sd,Sigd{1}); % TEST_OMEGA_DERIV(n) to use the n-th named case from "testcase".
g = vech(Lud); %
for i=1:length(LSig), g = [g; vech(LSigd{i})]; end % Use "TEST_OMEGA_DERIV quiet" to suppress output.
end
end function test_omega_deriv(varargin)
[varargin,quiet] = getflags(varargin,'quiet');
randn('state', 8); rand('state', 8);
% TEST_OMEGA_BUILDING Test the functions omega_build and omega_remove_miss out = double(~quiet);
% pj = [0 1 1 1 1 2 2 2 2 2];
% TEST_OMEGA_BUILDING prints out OK (three times) if omega_build works ok for qj = [2 0 0 1 3 0 1 2 3 0];
% three test cases, all with r=2, p=3, n=7 but q is set to 2, 3 and 4 rj = [1 1 2 2 2 2 2 2 2 3];
% respectively. It may also be useful for debugging. fprintf('TESTING DERIVATIVES OF OMEGA, LL'', L''L, w, z, AND z''z...');
% fprintf(out,'\n');
% TEST_OMEGA_BUILDING QUIET runs more quietly fmt = ' %-28s = %.1e\n';
% ncase = testcase('number');
% TEST_OMEGA_BUILDING MISS tests also omega_remove_miss if ~isempty(varargin)
cases = varargin;
function test_omega_building(varargin) else
[varargin,quiet] = getflags(varargin,'quiet'); cases = num2cell([pj;qj;rj]);
[varargin,miss] = getflags(varargin,'miss'); end
out = double(~quiet && ~miss); for tcase = cases
S0=[3 4; 8 8]; [A, B, Sig, name] = testcase(tcase{:});
S1=[3 4; 1 1]; theta = mat2theta(A, B, Sig);
S2=[3 4; 2 2]; [p,q,r] = get_dimensions(A, B, Sig);
S3=[3 4; 3 3]; n = p+q+2;
G0=[4 5; 8 8]; fprintf(out, ' Testcase p=%d q=%d r=%d:\n', p, q, r);
G1=[4 5; 1 1]; [dmax,g,gnum] = diff_test(@fun, theta, p, q, r, n);
G2=[4 5; 2 2]; fprintf(out, fmt, 'Chol-factor max rel.diff', dmax(1));
G3=[4 5; 3 3]; fprintf(out, fmt, 'ltl max rel.diff', dmax(2));
G4=[4 5; 4 4]; fprintf(out, fmt, 'lambda_multiply max rel.diff', dmax(3));
W0=[5 6; 8 8]; fprintf(out, fmt, 'forward max rel.diff', dmax(4));
W1=[5 6; 1 1]; fprintf(out, fmt, 'backsub max rel.diff', dmax(5));
W2=[5 6; 2 2]; fprintf(out, fmt, 'logdet max rel.diff', dmax(6));
W3=[5 6; 3 3]; fprintf(out, fmt, 'z''z max rel.diff', dmax(7));
W4=[5 6; 4 4]; %gnum{2},g{2}
S={S0 S1 S2 S3}; assert(all(dmax < 2e-8));
G={G0 G1 G2}; end
W={W0 W1 W2}; disp(' OK');
r=2; p=3; q=2; n = 7; end
if ~miss, fprintf('TESTING OMEGA_BUILD...');
else fprintf('TESTING OMEGA_BUILD AND OMEGA_REMOVE_MISS...'); end function [f,g] = fun(theta,p,q,r,n)
fprintf(out, '\n test q<p...'); nPar = r^2*(p+q) + r*(r+1)/2;
[Su,Olow] = omega_build(S, G, W, p, n); ko = 0:r:r*n;
O = zeros(r); [A, B, Sig] = theta2mat(theta, p, q, r);
SuOK = [ [C, G, W, S] = find_CGWS(A, B, Sig);
S0 O O [Su, Olow] = omega_build(S, G, W, p, n);
S1 S0 O [Lu, Ll, info] = omega_factor(Su, Olow, p, q, ko);
S2 S1 S0]; if info ~= 0, error('Omega not positive definite'); end
OlowOK = [ [Mu, Ml, info] = omega_ltl(Su, Olow, p, q, ko); assert(info==0);
G2 G1 W0 X = reshape(1:r*n, r, n);
G2 W1 W0 w = lambda_multiply(A, X(:), false(r, n));
W2 W1 W0 z = omega_forward(Lu, Ll, w, p, q, ko);
W2 W1 W0]; zb = omega_back_sub(Lu, Ll, w, p, q, ko);
assert(isequal(tril(SuOK), tril(Su))) ld = omega_logdet(Lu, Ll, p, q, ko);
assert(isequal(OlowOK, Olow)) f{1} = [vech(Lu); getvec(Ll, r)];
f{2} = [vech(Mu); getvec(Ml, r)];
fprintf(out, 'OK\n test q=p+1...'); f{3} = w;
SOKB = [ f{4} = z;
S0 O O O f{5} = zb;
S1 S0 O O f{6} = ld;
S2 S1 S0 O f{7} = z'*z;
G3 G2 G1 W0]; if nargout>1
GB={G0 G1 G2 G3 G4}; [C, G, W, S, Cd, Gd, Wd, Sd] = find_CGWS(A, B, Sig);
WB={W0 W1 W2 W3 W4}; [Sud, Olowd] = omega_build_deriv(Sd, Gd, Wd, p, n);
[SuB,OlowB] = omega_build(S, GB, WB, p, n); [Lud, Lld] = omega_factor_deriv(Sud, Olowd, Lu, Ll, p, q, ko);
OlowOKB = [ [Mu, Ml, info, Mud, Mld] = omega_ltl(Su, Olow, p, q, ko, Sud, Olowd);
G4 G3 G2 W1 W0 xd = zeros(r*n, 1, nPar);
G4 G3 W2 W1 W0 [w, wd] = lambda_multiply(A, X(:), false(r, n), xd);
G4 W3 W2 W1 W0]; zd = omega_forward_deriv(Lu, Ll, Lud, Lld, z, wd, p, q, ko);
assert(isequal(tril(SOKB), tril(SuB))) [zb,zbd] = omega_back_sub(Lu, Ll, w, p, q, ko, Lud, Lld, wd);
assert(isequal(OlowOKB, OlowB)) [ld,ldd] = omega_logdet(Lu, Ll, p, q, 0:r:r*n, Lud, Lld);
g{1} = [vech(Lud); getvec(Lld,r)];
fprintf(out, 'OK\n test q=p+2...'); g{2} = [vech(Mud); getvec(Mld,r)];
SC = {S0 S1 S2}; g{3} = [squeeze(wd)];
[SuC,OlowC] = omega_build(SC, GB, WB, 2, n); g{4} = squeeze(zd);
SOKC = [ g{5} = squeeze(zbd);
S0 O O O g{6} = ldd;
S1 S0 O O g{7} = 2*z'*squeeze(zd);
G2 G1 W0 O end
G3 G2 W1 W0]; end
OlowOKC = [
G4 G3 W2 W1 W0 function x = getvec(Olow, r)
G4 W3 W2 W1 W0 % Convert Olow matrix to a vector.
W4 W3 W2 W1 W0]; m = size(Olow,1)/r;
assert(isequal(tril(SOKC), tril(SuC))) q = size(Olow,2)/r - 1;
assert(isequal(OlowOK, Olow)) M = size(Olow,3);
N1 = r^2*q;
if miss N2 = N1 + r*(r+1)/2;
miss = false(r, n); x = zeros(N2, m, M);
miss([1 3 4 7 9 10 14]) = true; K = 1:r;
SuoOK = [ for k = 1:m
8 0 0 if q>0, x(1:N1, k, :) = reshape(Olow(K, 1:q*r, :), N1,1,[]); end
4 3 4 x(N1+1:end, k, :) = vech(Olow(K, q*r+1:end, :));
2 8 8]; end
OlowoOK = [ x = reshape(x, N2*m, M);
1 1 8 end
6 5 6
2 8 8
5 6 5]; % TEST_OMEGA_FORWARD Check function omega_forward
ko = find_missing_info(miss); %
[Suo, Olowo] = omega_remove_miss(Su, Olow, miss); % TEST_OMEGA_FORWARD calls omega_forward with a few testcases. Further testing
assert(isequal(tril(SuoOK), tril(Suo))); % of omega_forward comes with test_omega.
assert(isequal(OlowoOK, Olowo(:,1:3)))

27
 fprintf(out, ' Max numerical/actual relative derivative difference:\n')
function test_omega_forward testcases={'A1','A2','A3','B1','B2','C1','C2','C3','D1','D2','D3','G1',...
fprintf('PRELIMINARY TEST OF OMEGA_FORWARD AND OMEGA_FORWARD_DERIV...') 'G2','H1','H2'};
% TEST SIMPLE CASES WITH r=1 AND (p,q) = (1,0), (1,1), (0,1). for i = 1:length(testcases)
% d(i) = diff_test(@fun, x, r, np, A, testcases{i});
% TEST CASE 1: fprintf(out, ' Testcase %s: %9.1e\n', testcases{i}, d(i));
L = [5 0 0; 0 2 0; 0 0 2]; assert(d(i)<1e-8);
Lu = 5; end
Ll = [2;2]; if nargout > 0, diffout = d; end
p = 1; % cover
q = 0; disp(' OK');
nPar = 2;
nY = 2; end
ko = [0 1 2 3];
y = rand(3,nY); function [f,g] = fun(x, r, np, A, tstcase)
x = omega_forward(Lu, Ll, y, p, q, ko); % returns a column m-vector f and an m×n Jacobian where n = length(x)
assert(almostequal(x, L\y)) x = x(:);
Lud = ones(1,1,nPar); P = reshape(x,r,[]);
Lld = ones(2,1,nPar); X = P(:,1:r);
Lld(:,:,1)=Lld(:,:,1)/2; Y = P(:,r+1:2*r);
yd = ones(3,nY,nPar); Z = P(:,2*r+1:3*r);
xd = omega_forward_deriv(Lu, Ll, Lud, Lld, x, yd, p, q, ko); a = A(:,1);
for i=1:nPar if nargout == 1
Ldi = diag([Lud(1,1,i);Lld(:,1,i)]); switch tstcase
assert(almostequal(L*xd(:,:,i), (yd(:,:,i) - Ldi*x))); case 'A1', F = X*Y;
end case 'A2', F = X*Y';
if exist('omega_forward_deriv')==3 % It is a dll-file, so try transpose also case 'A3', F = X*A;
ydt = permute(yd,[2,1,3]); case 'B1', F = X + Y*Z;
tmin=1; tmax=3; case 'B2', F = X*Y + Y*Z;
xdt = omega_forward_deriv(Lu, Ll, Lud, Lld, x', ydt, p, q,ko,tmin,tmax,'T'); case 'C1', F = X*(Y);
diff = xd - permute(xdt,[2,1,3]); case 'C2', F = X*(Y)';
assert(norm(diff(:)) < 1e-14); case 'C3', F = X*(X) + X*(X)';
end case 'D1', F = X*(Y*Z') + Z*(Y*Z');
% TEST CASE 2: case 'D2', F = X*(Y*X) + X*(Y*X)';
q = 1; case 'D3', F = X*(Y*Z')' + Z*(Y*Z')';
Ll = [1 2;1 2]; case 'G1', F = Z + X*(Y + Y*Z + X*(X*Y+X*Y*Z))';
L = [5 0 0; 1 2 0; 0 1 2]; case 'G2', F = Z*(X*Y + Y*Z + X*(X*Y + Y*Z)');
x = omega_forward(Lu, Ll, y, p, q, ko); case 'H1', F = X*a;
assert(almostequal(x, L\y)) case 'H2', F = Y*(X*[A -A]);
% otherwise, error(['unknown case ' tstcase]);
p = 0; end
x = omega_forward(Lu, Ll, y, p, q, ko); f = F(:)';
assert(almostequal(L*x, y)) else
% switch tstcase
% TEST KMIN-KMAX FEATURE: case 'A1' % F = X*Y
y = [1;1]; F = setprod(np,X,Y,1,2);
x = omega_forward(Lu, Ll, y, p, q, ko, 2); case 'A2' % F = X*Y'
assert(almostequal([0;x], L\[0;y])); F = setprod(np,X,Y,'T',1,2);
x = omega_forward(Lu, Ll, y, p, q, ko, 1, 2); case 'A3' % F = X*A
assert(almostequal(x, L(1:2,1:2)\y)) F = setprod(np,X,A,1);
y = 3; case 'B1' % F = X + Y*Z
x = omega_forward(Lu, Ll, y, p, q, ko, 1, 1); G = mds_set_parmat(np,X,1);
assert(almostequal(x, L(1,1)\y)); F = mds_add_prod(G,Y,Z,2,3);
x = omega_forward(Lu, Ll, y, p, q, ko, 2, 2); case 'B2' % F = X*Y + Y*Z
assert(almostequal(x, L(2,2)\y)) G = setprod(np,X,Y,1,2);
x = omega_forward(Lu, Ll, y, p, q, ko, 3, 3); F = mds_add_prod(G,Y,Z,2,3);
assert(almostequal(x, L(3,3)\y)) case 'C1' % F = X*(Y)
% G = mds_set_parmat(np,Y,2);
% NOW LET r=2 F = setprod(np,X,G,1);
r = 2; case 'C2' % F = X*(Y)'
O = zeros(r,r); G = mds_set_parmat(np,Y,2);
Lu = chol(hilb(r))'; F = setprod(np,X,G,'T',1);
L1 = ones(r,r); case 'C3' % F = X*(X) + X*(X)'
L0 = chol(gallery('minij',r))'; G = mds_set_parmat(np,X,1);
Ll = [L1 L0; L1 L0]; F = setprod(np,X,G,1);
ko = 0:r:3*r; F = mds_add_prod(F,X,G,'T',1);
L = [Lu O O; L1 L0 O; O L1 L0]; case 'D1' % F = X*(Y*Z') + Z*(Y*Z')
nY = 1; G = setprod(np,Y,Z,'T',2,3);
Y = ones(3*r,nY); F = setprod(np,X,G,1);
X = omega_forward(Lu, Ll, Y, p, q, ko); % p=1, q=1 F = mds_add_prod(F,Z,G,3);
assert(almostequal(X, L\Y)) case 'D2' % F = X*(Y*X) + X*(Y*X)'
G = setprod(np,Y,X,2,1);
% Check derivatives with r=2: F = setprod(np,X,G,1);
nPar = 1; F = mds_add_prod(F,X,G,'T',1);
Lud = tril(ones(size(Lu))); case 'D3' % F = X*(Y*Z')' + Z*(Y*Z')'
L0d = tril(ones(r,r)); G = setprod(np,Y,Z,'T',2,3);
L1d = ones(r,r); F = setprod(np,X,G,'T',1);
Lld = [L1d L0d; L1d L0d]; F = mds_add_prod(F,Z,G,'T',3);
Ld = [Lud O O; L1d L0d O; O L1d L0d]; case 'G1' % F = Z+Y*(Y+Y*Z+X*(X*Y+X*Y*Z))'
if nPar==2 G0 = setprod(np,Y,Z,2,3);
Lud = cat(3, Lud, Lud/2); G1 = setprod(np,X,Y,1,2);
Lld = cat(3, Lld, Lld/2); G1 = mds_add_prod(G1,X,G0,1);
Ld = cat(3,Ld,Ld/2); G2 = mds_set_parmat(np,Y,2);
end G2 = mds_add_prod(G2,Y,Z,2,3);
Yd = ones(3*r,nY,nPar); G2 = mds_add_prod(G2,X,G1,1);
Xd = omega_forward_deriv(Lu, Ll, Lud, Lld, X, Yd, p, q, ko); F = mds_set_parmat(np,Z,3);
for i=1:nPar F = mds_add_prod(F,X,G2,'T',1);
assert(almostequal(L*Xd(:,:,i), (Yd(:,:,i) - Ld(:,:,i)*X))); case 'G2' % F = Z*(X*Y + Y*Z + X*(X*Y + Y*Z)');
end G = setprod(np,X,Y,1,2);
G = mds_add_prod(G,Y,Z,2,3);
disp('OK'); G = mds_add_prod(G,X,G,'T',1);
end F = setprod(np,Z,G,3);
case 'H1' % F = X*a;
% TEST_PARMATPROD Test parameter matrix product derivative functions F = setprod(np,X,a,1);
% case 'H2'
% TEST_PARMATPROD tests mds_set_zero, mds_set_parmat and mds_add_prod. Use G = setprod(np,X,[A -A],1);
% TEST_PARMATPROD QUIET to reduce output. F = setprod(np,Y,G,2);
% otherwise
% For mds_add_prod(FS,X,GS,kX) and mds_add_prod(FS,X,GS,'T',kX) the tests are error(['unknown case ' tstcase]);
% designed to cover all possible combinations of spcod and transpose on G both end
% for derivative w.r.t. X and w.r.t. another parameter. To ascertain this, f = F.mat(:);
% uncomment the block referring to cover both in mds_add_prod and here. nf = length(f);
g=[];
function diffout = test_parmatprod(verbosity) for k=1:np
% global cover g = [g reshape(cat(2,F.der{k}{:}),nf,[])];
% cover = [ end
% ' zercxf' end
% 'A ' end
% 'X '
% 'AT ' % SETPROD Set matrix and its derivative to a product of matrices
% 'XT ']; %
if nargin<1, out=1; else out=0; end % FS = SETPROD(nparmat,X,Y,kX,kY) sets FS to a structure appropriate for
fprintf('TESTING PARAMETER MATRIX PRODUCT DERIVATIVES...'), fprintf(out,'\n') % mds_add_prod with the product X·Y and its derivative w.r.t. all the nparmat
rand('state',1); % parameter matrices (nparmat should be p+q+1). The following can also be used,
r = 3; % with meanings as in mds_add_prod:
np = 3; % FS = SETPROD(nparmat,X,Y,'T',kX,kY) FS = SETPROD(nparmat,X,A,'T',kX)
n = np*r^2; % FS = SETPROD(nparmat,X,G,kX)
x = 10*rand(n,1); % FS = SETPROD(nparmat,X,G,'T',kX)
A = rand(r,r); % can also be used, with meanings as in mds_add_prod.

28
 assert(almostequal(LV,L'*V) && almostequal(LVd,atb_deriv(L,Ld,V,Vd)));
function FS = setprod(nparmat,X,Y,varargin) end
r = size(X,1); end
if ~isstruct(Y), n=size(Y,2); else, n=size(Y.mat,2); end disp('OK');
FS = mds_set_zero(nparmat,r,n); end
FS = mds_add_prod(FS,X,Y,varargin{:});
end function Lequal = Lequal(x,y)
Lequal = almostequal(vech(x),vech(y));
end
% TEST_PRODUCT_DERIV Test matrix product derivative functions
%
% TEST_PRODUCT_DERIV checks that aat_deriv, ata_deriv and atb_deriv, which % TEST_PROFILE_AR
% differentiate matrix products, work correctly. %
% TEST_PROFILE_AR tests the function profile_ar_trisolve with input obtained
function test_product_deriv % from omega_ar_factor for pure autoregressive cases from testcase.m and
fprintf('TESTING ATB_DERIV, ATA_DERIV AND AAT_DERIV...'); % several different missing value patterns for each.
A1 = rand(3,4); %
A2 = rand(3,4); % The results of profile_ar_trisolve is compared with that from
B1 = rand(4,5); % omega_ar_trisolve. See also test_profile.
B2 = rand(4,5); %
x = [0.74; 0.33]; % TEST_PROFILE_AR(N) uses only the N-th testcase.
d = diff_test(@fun, x, A1, A2, B1, B2, 2);
assert(all(d<1e-8)); function test_profile_ar(caseno)
disp('OK'); fprintf('TESTING PROFILE_AR_TRISOLVE... ');
end rand('state',2); randn('state',1);
if nargin<1, cases=1:11; else cases=caseno; end
function [f,g] = fun(x,A1,A2,B1,B2,nPar) for tcase = cases
A = x(1)*A1 + x(2)*A2; Ad = cat(3, A1, A2); Adt = permute(Ad,[2,1,3]); [A, B, Sig, p, q, r, name] = testcase(tcase);
B = x(1)*B1 + x(2)*B2; Bd = cat(3, B1, B2); Bdt = permute(Bd,[2,1,3]); if q==0
[f{1},h{1}] = atb_deriv(A', Adt, B, Bd); assert(almostequal(f{1}, A*B)); n = p+5;
[f{2},h{2}] = aat_deriv(A, Ad); assert(Lequal(f{2}, A*A')); nPar = r^2*p + r*(r+1)/2;
[f{3},h{3}] = ata_deriv(A, Ad); assert(Lequal(f{3}, A'*A)); [C, G, W, S, Cd, Gd, Wd, Sd] = find_CGWS(A, B, Sig);
f{1} = f{1}(:); [Scol, Scold] = S_extend(A, G, S, n, Gd, Sd);
g{1} = reshape(h{1},length(f{1}),nPar); [Gneg, Gnegd] = find_Gneg(A, B, C, n, Cd);
for i = 2:3 [Su,Olow] = omega_build(S, G, W, p, n);
f{i} = vech(f{i}); [Sud, Olowd] = omega_build_deriv(Sd, Gd, Wd, p, n);
for ip=1:nPar for k = 0:4
g{i}(:,ip) = vech(h{i}(:,:,ip)); miss = false(r,n);
end switch k
end case 0, pct = 0; % Missing percentage
end case 1, pct = 0.25;
case 2, pct = 0.13;
function e = Lequal(A,B) case 3, pct = 0.13;
e = almostequal(tril(A), tril(B)); otherwise pct = 0.20;
end end
miss(unique(floor(r*n*rand(round(r*n*pct),1)) + 1)) = true;
[ko,ro,km] = find_missing_info(miss);
% TEST_PROFILE N = ko(n+1);
% [Suo,Olowo,Suod,Olowod] = omega_remove_miss(Su,Olow,miss,Sud,Olowd);
% TEST_PROFILE checks the functions profile_ata, profile_mult and [V,Vd] = find_V(G, Gneg, Scol, miss, Gd, Gnegd, Scold);
% profile_back_sub for 9 testcases and 2 missing patterns for each case SigS = mds_set_parmat(p+1, Sig, p+1);
% Sigd = der2array(SigS);
% The result of profile_ata(V...) is compared with V'·V and ata_deriv, that of [Lu,LSig,jpat,Lud,LSigd] = omega_ar_factor(S, Sig, miss, Sd, Sigd{1});
% profile_mult(V,Z...) with V'·Z and atb_deriv and profile_back_sub is checked [Y,Yd] = omega_ar_trisolve(Lu, LSig, V, jpat, ko, 'N', Lud, LSigd, Vd);
% against omega_back_sub. V is calculated as in varma_llm to obtain a Y1 = profile_ar_trisolve(Lu, LSig, V, jpat, ko, km, p, 0, 'N');
% convenient test matrix (but could have been assigned random values from assert(almostequal(Y,Y1));
% scratch). [Y1,Y1d]=profile_ar_trisolve(Lu,LSig,V,jpat,ko,km,p,0,'N',Lud,LSigd,Vd);
assert(almostequal(Y,Y1))
function test_profile(caseno) assert(almostequal(Yd,Y1d))
fprintf('TESTING PROFILE_ATA, PROFILE_MULT AND PROFILE_BACK_SUB... '); end
rand('state',2); randn('state',1); end
if nargin<1, cases=1:9; else cases=caseno; end end
for tcase = cases disp('OK');
[A, B, Sig, p, q, r, name] = testcase(tcase); end
n = p+q+5;
nPar = r^2*(p+q) + r*(r+1)/2;
[C, G, W, S, Cd, Gd, Wd, Sd] = find_CGWS(A, B, Sig); % TEST_TRISOLVE_DERIV Test functions forward_sub_deriv and back_sub_deriv
[Scol, Scold] = S_extend(A, G, S, n, Gd, Sd); %
[Gneg, Gnegd] = find_Gneg(A, B, C, n, Cd); % TEST_TRISOLVE_DERIV does some preliminary testing of the trianglar solve
[Su,Olow] = omega_build(S, G, W, p, n); % derivative functions forward_sub_deriv and back_sub_deriv. Further tests are
[Sud, Olowd] = omega_build_deriv(Sd, Gd, Wd, p, n); % done by e.g. test_chol_deriv.
for k = 0:3
miss = false(r,n); function test_trisolve_deriv
switch k fprintf('PRELIMINARY TEST OF FORWARD_SUB_DERIV AND BACK_SUB_DERIV...')
case 0, pct = 0; % Missing percentage L = 1;
case 1, pct = 0.25; Ld = ones(1,1,2);
case 2, pct = 0.13; Z = 1;
case 3, pct = 0.13; Zd = ones(1,1,2);
end X = L\Z;
miss(unique(floor(r*n*rand(round(r*n*pct),1)) + 1)) = true; Y = L'\Z;
[ko,ro,km] = find_missing_info(miss); Xd = forward_sub_deriv(L,Ld,X,Zd);
N = ko(n+1); Yd = back_sub_deriv(L,Ld,Y,Zd);
[Suo,Olowo,Suod,Olowod] = omega_remove_miss(Su,Olow,miss,Sud,Olowd); assert(norm(Xd(:,:,1))<1e-14);
[Lu,Ll,info,Lud,Lld] = omega_ltl(Suo, Olowo, p, q, ko, Suod, Olowod); assert(norm(Xd(:,:,2))<1e-14);
[V,Vd] = find_V(G, Gneg, Scol, miss, Gd, Gnegd, Scold); assert(norm(Yd(:,:,1))<1e-14);
Y = profile_back_sub(Lu, Ll, V, ko, km, p, q, q); assert(norm(Yd(:,:,2))<1e-14);
Y1 = omega_back_sub(Lu, Ll, V, p, q, ko); %
assert(almostequal(Y,Y1)); L = chol(hilb(2))';
[Y,Yd] = profile_back_sub(Lu, Ll, V, ko, km, p, q, q, Lud, Lld, Vd); Ld = zeros(2,2,3);
[Y1,Y1d] = omega_back_sub(Lu, Ll, V, p, q, ko, Lud, Lld, Vd); for i=1:3, Ld(:,:,i) = tril(rand(2,2)); end
assert(almostequal(Y,Y1) && almostequal(Yd,Y1d)); Z = rand(2,2);
VV = profile_ata(V, ko, km, p, q); Zd = rand(2,2,3);
assert(Lequal(VV, V'*V)); X = L\Z;
[VV,VVd] = profile_ata(V, ko, km, p, q, Vd); Y = L'\Z;
[VV1,VV1d] = ata_deriv(V, Vd); Xd = forward_sub_deriv(L,Ld,X,Zd);
assert(Lequal(VV,VV1) && Lequal(VVd,VV1d)); Yd = back_sub_deriv(L,Ld,Y,Zd);
[Acol, Acold] = find_Acol(A, r, nPar); for i=1:3
[L,Ld] = find_lambda_om(Acol, miss, Acold); assert(almostequal(L*Xd(:,:,i) + Ld(:,:,i)*X, Zd(:,:,i)));
[mV,nV] = size(V); assert(almostequal(L'*Yd(:,:,i) + Ld(:,:,i)'*Y, Zd(:,:,i)));
[mL,nL] = size(L); end
V = [V; zeros(N-mV, nV)]; disp('OK')
L = [L; zeros(N-mL, nL)]; end
Vd = cat(1, Vd, zeros(N-mV, nV, nPar));
Ld = cat(1, Ld, zeros(N-mL, nL, nPar));
Z = rand(N,3); % TEST_VARMA_JAC Test Jacobian feature of varma_llc and varma_llm
Zd = rand(N,3,nPar); %
VZ = profile_mult(V, Z, ko, km, p, q, n); % TEST_VARMA_JAC compares gradients of varma_llc and varma_llm with numerical
LZ = profile_mult(L, Z, ko, km, p, p, n); % gradient when change of variables is used. It also compares the likelihood
VL = profile_mult(V, L, ko, km, p, q, p); % function value with the one obtained when no Jacobian is specified.
LV = profile_mult(L, V , ko, km, p, p, q);
assert(almostequal(VZ,V'*Z) && almostequal(LZ,L'*Z)) function test_varma_llc_jac
assert(almostequal(VL,V'*L) && almostequal(LV,L'*V)) fprintf('TESTING JACOBIAN FEATURE OF VARMA_LLC and VARMA_LLM...\n');
[VZ,VZd] = profile_mult(V, Z, ko, km, p, q, n, Vd, Zd); ncase = testcase('number');
[LZ,LZd] = profile_mult(L, Z, ko, km, p, p, n, Ld, Zd); randn('state',3); rand('state',3);
[VL,VLd] = profile_mult(V, L, ko, km, p, q, p, Vd, Ld); for r=1:2:3
[LV,LVd] = profile_mult(L, V, ko, km, p, p, q, Ld, Vd); for pq = {0 1; 0 2; 1 0; 1 2; 2 0; 2 1}'
assert(almostequal(VZ,V'*Z) && almostequal(VZd,atb_deriv(V,Vd,Z,Zd))); [p,q] = deal(pq{:});
assert(almostequal(LZ,L'*Z) && almostequal(LZd,atb_deriv(L,Ld,Z,Zd))); n = p+q+3;
assert(almostequal(VL,V'*L) && almostequal(VLd,atb_deriv(V,Vd,L,Ld))); X = rand(r,n);

29
 m1 = r*r*p; % A
m2 = r*r*q; % B function [ll,lld] = loglik(theta, X, p, q, r)
m3 = r*(r+1)/2; % Sig [A, B, Sig] = theta2mat(theta, p, q, r);
n1 = ceil(m1/5); % thA if nargout==1
n2 = ceil(m2/5); % thB [ll, ok] = varma_llc(X, A, B, Sig);
n3 = 2; % thSig else
S1 = gallery('lehmer',r); [ll, ok, lld] = varma_llc(X, A, B, Sig);
S2 = gallery('minij',r); end
th = (rand(n1 + n2 + n3, 1))/(p+q)/r/r; assert(ok);
Ja = rand(r^2*p, n1); end
Jb = rand(r^2*q, n2);
Jc = [vech(S1) vech(S2)];
J = blkdiag(Ja, Jb, Jc); % TEST_VARMA_LLM Test the missing value likelihood function varma_llm
Xm = makemissing(X, 8); %
thmu = [th; 0.1*(1:r)']; % TEST_VARMA_LLM compares likelihood calculated with varma_llm with a direct
d(1) = diff_test(@fun_c, th, X, p, q, r, J); % calculation for all test cases that "testcase" creates and
d(2) = diff_test(@fun_m, thmu, Xm, p, q, r, J); % six different missing value patterns.
fprintf(' p,q,r = %d %d %d; max-diff = %.1e\n',p,q,r,max(d)); % TEST_VARMA_LLM QUIET runs quietly
assert(max(d) < 1e-8); % TEST_VARMA_LLM N runs only the N-th named testcase from "testcase"
end % TEST_VARMA_LLM N PAT runs N-th case on M-th missing value pattern
end % TEST_VARMA_LLM FULL runs a thorough test with more combinations of (p,r,q)
disp(' OK') % and more missing value patterns
end
function test_varma_llm(varargin)
function [ll,lld] = fun_c(theta, X, p, q, r, J) [args,quiet] = getflags(varargin,'quiet');
[A, B, Sig] = theta2mat(theta, p, q, r, J); [args,full] = getflags(args,'full');
ll = varma_llc(X, A, B, Sig); out = double(~quiet);
if nargout>1 fprintf('TESTING VARMA_LLM'); if full, fprintf(' (COMPREHENSIVE CHECK)'); end
[ll1, ok, lld] = varma_llc(X, A, B, Sig, J); disp ' '
assert(ok); patt = [];
assert(almostequal(ll, ll1)); if full
end cases = {};
end for p=0:3, for q=0:3, for r=1:3, cases{end+1} = {p,q,r}; end, end, end
elseif ~isempty(varargin)
function [ll,lld] = fun_m(theta, X, p, q, r, J) cases = {varargin(1)};
miss = isnan(X); if length(varargin) > 1, patt = varargin{2}; end
[A, B, Sig, mu] = theta2mat(theta, p, q, r, J); else
ll = varma_llm(X, A, B, Sig, mu, miss); cases = num2cell(num2cell(1:testcase('number')-1));
if nargout>1 end
[ll1, ok, lld] = varma_llm(X, A, B, Sig, mu, miss, J); randn('state',1);
assert(ok); rand('state',1);
assert(almostequal(ll, ll1)); maxdiff = 0;
end % Check non-stationary models:
end A = [0.5 0.51]; B = [];
X = zeros(1,10); miss = isnan(X);
% TEST_VARMA_LLC Compare varma_llc with as311 mu = 0;
% Sig = -1; [l, ok] = varma_llm(X, A(1), B, Sig, mu, miss); assert(~ok);
% TEST_VARMA_LLC checks that varma_llc returns the same value as as311 for all Sig = 1; [l, ok] = varma_llm(X, A, B, Sig, mu, miss); assert(~ok);
% test cases that "testcase" creates. Also compares with likelihood calculated fmt = ' Testcase %s, max diff.=%.1e, max res.diff.=%.1e\n';
% directly from S matrix. for j = 1:length(cases)
tcase = cases{j};
function test_varma_llc if full, tcstring = sprintf('p=%d q=%d r=%d', tcase{:});
fprintf('TESTING VARMA_LLC...\n'); else tcstring = int2str(tcase{1}); end
ncase = testcase('number'); [A, B, Sig, p, q, r, name] = testcase(tcase{:});
randn('state',1); mu = 0.01*(1:r)';
disp(' Case varma_llc full as311 rel.diff.') r = size(Sig,1);
for j=1:ncase-2 n = max(p,q)+5;
[A, B, Sig, name] = testcase(j); X = varma_sim(A, B, Sig, n, mu);
r = size(Sig,1); mubar = repmat(mu, n, 1);
n = 10; [SS, CC] = SC_build(A, B, Sig, n);
X = varma_sim(A, B, Sig, n); maxdiff(j) = 0; rdiff(j) = 0; mdiff(j) = 0;
[logelf,F1,F2,a,ifault] = as311(X, A, -B, Sig, [], 1, 1, -1); if ~isempty(patt), misspat = patt;
[l, ok, res] = varma_llc(X, A, B, Sig, 'res'); elseif ~full, misspat = [0 -4 -3 -2 -1 20];
assert(ok) elseif full, misspat = [0 -4 -3 -2 -1 2 4 10 15 20 25]; end
[lS, resS] = llresS(A, B, Sig, X); for i = misspat
ldiff = norm([l-lS,l-logelf,lS-logelf],inf)/abs(l); X1 = makemissing(X,i);
rdiff = norm(res-resS); miss = isnan(X1);
fmt = ' %-12s %12.9f %12.9f %12.9f %.1e %.1e\n'; if all(miss(:)==0) % CHECK THAT LLM AND LLC RETURN THE SAME:
fprintf(fmt, name, l, logelf, lS, ldiff,rdiff); zmu = zeros(r,1);
assert(ldiff<1e-14 && rdiff<5e-14); [l, ok] = varma_llm(X1, A, B, Sig, zmu, miss);
end assert(ok)
disp(' OK'); l1 = varma_llc(X1, A, B, Sig);
end assert(almostequal(l, l1));
end
function [lS, resS] = llresS(A, B, Sig, X) [l, ok] = varma_llm(X1, A, B, Sig, mu, miss); assert(ok)
% likelihood and residuals directly from S matrix assert(~isnan(l));
n = size(X,2); % CHECK FOR SAME LIKELIHOOD WHEN RES AND/OR XM ARE ALSO RETURNED:
r = size(Sig,1); [l1, ok, res, xm] = varma_llm(X1, A, B, Sig, mu, miss, 'res_miss');
q = length(B)/r; assert(ok);
x = X(:); assert(almostequal(l, l1));
[C,G,W,S] = find_CGWS(A, B, Sig); if j==2 % check that only 'res' works
SS = S_build(S, A, G, n); [l, ok, res1] = varma_llm(X1, A, B, Sig, mu, miss, 'res');
lS = (-n*r*log(2*pi) - log(det(SS)) - x'*(SS\x))/2; assert(ok && isequal(res,res1));
end
CC = cell(n,n); obs = ~miss;
for k=1:n nObs = sum(obs(:));
for j=1:n, CC{k,j} = zeros(r,r); end SSo = SS(obs,obs);
for j=0:min(q,k-1) xo = reshape(X(obs),[],1) - mubar(obs);
CC{k-j, k} = C{j+1}'; lS = (-nObs*log(2*pi) - log(det(SSo)) - xo'*(SSo\xo))/2;
end resS = reshape(CC(:, obs)*(SSo\xo), r, n);
end xmS = SS(miss,obs)*(SSo\xo) + mubar(miss);
CC = cell2mat(CC); maxdiff(j) = max(maxdiff(j), reldiff(l,lS));
resS = reshape(CC*(SS\x), r, n); rdiff(j) = max(rdiff(j), reldiff(res,resS));
end if ~isempty(miss), mdiff(j) = max(mdiff(j), reldiff(xmS,xm)); end
end;
fprintf(out, fmt, tcstring, maxdiff(j), rdiff(j));
% TEST_VARMA_LLC_DERIV end
% fmt = ' Overall, max diff.=%.1e, max res.diff.=%.1e, max xmiss diff=%.1e, ';
% TEST_VARMA_LLC_DERIV compares gradient of log-likelihood function with a fprintf(fmt, max(maxdiff), max(rdiff), max(mdiff));
% numerical gradient for two testcases (the gradient is further checked with disp ' '
% test_omega_deriv). disp ' ("max diff" is the difference between varma_llm function value and a'
disp ' value calculated directly from the large covariance matrix S=Cov(x))'
function test_varma_llc_deriv(varargin) assert(max(maxdiff)<1e-14);
disp('TESTING VARMA_LLC_DERIV...') assert(max(rdiff)<2e-14);
disp(' Max analytic / numerical gradient difference:') assert(max(mdiff)<2e-14);
if nargin==0, cases = {'mediumAR', 'mediumARMA1'}; disp(' OK');
else cases = num2cell(varargin{1}); end end
for tcase = cases
[A, B, Sig, p, q, r, name] = testcase(tcase{:}); function r = reldiff(a,b)
n = 10; assert(isequal(size(a),size(b)));
rand('state',0); randn('state',0); if isempty(a)
X = varma_sim(A, B, Sig, n); r = 0;
theta = mat2theta(A, B, Sig); else
d = diff_test(@loglik, theta, X, p, q, r); rmx = max([1,max(abs(a(:))), max(abs(b(:)))]);
fprintf(' Testcase %-12s %.1e\n', [name ':'], d); r = max(abs(a(:)-b(:)))/rmx;
assert(d<1e-8) end
end end
disp(' OK')
end function [SS, CC] = SC_build(A, B, Sig, n) % cov(x,x) and cov(x,eps)

30
 [C,G,W,S] = find_CGWS(A, B, Sig); SS = S_build(S, A, G, h+3); % theoretical covariance
r = size(Sig,1); d = max(abs(SSd(:)-SS(:)))/max(abs(SS(:)));
q = length(B)/r; dmax = max(dmax,d);
SS = S_build(S, A, G, n); if ~quiet
CC = cell(n,n); disp(' Data covariance for multiple series of length h+3:')
for k=1:n for j=1:r, fprintf(1,' %6.3f', SSd(j,:)); disp(' '); end
for j=1:n, CC{k,j} = zeros(r,r); end fprintf(' Max relative difference: %.1f%%\n', d*100);
for j=0:min(q,k-1) end
CC{k-j, k} = C{j+1}'; % CHECK CC_BUILD ON A SERIES OF LENGTH 3*h; COMPARE CC WITH DATA COV(X,EPS)
end CC = CC_build(A, C, h);
end [X,eps] = varma_sim(A, B, Sig, 2*h, [], M);
CC = cell2mat(CC); if r==1, X=shiftdim(X,-1); end
end if r==1, eps=shiftdim(eps,-1); end
X1 =reshape(X (:,1:h,:),r*h,[]); X2 =reshape(X (:,end-h+1:end,:),r*h,[]);
ep1=reshape(eps(:,1:h,:),r*h,[]); ep2=reshape(eps(:,end-h+1:end,:),r*h,[]);
% TEST_VARMA_LLM_DERIV CC1=cov([X1' ep1']); CC2=cov([X2', ep2']);
% CC1=CC1(1:r*h,r*h+1:end); CC2=CC2(1:r*h,r*h+1:end);
% TEST_VARMA_LLM_DERIV compares gradient of missing value log-likelihood d1 = max(abs(CC1(:)-CC(:)))/max(abs(CC(:)));
% function with a numerical gradient for a few testcases. d2 = max(abs(CC2(:)-CC(:)))/max(abs(CC(:)));
dmax = max([dmax,d1,d2]);
function test_varma_llm_deriv if ~quiet
disp('TESTING VARMA_LLM_DERIV...') disp(' Theoretical cov(x,eps):')
disp(' Max analytic / numerical gradient difference:') for j=1:r*h, fprintf(1,' %6.3f',CC(j,:)); disp ' '; end
rand('state',2); randn('state',3); disp(' CC1:')
for j = 1:8 for j=1:r*h, fprintf(1,' %6.3f',CC1(j,:)); disp ' '; end
[A, B, Sig, p, q, r, name] = testcase(j); disp(' CC2:')
mu = 0.1*(1:r)'; for j=1:r*h, fprintf(1,' %6.3f',CC2(j,:)); disp ' '; end
n = p+q+4; fprintf(' Max relative difference: %.1f%%\n', max(d1,d2)*100);
X = varma_sim(A, B, Sig, n, mu); end
if r>=3, I=[-1 0]; elseif r==2, I=[-4 0]; else I=[-4:0 20]; end %
for i=I % TEST STARTING FROM GIVEN X'S
Xm = makemissing(X,i); X0 = 2+repmat((1:r)', 1, h);
miss = isnan(Xm); X = varma_sim(A, B, Sig, h+2, mu, M, X0); % simulated forecast
theta = mat2theta(A, B, Sig, mu); if r==1, X=shiftdim(X,-1); end
d(i+5) = diff_test(@loglik, theta, Xm, p, q, r, miss); X1 = reshape(X(:,h+1,:),r,M);
if d(i+5)>=1e-8, d(i+5), end X2 = reshape(X(:,h+2,:),r,M);
assert(d(i+5)<1e-8) [Ex1,Ex2,Vx1] = expX(A,B,C,X0,mu,SS(1:r*h,1:r*h),Sig); % theoretical E & V
end d1 = max(abs(Ex1 - mean(X1,2)))/max(abs(Ex1)); % difference one step ahead
fprintf(' Testcase %-12s %.1e\n', name, max(d)); d2 = max(abs(Ex2 - mean(X2,2)))/max(abs(Ex2)); % difference two steps ahead
end d3 = max(max(abs(Vx1-cov(X1'))))/max(abs(Vx1(:))); % x1 covar difference
disp(' OK') if ~quiet
end fprintf(' Max one-step-ahead forecast difference: %.1f%%\n', d1*100);
fprintf(' Max two-step-ahead forecast difference: %.1f%%\n', d2*100);
fprintf(' Max one-step-ahead forecast var difference: %.1f%%\n', d3*100);
% TEST_VARMA_SIM Check that varma_sim and CC_build work as documented end
% dmax = max([dmax,d1,d2,d3]);
% TEST_VARMA_SIM prints theoretical covariance, data covariance of simulated end
% series, and the max relative difference between the two, for 8 testcases. fprintf('Max difference: %.1f%%\n', dmax*100);
% TEST_VARMA_SIM QUIET prints only the max difference and only for 4 cases. end
% TEST_VARMA_SIM(N) tests only case N
% function [Ex1,Ex2,Vx1] = expX(A,B,C,X,mu,SS,Sig)
% The routine CC_build is also checked. % Calculate theoretical one and two step expected forecast & one step variance
r=size(C{1},1);
function test_varma_sim(varargin) t = size(X,2)+1;
if nargin > 0 CC = CC_build(A, C, t-1);
if isequal(lower(varargin{1}),'quiet'), tcase = 1:2:7; quiet = true; A=makecell(A);
else tcase = varargin{1}; quiet = false; end B=makecell(B);
else X=X-repmat(mu,1,t-1);
quiet = false; p=length(A);
tcase = 1:1:8; q=length(B);
end Ex1 = zeros(r,1); Ex2 = zeros(r,1);
n = 20000; xp = reshape(X,[],1);
M = 20000; for i=1:p
dmax = 0; Ex1 = Ex1 + A{i}*X(:,t-i);
fprintf('TESTING VARMA_SIM... '); end
if quiet, step=2; else, step=1; end Eeps = reshape(CC'*(SS\xp), r, t-1);
for i=tcase BB = zeros(r,0);
[A, B, Sig, p, q, r, name] = testcase(i); EE = [];
h = max(p,q); for i=1:q
% Ex1 = Ex1 + B{i}*Eeps(:,t-i);
% TEST DIMENSIONS OF RESULTS BB = [B{i} BB];
X1 = varma_sim(A,B,Sig,10); EE = blkdiag(EE, Sig);
X2 = varma_sim(A,B,Sig,10,[],1); end
X3 = varma_sim(A,B,Sig,10,[],5); Vx1 = CC'*(SS\CC);
[X4,eps4] = varma_sim(A,B,Sig,10); Vx1 = Sig + BB*(EE - Vx1(end-r*q+1:end,end-r*q+1:end))*BB';
[X5,eps5] = varma_sim(A,B,Sig,10,[],5); for i=1:p
assertsize(X1,r,10); if i==1, Ex2 = Ex2 + A{i}*Ex1; end
assertsize(X2,r,10); if i> 1, Ex2 = Ex2 + A{i}*X(:,t-i+1); end
assertsize(X4,r,10); assertsize(eps4,r,10); end
if r==1, for i=2:q
assertsize(X3,10,5); Ex2 = Ex2 + B{i}*Eeps(:,t-i+1);
assertsize(X5,10,5); assertsize(eps5,10,5); end
else Ex1 = Ex1 + mu;
assertsize(X3,r,10,5); Ex2 = Ex2 + mu;
assertsize(X5,r,10,5); assertsize(eps5,r,10,5); end
end
% function assertsize(A,n,m,k)
% TEST MEAN if nargin == 3 || k==1
mu = (1:r)'; assert(isequal(size(A), [n,m]));
X = varma_sim(A, B, Sig, 200, mu, 200); else
if r==1, X=reshape(X,1,200,200); end assert(isequal(size(A), [n,m,k]));
mud = mean(mean(X,3),2); end
d = max(abs(mud-mu))/max(mu); end
if ~quiet
fprintf('\nTESTCASE %s', name);
if is_stationary(A, Sig) fprintf(' (model is stationary)'); % TEST_VAR_LL Test the pure autoregressive likelihood function var_ll
else fprintf(' *** WARNING--NONSTATIONARY MODEL--'); end %
fprintf(':\n'); % TEST_VAR_LL compares likelihood calculated with var_ll with that calculated
fprintf(' Difference between theoretical and data mean %.1f%%\n', d*100); % with varma_llm for test cases that "testcase" creates. The gradient is also
end % compared.
dmax = max(dmax,d);
% function test_var_ll
% START FROM SCRATCH; COMPARE DATA COVARIANCE WITH THEORETICAL fprintf('TESTING VAR_LL...');
X = varma_sim(A, B, Sig, n, [])'; % simulate a long single series ncase = testcase('number');
SSd = cov([X(1:end-1,:) X(2:end,:)]); % data covariance, lag 0 and 1 randn('state',3); rand('state',3);
[C,G,W,S] = find_CGWS(A, B, Sig); maxdiff = 0;
SS = S_build(S, A, G, 2); % theoretical covariance % Check non-stationary models:
d = max(abs(SSd(:)-SS(:)))/max(abs(SS(:))); A = [0.5 0.51]; B = [];
dmax = max(dmax,d); X = zeros(1,10); miss = isnan(X);
if ~quiet mu = 0;
disp(' Theoretical covariance at lags 0 and 1:') Sig = -1; [l, ok] = var_ll(X, A, Sig, mu, miss); assert(~ok);
for j=1:r, fprintf(1,' %6.3f',SS(j,:)); disp ' '; end Sig = 1; [l, ok] = var_ll(X, A, Sig, mu, miss); assert(~ok);
disp(' Data covariance at lags 0 and 1:') for j=1:ncase-1
for j=1:r, fprintf(1,' %6.3f',SSd(j,:)); disp ' '; end [A, B, Sig, p, q, r, name] = testcase(j);
fprintf(' Max relative difference: %.1f%%\n', d*100); if q==0
end mu = 0.01*(1:r)';
X = varma_sim(A, B, Sig, h+3, [], M); % simulate multiple short series r = size(Sig,1);
X = reshape(X,r*(h+3),M); n = p+3;
SSd = cov(X'); % data covariance X = varma_sim(A, [], Sig, n, mu);

31
 compare(A, Sig, X, p, r); for i=1:length(cases)
mubar = repmat(mu, n, 1); [A{end+1},B{end+1},Sig{end+1},name{end+1}] = testcase(cases(i));
for i=[0 20] end
X1 = makemissing(X,i); nNamed = 0;
miss = isnan(X1); else
[l1, ok] = varma_llm(X1, A, [], Sig, mu, miss); [A,B,Sig,name] = testcase('all'); % OBTAIN NAMED TESTCASES
[l, ok] = var_ll(X1, A, Sig, mu, miss); assert(ok) nNamed = length(A);
assert(almostequal(l, l1)); for p=0:3 % ADD UNNAMED CASES FOR SEVERAL COMBINATIONS OF p,q,r
[l2, ok, res, xm] = var_ll(X1, A, Sig, mu, miss, 'res_miss'); for q=0:3
assert(isequal(l, l2)); for r=1:3
[l2, ok, res1, xm1] = varma_llm(X1, A, [], Sig, mu, miss, 'res_miss'); [A{end+1},B{end+1},Sig{end+1}] = testcase(p,q,r);
assert(ok); name{end+1}='Unnamed';
assert(almostequal(res1, res)) end
assert(isempty(xm) || almostequal(xm1, xm)); end
[l, ok, ld] = var_ll(X1, A, Sig, mu, miss); end
[l, ok, ld] = varma_llm(X1, A, [], Sig, mu, miss); end
end; for i = 1:length(A)
end % PRINT INFO
end r = size(Sig{i},1);
disp(' OK'); p = size(A{i},2)/r;
end q = size(B{i},2)/r;
fprintf(out,'\n %s %s, p=%d, q=%d, r=%d\n ','Testcase:',name{i},p,q,r);
function compare(A, Sig, X, p, r) if is_stationary(A{i}, Sig{i})
% Compare function values of (a) var_ll with zero mu and all-false miss, (b) fprintf(out, 'stationary model\n')
% var_ll without mu- and miss-parameters and (c) varma_llc. Also check else
% redidual calculation. fprintf(out, 'non-stationary model\n')
miss = false(size(X)); end
ll1 = var_ll(X, A, Sig, zeros(r,1), miss); [C,G,W] = find_CGW(A{i},B{i},Sig{i});
ll2 = var_ll(X, A, Sig); PLU = vyw_factorize(A{i});
ll3 = varma_llc(X, A, [], Sig); S = vyw_solve(A{i},PLU,G);
assert(ll1==ll2 && almostequal(ll3, ll1)); check_solution(out,A{i},G,S);
[ll4, ok, res1] = var_ll(X, A, Sig, zeros(r,1), miss, 'res'); if i<=nNamed && mrhs
[ll5, ok, res2] = var_ll(X, A, Sig, 'res'); % CHECK MULTIPLE RHS FEATURE FOR NAMED TESTCASES:
[ll6, ok, res3] = varma_llm(X, A, [], Sig, zeros(r,1), miss, 'res'); Bc = cell(1,q); Gc = cell(1,q+1);
assert(almostequal(ll4,ll1) && almostequal(ll5,ll2) && almostequal(ll6,ll3)); for c=1:3
assert(almostequal(res1, res2)); for j=1:q, Bc{j} = B{i}{j}-0.1*c; end
assert(almostequal(res2, res3)); Sigc = Sig{i} + 0.1*c;
end [C,Gc,W] = find_CGW(A{i},Bc,Sigc);
for j=1:q+1, G{j}(:,:,c) = Gc{j}; end
end
% TEST_VAR_LL_JAC Test Jacobian feature of var_ll PLU = vyw_factorize(A{i}); % SOLVE
% S = vyw_solve(A,PLU,G); % SOLVE
% TEST_VAR_LL_JAC compares gradient of missing value log-likelihood given by fprintf(out,'MRHS: ')
% var_ll with a numerical gradient for a few testcases, when change of check_solution(out,A{i},G,S);
% variables is used (further testing is carried out by test_var_ll). end
end
function test_var_ll_jac disp('OK')
fprintf('TESTING JACOBIAN FEATURE OF VAR_LL...') end
rand('state',3); randn('state',1);
for r=1:2:3 function check_solution(out,A,G,S)
for p=1:3 % Check residuals of original equations
n = p+4; q = length(G) - 1;
X = rand(r,n); r = size(G{1},1);
m1 = r*r*p; % A p = size(A,2)/r;
m2 = r*(r+1)/2; % Sig N = size(G{1},3);
n1 = ceil(m1/4); % thA maxres = 0;
n2 = 2; % thSig Sc = {};
n3 = r; % mu for c=1:N
S1 = gallery('lehmer',r); for i=1:p+1, Sc{i} = S{i}(:,:,c); end
S2 = hilb(r); % Sig will be th(n1+1)·S1+th(n1+2)·S2 for i=1:q+1, Gc{i} = G{i}(:,:,c); end
Ad = rand(m1, n1); maxres = max(maxres, find_max_residual(A,Sc,Gc));
s1 = vech(S1); end
s2 = vech(S2); fprintf(out,'Max residual = %.1e',maxres)
Sigd = [s1(:) s2(:)]; assert(maxres < 1e-12);
th = (1 + rand(n1 + n2 + n3, 1))*0.2/p/r/r; fprintf(out,'...OK\n ');
J = blkdiag(Ad, Sigd); end
[A, B, Sig, mu] = theta2mat(th, p, 0, r, J);
if r>=3 || p >= 3, I = 2; else I = [-4:0 10]; end function maxres = find_max_residual(A,S,G)
for i = I % Max residual of modif. VYW-system. A=[A1...Ap], S={S0...Sp}, G={G0...Gq}
%fprintf(' r,p,i=%d %d %d\n',r,p,i); q = length(G) - 1;
if i==0 % CHECK VAR_LL WITH NO MISSING VALUES r = size(G{1},1);
d = diff_test(@fun_c, th(1:end-r), X, p, r, J); A = makecell(A);
if d >= 1e-8, d, end p = length(A);
assert(d < 1e-8); S0 = S{1}; S = S(2:end);
end G0 = G{1}; G = G(2:end);
Xm = makemissing(X,i); maxres = 0;
d = diff_test(@fun_m, th, Xm, p, r, J); for i = 0:p
if d >= 1e-8, d, end if i == 0, sum = S0 - G0 ;
assert(d < 1e-8) elseif i <= q, sum = S{i} - G{i};
end else , sum = S{i} ;
end end
end for j = 1:p
disp(' OK') if j < i, sum = sum - A{j}*S{i-j} ; end
end if j == i, sum = sum - A{j}*S0 ; end
if j > i, sum = sum - A{j}*S{j-i}'; end
function [ll,lld] = fun_c(theta, X, p, r, J) % Complete data end
[A, B, Sig, mu] = theta2mat(theta, p, 0, r, J); maxres = max(maxres,norm(sum));
if nargout==1, [ll, ok] = var_ll(X, A, Sig); end
else [ll, ok, lld] = var_ll(X, A, Sig, J); end end
assert(ok);
end
% TEST_VYW_DERIV Test derivative of vyw solutions
function [ll,lld] = fun_m(theta, X, p, r, J) % Missing values %
q = 0; % TEST_VYW_DERIV checks the derivative returned by vyw_deriv_rhs and subsequent
miss = isnan(X); % vyw_solve (for the derivative of S) against numerical differentiation.
[A, B, Sig, mu] = theta2mat(theta, p, q, r, J); % TEST_VYW_DERIV QUIET minimizes output and TEST_VYW_DERIV(k) runs testcase k.
if nargout==1, [ll, ok] = var_ll(X, A, Sig, mu, miss);
else [ll, ok, lld] = var_ll(X, A, Sig, mu, miss, J); end function test_vyw_deriv(varargin)
assert(ok); [varargin, quiet] = getflags(varargin,'quiet');
end out = double(~quiet);
fprintf('TESTING VYW_DERIV...'); fprintf(out,'\n');
tcspec = ~isempty(varargin);
% TEST_VYW Tests solution of vector-Yule-Walker system if tcspec
% cases = varargin{1};
% TEST_VYW QUIET prints out "Passed vyw_tests OK" if all tests are passed. else
% TEST_VYW (without an argument) prints more information. pj = [1 1 2 2 2 2 3 2];
% TEST_VYW(i:j) runs testcases number i through j. qj = [0 1 0 1 2 3 4 0];
% TEST_VYW MRHS tests multiple rhs feature. rj = [1 2 2 2 2 2 2 3];
% TEST_VYW MRHS QUIET, TEST_VYW('MRHS',i) etc. may also be used. cases=1:length(pj);
% end
% See also: testcase for j=cases
if tcspec, [A, B, Sig, p, q, r, name] = testcase(j);
function test_vyw(varargin) else [A, B, Sig, p, q, r, name] = testcase(pj(j),qj(j),rj(j)); end
[varargin,quiet,mrhs] = getflags(varargin,'quiet','mrhs'); theta = mat2theta(A, B, Sig);
if quiet, out=0; else out=1; end fprintf(out, ' Testcase p=%d q=%d r=%d: ', p, q, r);
fprintf('TESTING VYW_FACTORIZE AND VYW_SOLVE...'); fprintf(out,'\n'); dmax = diff_test(@fun, theta, p, q, r);
A={}; B={}; Sig={}; name={}; fprintf(out, 'max rel.diff=%.1e\n', dmax);
if ~isempty(varargin) && isnumeric(varargin{1}) assert(dmax<1e-8);
cases = varargin{1}; end

32
 disp(' OK'); Sig(i,i:r) = theta(m:m2)';
m = m2+1;
function [f,g]=fun(theta,p,q,r) end
N = length(theta); if nargout == 4,
[A, B, Sig] = theta2mat(theta,p,q,r); if m<=length(theta), mu = theta(m:end);
[C, G, W] = find_CGW(A, B, Sig); else mu = []; end
PLU = vyw_factorize(A); end
S = vyw_solve(A, PLU, G); end
f0 = vech(S{1});
f1 = reshape(cell2mat(S(2:end-1)), r^2*(p-1), 1);
f = [f0; f1]; % VEC Change matrix to column vector
if nargout>=2 %
[CCd, GGd, WWd] = find_CGW_deriv(A, B, Sig); % v = VEC(A), where A is an m × n matrix, returns the columns of A one after
RHS = vyw_deriv_rhs(A, GGd, S); % another. It is equivalent to v=A(:).
Sd = vyw_solve(A, PLU, RHS); %
Sd0 = vech(Sd{1}); % v = VEC(A), where A is m × n × N, returns an m·n × N matrix with
Sd1 = reshape(cell2mat(Sd(2:end-1)), r^2*(p-1), N); % vec(A(:,:,j)) in its j-th column.
g = [Sd0; Sd1];
end function v = vec(A)
end [m,n,N] = size(A);
end if N==1
v = A(:);
else
% THETA2MAT Convert parameter vector to parameter matrices v = zeros(m*n, N);
% for j=1:N, v(:,j) = reshape(A(:,:,j), m*n, 1); end
% [A,B,Sig] = THETA2MAT(THETA,p,q,r) converts a vector of the parameters of a end
% VARMA model to the parameter matrices A = [A1...Ap], B = [B1...Bq] and Sig. end
%
% [A,B,Sig,mu] = THETA2MAT(THETA,p,q,r) includes the mean mu among the
% parameters. % VECH Change lower triangle to column vector
% %
% [A,B,Sig] = THETA2MAT(THETA,p,q,r,J) allows for a linear change of variables, % v = VECH(A), where A is an n×n matrix returns an n·(n+1)/2 dimensional
% so that [vec(A); vec(B); vech(Sig)] = J·THETA. If J has too few rows, an % column vector with the columns of the lower triangle of A placed one after
% identity matrix of the necessary size is appended to its bottom right: J := % another.
% blkdiag(J, I), enabling, for instance, Sig to be excluded from the variable %
% change. % V = VECH(A) where A is n×n×N returns an n·(n+1)/2 × N dimensional matrix
% % with VECH(A(:,:,j)) in its j-th column.
% [A,B,Sig,mu] = THETA2MAT(THETA,p,q,r,J) sets [A(:); B(:); Sig(:)] to
% J·THETA(1:k) and mu to THETA(k+1: end) where k is the column count of J. function v = vech(A)
if isempty(A), v=[];
function [A,B,Sig,mu] = theta2mat(theta,p,q,r,J) else
theta = theta(:); [n,m,N] = size(A);
if nargin>=5 assert(n==m);
m = size(J,2); v = zeros(n*(n+1)/2, N);
theta = [J*theta(1:m); theta(m+1:end)]; m = 1;
end for i=1:n
i1 = p*r^2; i2 = i1 + q*r^2; m1 = m + n-i;
A = reshape(theta(1:i1) , r, r*p); v(m:m1, :) = A(i:n, i, :);
B = reshape(theta(i1+1:i2), r, r*q); m = m1 + 1;
m = i2+1; end
for i=1:r end
m2 = m+r-i; end
Sig(i:r,i) = theta(m:m2);

REFERENCES
George B. and Williams, L. 2004. A structured experiment of test-driven development. Information And Software
Technology 46, 5, 337–342.

Jonasson, K. and Ferrando, S. E. 2006, Efficient likelihood evaluation for VARMA processes with missing values.
Report VHI-01-2006, Engineering Research Institute, University of Iceland.

Mauricio, J. A. 1997. Algorithm AS 311: The exact likelihood function of a vector autoregressive moving average
model. Appl. Statist. 46, 1, 157–171.

Schneider, T. and Neumaier, A. 2001. Algorithm 808: ARfit - A Matlab package for the estimation of parameters
and eigenmodes of multivariate autoregressive models. ACM Trans. Math. Softw. 27, 1, 58–65.

Shea, B. L. 1989. Algorithm AS 242: The exact likelihood of a vector autoregressive moving average model.
Appl. Statist. 38, 1, 161–204.

Terceiro, J. 1990. Estimation of Dynamic Econometric Models with Errors in Variables. Springer-Verlag, Berlin.

33