International Journal of Automation and Computing

04(1), January 2007, 88-94

DOI: 10.1007/s11633-007-0088-2

A Fast LDL-factorization Approach for Large Sparse

Positive Definite System and Its Application to
One-to-one Marketing Optimization Computation
Min Wu1
1

Bei He1

Jin-Hua She2

School of Information Science and Engineering, Central South University, Changsha 410083, PRC
2

School of Bionics, Tokyo University of Technology, Tokyo 192-0982, Japan

Abstract:
LDL-factorization is an efficient way of solving Ax = b for a large symmetric positive definite sparse matrix A. This
paper presents a new method that further improves the efficiency of LDL-factorization. It is based on the theory of elimination trees
for the factorization factor. It breaks the computations involved in LDL-factorization down into two stages: 1) the pattern of nonzero
entries of the factor is predicted, and 2) the numerical values of the nonzero entries of the factor are computed. The factor is stored
using the form of an elimination tree so as to reduce memory usage and avoid unnecessary numerical operations. The calculation
results for some typical numerical examples demonstrate that this method provides a significantly higher calculation efficiency for the
one-to-one marketing optimization algorithm.
Keywords:

Sparse matrix factorization, elimination tree, structure prediction, one-to-one marketing optimization.

Introduction

One-to-one marketing optimization is a crucial technology for analytical customer relations management (CRM).
Its main function is to assist an enterprise in one-to-one
marketing activities by providing suitable decisions. It is
also used as an approach to understand and influence customer behavior through meaningful communication in order
to improve customer retention, customer loyalty, and profitability. Since optimization technology brings great benefits to an enterprise, increasing numbers of enterprises have
been researching and employing it[13] .
A one-to-one marketing optimization problem involves
finding a binary vector (the best marketing plan) that maximizes a linear objective function (profit), which is subject
to constraints, i.e.
max
s.t.

T
,
= [1 n ]T ,
i {0, 1}, i = 1, , n

(1)

where Rn is the design vector, Rn is the vector of

coefficients of the objective function, Rmn is a constraint matrix, and Rm is the right-hand-side vector of
the constraint. Clearly, the key to the computations is how
to solve the optimization problem (1) for quickly and accurately because is usually a huge sparse matrix. Since a
large enterprise has a great variety of products, many communication channels, and millions of customers, the number of variables and the number of constraints may reach
the order of 108 and 106 , respectively, in a one-to-one marManuscript received September 19, 2005; revised July 31, 2006.
This work was supported in part by the National Natural Science
Foundation of PRC (No. 60425310) and the Teaching and Research
Award Program for Outstanding Young Teachers in Higher Education Institutions of MOE, PRC.
*Corresponding author. E-mail address: min@mail.csu.edu.cn

keting optimization model. An optimization problem for

a customer base of order of 105 or higher is usually called
a large-scale optimization problem. Wu, He and Chen examined the relationships between the scale (i.e., the size of
the customer base) on the one hand, and the computation
time and memory usage needed to solve the problem on the
other hand[4] . Their results show that conventional linear
programming (LP) methods (e.g., Simplex and Depth-First
Search) cannot solve a large-scale one-to-one marketing optimization problem because both the computation time and
the memory usage increase exponentially with the number
of customers.
Over the past few years, some interior-point methods
of LP for large-scale problems have been developed based
on an iterative algorithm(e.g., the predictor-corrector and
Krylov-subspace methods[5] ) . All of the methods developed
so far involve solving the following linear matrix equation:
Ax = b

(2)

where A is a constant matrix and b is a constant vector.

Since A is huge for a large-scale one-to-one marketing optimization problem, the problem is how to solve (2) quickly.
In fact, how to efficiently handle a large sparse matrix is
a common problem in many engineering fields, such as fluid
dynamics, structural engineering, electrical circuits, heat
transport, semiconductor modeling, molecular dynamics,
geophysical reservoirs, etc. Along with the rapid progress in
computer technology, techniques of information processing
have also been advancing quickly. Not only have existing
algorithms been improved, but new methods of handling
large sparse matrices have also been devised.
This paper considers the problem of using LDLfactorization to solve the linear matrix equation (2) for a
large sparse symmetric positive definite matrix, A. An elimination tree and a two-stage strategy are employed in conjunction with LDL-factorization to derive a new computa-

M. Wu et al./A Fast LDL-factorization Approach for Large Sparse Positive Definite System and Its Application to

tion method. Based on the structure of the elimination tree

for a given matrix, this method performs LDL-factorization
in two stages: structural prediction (symbolic stage) and
numerical computation (numerical stage). It provides an efficient storage scheme for storing the elimination tree and a
sparse matrix. It thus avoids redundant memory usage and
requires less time for numerical calculations. As a result,
this method substantially improves the speed and reduces
memory usage in the computation of the LDL factor.
Throughout this paper, for a matrix A = (aij ) Rnn ,
GA = (V, E) is an undirected graph of A, where the nodes of
V {1, 2, , n} correspond to the rows and columns of A,
and the edges of E {(j, i)|j > i and aij 6= 0} correspond
to the nonzero entries of A. Ai:j,k:l means a submatrix of
A obtained by extracting rows i through j and columns k
through l. Ai:i,k:l is abbreviated as Ai,k:l , and likewise for
a column submatrix of A. In particular, A(k) is a leading
principal submatrix of A of order k, i.e. A(k) = A1:k,1:k .
In is an n n identity matrix, and 1n is an n-dimensional
column vector in which all the entries are 1.
The rest of this paper is organized as follows. Section 2
introduces some related works about factorization methods
for solving the linear matrix equation (2). Section 3 analyzes the LDL-factorization algorithm and presents new algorithms for LDL-factorization based on the theory of elimination trees. Section 4 describes the storage scheme for the
LDL factor and the elimination tree. Section 5 presents the
calculation results for typical numerical examples. Section
6 explains the application of this method to computations
for one-to-one marketing optimization. Finally, Section 7
contains some concluding remarks.

Related works

At present, several robust algorithms and software packages have been developed to solve the linear matrix equation
(2). Specifically, the algorithms presented in [6] are mainly
used in Cholesky factorization; they factor A into CC T ,
where C is a lower triangular matrix called the Cholesky
factor of A. To avoid the storage of, and operations on,
the zero entries of C, Rose devised a new computational
method for Cholesky factorization[7] , in which the computation is broken down into two stages: First, in the symbolic
stage, the pattern of nonzero entries of C is predicted; and
then, in the numerical stage, this structure is used to carry
out numerical calculations. The advantage of this method
is that the time needed for the first stage is proportional to
the size of C. This method was improved by George and
Reid, who developed SPARSPAK[8] , which uses asymptotically fastest algorithms to compute the Cholesky factorization in the symbolic stage. Later, Liu presented a compact
row-oriented storage scheme for Cholesky factorization[9] .
This structural representation can be viewed as the minimal structure of a given matrix that preserves the symbolic
Cholesky factor.
On the other hand, Gilbert and Peierls devised a new
solution for general Gaussian elimination, A = GU , where
G is a lower triangular matrix and U is an upper triangular
matrix. It also breaks the computation of the factorization
factor into a symbolic and a numerical stages[10] . In the
calculation procedure, the pattern of nonzero entries in each

89

column of the factor is first predicted. Then, numerical

calculations are carried out based on that structure. This
method restricts the numerical factorization to necessary
arithmetic, and requires fewer operations.
LDL-factorization involves factoring symmetric positive
definite sparse matrix A into LDLT , where D is a diagonal
matrix and L is a unit lower triangular matrix called the
LDL factor. It is also viewed as a special case for Cholesky
factorization. Timothy A. Davis presented an LDL software
package to illustrate much of the basic theory of sparse matrix algorithms in as concise a code as possible, including
an elegant method of sparse symmetric factorization that
computes the factorization row-by-row but stores it columnby-column[11] .

Algorithms

This section considers how to use LDL-factorization to

solve (2), where A Rnn is a large sparse symmetric
positive definite matrix. The elimination tree is defined
specifically for use in LDL-factorization, and some new algorithms for LDL-factorization are presented. Special care
is taken in the construction of algorithms with the goal of
reducing computation time and memory usage.
Consider the following LDL-factorization:
A = LDLT .

(3)

Assume that matrix A has already been ordered by some

fill-reducing symmetric permutation (e.g., minimum-degree
ordering [12] or nested-dissection ordering[13] ). A wellknown algorithm that performs LDL-factorization[14] is
Algorithm 1. LDL-factorization of A Rnn
L(1) = 1, D(1) = a11 .
for k = 2 to n
Step 1. solve L(k 1)x = A1:k1,k for x.
Step 2. Lk,1:k1 = (D1 (k 1)x)T .
Step 3. lkk = 1.
Step 4. dkk = akk Lk,1:k1 x.
end for
In this algorithm, lii , dii , and aii (i = 1, , n) are the
diagonal entries of L, D, and A, respectively, l11 = L(1) =
1, and d11 = D(1) = a11 . The k-th row of L and the (k, k)
entry, dkk , of D (k = 2, , n) are found in the k-th step
for a lower triangular matrix belonging to R(k1)(k1) .
To simplify the explanation in the rest of the paper, the
= b, where b :=
equation in Step 1 is abbreviated as Lx
when there is
A1:k1,k and L(k 1) is abbreviated as L
no possibility of confusion. and represent the sets of
indices of the nonzero entries of x and b, respectively. In
fact, is the set of nodes reachable via paths in the graph
GL from all the nodes in the set . The closure of is the
smallest set containing [15] , i.e.
\
closure() =
{ : and is closed} .
(4)
With this preparation, we are ready to deal with the
problem of simplifying the calculations. In Algorithm 1,
the computational complexity is mainly centered in Step 1,
= b is solved for x. Note that Algorithm 1
in which Lx
and performs
requires the storage of the zero entries of L

90

International Journal of Automation and Computing 04(1), January 2007

calculations on them, which is unnecessary. The following

algorithm was created to avoid that.
= b for the lower triangular
Algorithm 2. Solve Lx
which has unit diagonal entries
matrix L,
Step 1:. =closure().
Step 2:. x = b.
Step 3:. for i
i+1:k1,i xi ;
xi+1:k1 = xi+1:k1 L
end for
In this algorithm, the first step is the symbolic stage, in
which the nonzero structure of x is predicted, and the other
steps constitute the numerical stage. In other words, it first
finds the nonzero pattern, , and then computes the value
of x using that structure. It is known that, given the struc and b, there exist nonzero values of =closure()
tures L
[15]
. Note that the time needed to compute is proportional
to the number of floating-point operations performed, and
the same is true for x.
As can be seen in Algorithm 2, GL must be predicted
before can be computed. Note that the LDL-factorization
is given by
T
A=L D L D
(5)
and the Cholesky factorization is
A = CC T .

(6)

So, we can define

C := L D = Ldiag (1 , 2 , , n )

(7)

where i > 0 (i = 1, , n) are the diagonal entries of D.

From (7), L and C have the same nonzero pattern, i.e.,
GL = GC . The relationship between LDL-factorization and
Cholesky factorization is described in the following lemma.
Lemma 1. For a given sparse symmetric positive definite matrix A, the LDL factor L and the Cholesky factor
C have the same nonzero pattern.
As described in [16], for each column j (< n) of C, removing all the nonzero entries in the column, except for the
first one below the diagonal, yields the matrix Lt . Define
Ft = Lt + LT
t . Then, the graph GFt has a tree structure
that depends entirely on the structure of C and its initial
ordering. This tree structure is denoted by the symbol TC
and is called the elimination tree of C. TC has the same
node set as GC . The structure of TC can be represented by
the Parent vector of Ft , which is defined as follows: Let cij
(i, j = 1, , n) be the entries of C. Then, for each column
j (< n) of C,
P arent[j] = min{i > j|cij 6= 0}

(8)

is the parent node of node j in the elimination tree TC .

Node j does not have a parent if the entries below the diagonal in column j are all zero. In this case, node j is a root
of the elimination tree. And we define P arent[j] = 0. For
completeness, we also define P arent[n] = 0.
According to Lemma 1, the elimination trees TC (strictly
speaking, they constitute an elimination forest) that represent the nonzero pattern of L for matrix A are the same as

those for the Cholesky factor of A; and the elimination tree

vectors are (j < n)
P arent[j] = min{i > j|lij 6= 0}
P arent[n] = 0

(9)

where lij is the (i, j) entry of L (i, j = 1, , n). Instead

of using the traversal of the general graph GL through the
nodes reachable from the nodes of , this paper employs
the traversal of a simplified elimination tree representing
the nonzero pattern, , which is the set of all paths from
each node j (j < n) for which bj 6= 0 to the root of the tree.
The process of constructing the elimination tree is recursive, as can be seen in Algorithm 1. The elimination trees
of L(k) are determined based on the elimination trees of
L(k 1) (2 6 k < n). If the parent node of node j in the
elimination tree, TL(k) , is i, then i = min{i > j|lij 6= 0} and
6 n), and
j 6 k. Since GL(k) is a subgraph of GL(k)
(k < k
the nodes that do not belong to GL(k) are all larger than k,
the parent node of node j in TL(k)
remains i. Summarizing
the above result yields the following lemma.
Lemma 2. If node j has a parent node i in TL(k) , then
6 n.
node i remains the parent node of j in TL(k)
for k < k
Based on Lemma 2, we obtain the following algorithm
for computing the structure of the elimination trees of the
LDL-factorization, where P arentk means the parent vector
for the elimination tree of L(k).
Algorithm 3. Calculation of P arentk
if k = 0
P arentk = 0;
else
P arentk = P arentk1 ;
//Call construction algorithm P arentk1 .
for j {lkj 6= 0, j 6 k}
if P arentk [j] = 0
P arentk [j] = k;
end if
end for
end if
This algorithm is employed in Step 1 (symbolic stage) of
Algorithm 2.
Consider Step 3 of Algorithm 2:
i+1:k1,i xi .
xi+1:k1 = xi+1:k1 L

(10)

= b. If
This step computes the numerical solution of Lx
there is a path from j to i in the elimination tree TL , then
the solution of xj+1 requires the computation of xi . So,
sorting is necessary. Sorting it in a natural order is not
the best way because it has the drawback of a very large
computational load. In order to avoid unnecessary operations, a new topological sorting algorithm for was developed in which the result is assumed to be stored in the
stack xs .
Algorithm 4. Sorting x
Step 1.
for i
j = 0;
while i is not marked
Step 1.1. xs [j] = i; j = j + 1;

M. Wu et al./A Fast LDL-factorization Approach for Large Sparse Positive Definite System and Its Application to

Step 1.2. mark i;

Step 1.3. i = P arent[i];
end while
end for
Step 2. Adjust the storage order of xs .
//Let i be at the top of the stack and
//the root be at the bottom.
Algorithm 4 first selects a nonzero entry, bi , and stores
the path from i to the root of the elimination tree. Then,
it marks the nodes along the path and stores the path in
xs with i at the top and the root at the bottom. It repeats
this procedure for all the nonzero entries of bi and stops
when it reaches a marked node. Note that the result may
be empty if i is already in xs . So, xs contains the topological ordering, , of the nonzero pattern of x, and thus
= b. In fact, this
can be used in Algorithm 2 to solve Lx
algorithm is employed before Step 3 in Algorithm 2 to obtain , which is used in the conditional part of Step 3. It
is worth pointing out that the time needed to compute
using Algorithm 4 (traversal of an elimination tree) is much
shorter than that for the transversal of a general graph, and
is proportional to the size of rather than to the number
of floating-point operations in the computation of x. So,
Algorithm 4 markedly reduces the computation time.

91

The dimension of A is n = 8 and the number of nonzero

entries in the upper triangular matrix is m = 17.
8
T
9
>
> Ap = [1, 2, 3, 4, 6, 8, 10, 14, 18] R ,
>
>
>
< Ai = [1, 2, 2, 3, 4, 3, 5, 3,
(12)
6, 1, 3, 6, 7, 2, 3, 5, 8]T R17 ,
>
>
> Ax = [7, 1, 2, 6, 2, 6, 3, 9, 6, 3, 2,
>
>
:
1, 4, 1, 3, 6, 1]T R17 .
As explained in Section 2, the elimination trees are stored
in the array P arent. Fig. 1 shows the elimination tree of L,
the LDL factor of A, and its P arent.

Format of storage

Since Algorithm 2 traverses L in column order, L is stored

in that order. The Yale sparse matrix[17] is a storage format
for sparse matrices that stores the nonzero entries in column order. More specifically, for a sparse matrix, A, with
n columns and m nonzero entries, the Yale sparse matrix
contains three arrays:
Ax : Holds the values of the nonzero entries in column order.
Ai : Holds the row indices in column order.
Ap : Holds the index of the first entry of each column in Ai
and Ax , and the last entry is set to the number m + 1.
The length of Ap is n + 1, and the lengths of Ai and Ap
are both m. The numerical values of the entries of column
j are given by Ax [Ap [j], , Ap [j + 1] 1]; and the corresponding row indices are in Ai [Ap [j], , Ap [j + 1] 1].
Clearly, Ap [1] = 1; and the number of entries in A is
m = Ap [n + 1] 1.
In order to reduce the amount of memory needed to handle A in (2), we store only the entries of the upper triangular matrix of A in the Yale sparse-matrix format. We call
this format a compressed Yale sparse-matrix format. It is
illustrated in the following example. Consider
3
2
7 0 0 0 0 0 3 0
6 0 1 2 0 0 0 0 1 7
7
6
7
6
6 0 2 6 0 6 9 2 3 7
7
6
6 0 0 0 2 0 0 0 0 7
6
7.
(11)
A=6
7
6 0 0 6 0 3 0 0 6 7
6 0 0 9 0 0 6 1 0 7
7
6
7
6
4 3 0 2 0 0 1 4 0 5
0 1 3 0 6 0 0 1

Fig. 1

The elimination tree TL and its Parent.

Implementation and tests

Note that L is a lower triangular matrix with diagonal

entries equal to 1. In Algorithm 1, the format for storing L
can be further compressed by storing only the entries of the
lower triangular part of L, except for the diagonal entries.
Let the LDL-factorization be
+ I)D(L
+ I)T
A = (L

(13)

is a lower triangular matrix with diagonal entries

where L
equal to 0. Then, Step 3 in Algorithm 1 can be omitted.
The elimination tree, TL , in Algorithm 3 can be constructed directly from GA . Specifically, TL(k) can be computed by traversing only the nonzero pattern of A in the
order k = 1, 2, , n. So, the algorithm can be further
accelerated by using a form of path compression to avoid
unnecessary traversals.
This is accomplished by using a temporary vector,
Ancestor[n], based on P arent[n]. Ancestor provides shortcuts to determine the roots of the subtrees containing the
node i. That is, if node i is visited for the first time, then
we set Ancestor[i] = k; and if node i has previously been
visited, in which case Ancestor[i] = k, then the traversal is
stopped.
To evaluate the performance of the LDL-factorization
method described above, we implemented it using C++
in a VC++ 6.0 development environment. The tests employed examples in the Harwell-Boeing collection of sparse

92

International Journal of Automation and Computing 04(1), January 2007

matrices, which can be downloaded from the following Web

site:
http://math.nist.gov/MatrixMarket/data/HarwellBoeing/
The examples selected are listed in Table 1. The keys provided by Harwell-Boeing are used to identify the matrices.
The problems were carefully selected and are representative of the entire collection. A desktop computer (2.4-GHz
Pentium 4, Windows XP, 480-MB RAM) was used to carry
out the tests. For each example, the fill-reducing minimumdegree algorithm was used to order the equations (version:
COLACM2.3[18] ). Table 2 shows the test results. Clearly,
the elimination-tree-based LDL-factorization method presented in this paper efficiently solved problems with large
sparse matrices containing thousands of nonzero entries.

vector j for this kind of customer to be (i = 1, , m, j =

1, , q)
j = [1j mj ]T Rm .
(19)

and the second constraint can be

8
6 1pn ,
>
>
2 T
>
>
>
1l
0

>
>
<
6
..
6 0
.
1T
6
l
>

=
6
>
.
>
.
.
6
>
..
..
>
4 ..
>
>
:
0
0

Application

This section discusses the application of the method described above to the solution of a one-to-one marketing optimization problem.
Assume a company has p products and wants to sell them
to n customers through l channels (such as e-mail, telemarketing, direct mail, etc.). It makes m = p l offers.
We introduce the variable ij (i = 1, , m, j =
1, , n), which can take the following values:
(
1, if offer i is made to customer j,
ij =
(14)
0, otherwise.
And we define
(
i = [1i mi ]T Rm , i = 1, , n
= [1T nT ]T Rmn .

(15)

Note that i (i = 1, , n) is arranged as

[1i li (l+1)i (2l)i (2l+1)i mi ]T ; the first l
entries are for the first product, the next l are for the second product, and so on.
Let
i (i = 1, , m) be the profit made from offer i
when the product is sold to a customer. Define
(
i = [
i
i ]T Rm , i = 1, , n
(16)
T T
mn
= [T
.
1 n ] R
Now, we choose the objective function for the one-to-one
marketing optimization problem to be
J() = T .

ij 6 1, j = 1, , pn.

(18)

i=1

Let the number of customers who ordered the product

for offer i before receiving the offer be q. We define the

written as
3
0
.. 7
. 7
7
7 R(pn)(mn) .
7
0 5
1T
l

(21)
Summarizing the above discussion, we formulate the oneto-one marketing optimization problem as follows[19] :
max
s.t.

J() = T
= 1q ,
6 1pn ,
ij {0, 1}, i = 1, , m, j = 1, , n.

(22)

As pointed out in [20], we can relax the constraint ij

{0, 1} to 0 6 ij 6 1 (i = 1, , m, j = 1, , n) and
convert this 0-1 programming problem to the following LP
problem:
max
s.t.

J() = T
= 1q ,
6 1pn ,
0 6 ij 6 1, i = 1, , m, j = 1, , n.

(23)

Introducing the slack variables Rpn and s

R
, and defining = [ T T ]T R(m+p)n and
T T T
c = [ 0 ] R(m+p)n allows us to write the LP problem
(23) in the following standard form:
(m+p)n

cT
= 1q+pn
+ s = 1(m+p)n
> 0, s > 0

max
s.t.

(17)

In the marketing process, the following constraints must

be considered:
1) Order Form Constraint: If customer j (j = 1, , n)
has already ordered the product for offer i (i = 1, , m)
before receiving the offer, then, ij must be set to be 1.
2) Single Offer Constraint: Since only one channel is
needed to sell a product to a customer,
l
X

If customer j has already ordered the product for offer i,

set ij = 1; otherwise, set ij = 0. Now, the first constraint
can be written as
8
= 1q ,
>
>
2 T
3
>
>
>
1
0

0
>
>
<
6
.. 7
..
6 0
.
(20)
2T
. 7
6
7
>
= 6
7 Rq(mn)
>
>
..
..
6 ..
7
>
>
4 .
.
.
0 5
>
>
:
0
0
qT

where

"
=

(24)

0
I

R(q+pn)[(m+p)n] .

(25)

Instead of solving the primal LP problem (24), we solve

the following dual problem:
min
s.t.

yw
Ty + z w = c
z > 0, w > 0

(26)

M. Wu et al./A Fast LDL-factorization Approach for Large Sparse Positive Definite System and Its Application to
Table 1
Key
BCSPWR03
BCSPWR09
BCSPWR10
BCSSTK13
BCSSTK28
BCSSTK30

Number
of eqs
118
1 723
5 300
2 003
4 410
28 924

Test examples selected from Harwell-Boeing

Number of nonzero
entries in L
263
4 496
22 711
287 867
348 304
3 724 867

Table 2
Key
BCSPWR03
BCSPWR09
BCSPWR10
BCSSTK13
BCSSTK28
BCSSTK30

Table 3

93

Description
IEEE standard 118 bus test case power network
Western US power network 1723 bus
Western US power network 5300 bus
Fluid flow generalized eigenvalues
Solid element model, linear statics
Statics module of an off-shore generator platform

Test Result of the LDL-factorization

Number of nonzero
entries in L
263
4 496
22 711
287 867
348 304
3 724 867

Time for
multiplication
1 173
27 382
253 573
69 220 200
382 242 000
865 532 000

Time for symbolic

calculation (s)
0.001 800
0.000 241
0.001 088
0.003 119
0.004 632
0.053 670

Total calculation
time (s)
0.004 340
0.001 837
0.008 636
0.580 561
0.247 724
7.340 698

Comparison of calculation results obtained by Cholesky-Infinity and LDL-factorization.

Scale of one-to-one
marketing
optimization problem
Rows
Columns
Nonzero
entries
113
1 320
5 280
203
2 400
9 600
1 003
12 000
48 000
2 003
24 000
96 000
10 003
120 000
480 000
60 003
720 000
2 880 000
100 003
1200 000
4 800 000

Cholesky-Infinity
Memory
(MB)
0.9
1.3
7.0
15.1
93.3
455.4
672.3

Time
(s)
0.06
0.46
2.81
6.95
96.32
3 522.73
59 348.53

where y is a dual variable, and z and w are slack variables.

The solutions of both the primal and dual LP problems
satisfy the following Karush-Kuhn-Tucker conditions:
0
1
1q+pn
B + s 1
C
(m+p)n C
B
B T
C
F (u) = B y + z w c C = 0
(27)
B
C

@
A
z
sw
h
iT
T
z T sT wT > 0
(28)
iT
h
where u = T z T sT wT y T . He and Wu used the
predictor-corrector method to solve this linear quadratic
equation (27) with a non-negativity[19] . The algorithm is
iterative; and u can be made to approach the best solution
in the k-th iteration by setting
(
uk+1 = uk + k uk
(29)
uk = p uk + c uk
where p uk and c uk are the predictor and corrector directions of uk , respectively. In the k-th iteration, the main
computation involved in the calculation of (29) is the solving of yk , which is a sub-vector of uk , for
( Dk T )yk = (rbk + Dk rck )

(30)

where Dk is a diagonal constant matrix, and rbk and rck

are constant vectors. Since is a huge sparse constraint
matrix, D T is a huge square symmetric sparse matrix.

LDL

Objective
value
32 366.90
58 703.92
297 475.66
598 514.60
2 997 575.03
18 023 178.50
30 104 984.02

Memory
(MB)
0.9
1.2
6.5
14.2
90.5
432.3
520.0

Time
(s)
0.05
0.35
2.14
4.32
32.82
158.75
400.65

Objective
value
32 366.90
58 703.92
297 475.66
598 514.60
2 997 575.03
18 023 178.50
30 104 984.02

The Cholesky-Infinity factorization proposed by

Zhang[21] (which is contained in the one-to-one marketing
optimization software package LPMDP[19] ) and the method
described in this paper were used to solve yk in (30).
The results in Table 3 show that elimination-tree-base
LDL-factorization yields the same values for the objective
function as Cholesky-Infinity factorization does. In other
words, our method produces the same optimal results.
However, compared with the Cholesky-Infinity factorization method, our method requires significantly less
computation time and uses much less memory, especially
when the scale is large. For example, for the last case
in Table 3, the computation time for our method is only
0.68%, and the memory usage is only about 77% of the
Cholesky-Infinity factorization.

Conclusions

This paper has presented a method of performing LDLfactorization on a sparse symmetric positive definite matrix and described its application to one-to-one marketing
optimization. The computations for LDL-factorization are
broken down into two stages: structural prediction and numerical calculation. This method first uses an elimination
tree to construct the nonzero pattern of the factor, and then
uses this structure to compute numerical values. It uses
markedly less memory and involves much fewer numerical
calculations when solving a problem with a large symmetric
sparse matrix.
Tests using numerical examples of typical sparse matri-

94

International Journal of Automation and Computing 04(1), January 2007

ces and one-to-one marketing optimization problems have

revealed that this method is an efficient way of solving the
LDL-factorization problem for a large sparse matrix, and
thus provides significantly better performance in the solution of one-to-one marketing optimization problems.

References
[1] A. Berson, S. Smith, K. Thearling. Building Data Mining Applications for CRM, McGraw-Hill Press, New York, 1999.
[2] Lange, Kirsten. The Benefits of Crm. Pulp and Paper International, vol. 45, no. 11, pp. 4042, 2003.
[3] M. Tim. Crm Goes Small to Get Big Again. Telephony, vol. 246,
no. 19, pp. 3235, 2005.
[4] M. Wu, B. He, S. Chen. A Benchmark Method for One-to-one
Marketing Optimization Algorithms. Computer Engineering and
Applications, vol. 40, no. 19, pp. 4547, 2004. (in Chinese)
[5] E. John, Mitchell, K. Farwell, D. Ramsden. Interior Point
Methods for Large-scale Linear Programming, Technical Report, Mathematical Sciences Rensselaer Polytechnic Institute,
Troy, NY, 2004.
[6] G. W. Stewart. Building an Old-fashioned Sparse Solver, Technical Report, University of Maryland, USA, 2003.
[7] D. J. Rose. A Graph-theoretic Study of the Numerical Solution of Sparse Positive Definite System of Linear Equation.
Graph Theory and Computing, Academic Press, New York, pp.
183217, 1972.
[8] A. George, J. K. Reid. Computer Solution of Large Sparse Positive
Definite System, Prentice-Hall, Englewood Cliffs, NJ, 1981.
[9] J. W. H. Liu. A Generalized Envelope Method for Sparse Factorization by Rows. ACM Transations on Mathematical Software,
vol. 17, no. 1, 1991.
[10] J. R. Gilbert, T. Peierls. Sparse Partial Pivoting in Time Proportional to Arithmetic Operations. SIAM Journal on Scientific
and Statistical Computing, vol. 9, no. 5, pp. 862874, 1988.
[11] T. A. Davis. User Guide for LDL, a Concise Sparse Cholesky
Package, Technical Report, Department of Computer and
Information Science and Engineering, University of Florida,
Gainesville, FL, USA, 2005.
[12] P. R. Amestoy, T. Davis, I. Du. An Approximate Minimum
Degree Ordering Algorithm, SIAM Journal on Matrix Analysis
and Applications, vol. 17, no. 3, pp. 886905, 1996.
[13] I. Brainman, S. Toledo. Nested-dissection Orderings for Sparse
Lu with Partial Pivoting. SIAM Journal on Matrix Analysis and
Applications, vol. 23, no. 4, pp. 9981012, 2001.
[14] Q. Li, N. Wang, D. Yi, Eds., Numerical Analysis, 4th ed.,
Qinghua University Press, Beijing, China, 2004. (in Chinese)
[15] J. R. Gilbert. Predicting Structure in Sparse Matrix Computations. SIAM Journal on Matrix Analysis and Applications, vol. 15,
no. 1, pp. 6279, 1994.
[16] J. W. H. Liu. The Role of Elimination Trees in Sparse Factorization. SIAM Journal on Matrix Analysis and Applications, vol. 11,
no. 1, pp. 134172, 1990.
[17] S. C. Eisenstat, M. C. Gursky, M. H. Schultz, A. H. Sherman.
Yale Sparse Matrix Package I: the Symmetric Codes. International Journal for Numerical Methods in Engineering, vol. 18, no.
4, pp. 11451151, 1982.
[18] P. R. Amestoy, T. A. Davis, I. S. Du. Algorithm 837: Amd,
an Approximate Minimum Degree Ordering Algorithm. ACM
Transactions on Mathematical Software, vol. 30, no. 3, pp.
381388, 2004.

[19] B. He, M. Wu, H. Mitsuma, Y. C. Zhou. An One to One Marketing Optimization Algorithm Based on Minimum Degree Permutation. In Proceedings of 23rd Chinese Control Conference, X. W.
Daizhang Cheng, Ed., East China University of Science and
Techology Press, Shanghai, pp. 451453, 2004. (in Chinese)
[20] K. D. Lu. Single Objective, Multiple Objective and Integer Programming, Tsinghua University Press, Beijing, 1999. (in Chinese)
[21] Y. Zhang. Solving Large-scale Linear Programs by Interiorpoint Methods Under the Matlab Environment. Optimization
Methods and Software, vol. 10, no. 1, pp. 131, 1998.

Min Wu was born in 1963. He received

his B.Sc. and M.Sc. degrees in engineering
from the Central South University, Changsha, China in 1983 and 1986, respectively. In
July, 1986, he joined the staff of the university,
where he is currently a professor of automatic
control engineering. He was a visiting scholar
at the Department of Electrical Engineering,
Tohoku University, Sendai, Japan, from 1989
to 1990, and a visiting research scholar at the
Department of Control and Systems Engineering, Tokyo Institute of Technology, Tokyo, Japan, from 1996 to 1999.
He received his Ph.D. degree in engineering from Tokyo Institute of
Technology, Tokyo, Japan in 1999.
His research interests include robust control and its application,
process control, and intelligent control.
Dr. Wu received the best paper award at IFAC 1999 (jointly with
M. Nakano and J.-H. She). He is a member of the Nonferrous Metals
Society of China and the Chinese Automation Association.
Bei He was born in 1979. She received
her B.Sc. degree in engineering from Xiangtan Polytechnic University, Xiangtan, China
in 2001, and her M.Sc. degree in engineering from Central South University, Changsha, China in 2003. She is currently pursuing
her Ph.D. degree in control engineering at the
Central South University.
Her research interests include system engineering, optimization methods, and marketing optimization.
Jin-Hua She received his B.Sc. degree in
engineering from Central South University,
Changsha, China, in 1983, and his M.Sc. and
Ph.D. degrees in engineering from the Tokyo
Institute of Technology, Tokyo, Japan, in 1990
and 1993, respectively. In 1993, he joined the
Department of Mechatronics, School of Engineering, Tokyo University of Technology; and
in April, 2004 he transferred to the Universitys School of Bionics, where he is currently
an associate professor.
His current research interests include the application of control
theory, repetitive control, expert control, Internet-based engineering
education, and robotics.
Dr. She received the control engineering practice paper prize at
IFAC 1999 (jointly with M. Wu and M. Nakano). He is a member of
IEEE, the Society of Instrument and Control Engineers (SICE), and
the Institute of Electrical Engineers of Japan (IEEJ).