Professional Documents
Culture Documents
Products
Roger A. Horn and Roy Mathias
The Johns Hopkins University, Baltimore, Maryland 21218, USA
Yoshihiro Nakamura
Research Institute for Applied Electricity, Hokkaido University, Sapporo 060, Japan
May 4, 2001
Abstract
We give several criteria that are equivalent to the basic singular value majorization inequality
(1.1) that is common to both the usual and Hadamard products. We then use these criteria
to give a unied proof of the basic majorization inequality for both products. Finally, we
introduce natural generalizations of the usual and Hadamard products and show that although
these generalizations do not satisfy the majorization inequality, they do satisfy an important
weaker inequality that plays a role in establishing their submultiplicativity with respect to every
unitarily invariant norm.
1 Introduction
In this paper we consider three results that are true for both the Hadamard product and the
usual product, and in an attempt to better understand these results, study their generalizations to
families of bilinear products that contain these two familiar products.
Let Mm;n denote the space of m-by-n complex matrices, and dene Mn Mn;n . The Hadamard
product of A; B 2 Mm;n is A B [aij bij ] 2 Mm;n . We use : Mm1 ;n1 Mm2 ;n2 ! Mm3 ;n3 to
denote a bilinear function, which we interpret as a `product'. Given A 2 Mm;n let i (A) denote the
ith largest singular value of A for i = 1; : : :; minfm; ng. A norm k k on Mm;n is unitarily invariant
if kUAV k = kAk for all A 2 Mm;n and all unitary U 2 Mm and V 2 Mn ; a norm k k on Mn is
unitary similarity invariant if kUAU k = kAk for all A 2 Mn and all unitary U 2 Mn . A norm
kkQ on Mm;n is said to be a Q-norm if there is some unitarily invariant norm kk on Mn such that
kAkQ = kAAk1=2 for all A 2 Mm;n [2]; notice that a Q-norm is necessarily unitarily invariant. A
matrix P 2 Mm;n is called a partial isometry if its singular values are all either 0 or 1.
After some preliminary general remarks about bilinear products, in Theorem 3.1 we present
several equivalent conditions for a bilinear product to satisfy the basic inequality
k
X
i=1
i (A B)
k
X
i=1
(1.1)
where q minfm1; m2; m3; n1; n2; n3g. The inequality (1.1) is satised by both the Hadamard
product [8, Lemma 1] and the usual product [5]. Although the proofs of (1.1) for the two products
in [8] and [5] are very dierent, using the characterization in Theorem 3.1 one can readily verify
(1.1) for both the usual product and the Hadamard product by similar arguments.
Next we introduce two families of partitioned products, (3.8) and (3.9), that contain the
usual product and Hadamard product as special cases, and show that the usual product and the
Hadamard product are the only members of these two families for which the basic inequality (1.1)
holds.
In [10] (1.1) was used to prove that, for all A and B of appropriate dimensions and all unitarily
invariant norms, both the usual product and the Hadamard product satisfy an inequality of CauchySchwarz type:
kA Bk2 kAAk kB Bk:
(1.2)
In [8], (1.1) was used to prove
Theorem 1.1 Let k k be a unitarily invariant norm on Mn, and let be either the usual product
k = 1; : : :; n:
(1.3)
that tr AEij = aij . It follows from this observation that a given bilinear product on Mn is
completely determined by the values of tr (A B )C for all A; B; C 2 Mn . In particular, if 1 and
2 are two given bilinear products on Mn , then A 1 B = A 2 B for al A; B 2 Mn if and only if
tr A(1B )C = tr (A 2 B )C for all A; B; C 2 Mn .
Associated with any bilinear product on Mn is another bilinear product R characterized by
the adjoint-like identity
tr (AR B )C tr A(B C ) for all A; B; C 2 Mn :
(2.1)
The notation R is intended to remind us that the parentheses and position of the bilinear product
are moved one position to the right in (2.1). The same process associates with R another bilinear
product (R)R, which we denote simply by RR :
tr (ARRB )C tr A(B R C ) = tr (B R C )A = tr B (C A) for all A; B; C 2 Mn :
(2.2)
The construction of new bilinear products in this way stops at this point however, as one nds that
tr (A RRR B )C = tr B (C R A) = tr (C RA)B = tr C (A B ) = tr (A B )C
for all A; B; C 2 Mn , so RRR = .
It is easy to identity R and RR for several familiar bilinear products . For example, when
is the usual matrix product, we nd that
tr (ARB )C =; tr A(B C ) = tr ABC = tr (A B )C;
so AR B = AB ; it follows that ARRB = AB as well. Thus, when is the usual matrix product
the R and RR are also the usual matrix product.
When is the Hadamard product , it is straightforward to verify that tr (A R B )C = tr A(B
C ) = tr (A B T )C , and
tr (A RR BC = tr A(B R C ) = tr (B R C )A = tr (BC A) = tr (AT B )C:
Thus, A R B = A B T and A RR B = AT B .
Another simple bilinear product is given by A B AB T . One veries easily in this case that
AR B = B T AT and ARRB = AT B.
A fourth bilinear product is given by A B UAV BW , where U; V; W 2 Mn are given. In this
case,
tr (AR B )C = tr A(B C ) = tr A(UBV CW ) = tr (WAUBV )C;
so AR B = WAUBV ), a bilinear product of the same form as , but with the three structural
matrices U; V and W shifted cyclically one position to the right. It follows that aRRB = V AWBU .
This last product is of particular interest because any bilinear product on Mn may be represented as
N
X
A B = Uk AVk BWk for all A; B 2 Mn
(2.3)
k=1
for some N n6 and given Uk ; Vk ; Wk 2 Mn , k = 1; : : :; N . This is clear from the fact that
Eij AElm BEpq = ajl bmpEiq . If a representation of the form (2.3) is known for a given bilinear
product , then R and RR are given by
A R B =
N
X
k=1
N
X
k=1
(2.4)
3 Main Results
Theorem 3.1 Let : Mn Mn ! Mn be given bilinear product. Then the following are equivalent:
P
P
(a) k (A B ) k (A) (B ) for all k = 1; : : :; n, and all A; B 2 M .
i=1 i
i=1 i
(b) jtr P (Q R)j minfrank P; rank Q; rank Rg for all partial isometries P; Q; R 2 Mn .
(c) jtr P (Q R)j 1 for all partial isometries P; Q; R 2 Mn with minfrank P; rank Q; rank Rg =
1.
(d) For all A; B 2 Mn ,
n
X
i=1
and
n
X
i=1
1 (A B) 1 (A)1(B):
n
X
i (A B) 1 (A) i (B)
i=1
i (A B) 1 (B)
n
X
i=1
i(A):
(3.1)
(3.2)
(3.3)
1 (A B) 1 (A)1(B)
1 (ARB) 1 (A)1(B)
(3.4)
(3.5)
1(ARR B) 1 (A)1(B);
(3.6)
and
where R and RR are dened by (2.1) and (2.2).
Proof: Lemma 1.2 shows that (a) implies (b). By the linearity of the trace function, and the fact
that any partial isometry is the sum of rank one partial isometries, (c) implies (b). However, (c) is
a special case of (b), so (b) and (c) are equivalent. Now we show that (b) implies (a).
Suppose that is a bilinear product for which (b) holds, and let A; B be given. Dene
i = j (A) and
n
X
i=j
i = j (B):
A=
n
X
i=1
i Qi and B =
n
X
i=1
i Ri;
where Qi and Pi are rank i partial isometries. Now let P be a given rank k partial isometry and
compute
n
n
X
X
jtr P (A B)j = j tr P [( iQi) ( j Rj )] j
= j tr P (
=
=
n
X
i;j =1
n
X
i=1
n
X
i;j =1
j =1
i j Qi Rj ) j
jtr P (ij Qi Rj )j
ij minfi; j; kg
i;j =1
k X
n
n
X
X
( i )( j )
l=1 i=l
j =l
k
X
l (A)l(B):
l=1
The inequality (a) now follows from Lemma 1.2. We have shown that (a), (b) and (c) are equivalent.
It is clear that (a) implies (d). Now let us show that (d) implies (e). The inequality (3.4) is the
same as (3.1). To deduce (3.5) we use (3.2) to compute
maxf1(A)1(B)
n
X
i=1
i (C ) : C 2 Mn; rank C = 1; 1 (C ) 1g
= 1 (A)1(B ):
The inequality (3.6) follows in the same way from (3.3).
Finally we show that (e) implies (c). Let P; Q; R 2 Mn be partial isometries with minfrank P; rank Q; rank Rg =
1. If rank P = 1 then
jtr P (Q R)j 1(Q R) 1(Q)1(R) = 1
5
by Lemma 1.2 and (3.4). If rank Q = 1 (respectively, rank R =1) then proceed in the same way
using the denition of R (respectively, RR) and (3.6) (respectively, (3.5)).
It is now easy to use Theorem 3.1 (e) to verify the inequality (1.1) for the usual product and
the Hadamard product. This is immediate for the usual product since in this case both R and RR
are the usual product and we have the well known inequality 1 (AB ) 1 (A)1(B ). Since 1 ()
is also submultiplicative with respect to the Hadamard product (see [12] for the original proof or
[6] for a variety of dierent proofs), we have 1 (A B ) 1 (A)1(B ),
1 (ARB)
Now that we have a common proof of the basic inequality (1.1) for the usual product and the
Hadamard product it is natural to look for a common generalization of these two results. For this
reason we considered two natural families of products that contain both the usual product and the
Hadamard product.
Let p; q be positive integers, let n = pq , and let A; B 2 Mn be partitioned as
(3.7)
A22 B = [
p
X
k=1
22 is the
(3.8)
(3.9)
Lemma 3.2 Let p and q be given integers, let n = pq, and let A; B 2 Mn be partitioned as in
(3.7). Then
(p(k ? 1) + j ) = q(j ? 1) + k
j = 1; : : :; p k = 1; : : :; q:
(3.10)
Using the identity (3.10) and the unitary similarity invariance of the singular values, it is clear
that the basic inequality (1.1) holds for the product 21 if and only if it holds for 22 (with respect
to the partitioning pq = n). Consider the following two matrices, where p = q = 2 and n = pq = 4.
01 0 0 11
01 0 0 01
B 0 0 0 0 CC
B
C
A=B
B@ 0 0 0 0 CA ; B = BB@ 00 01 10 00 CCA :
1 0 0 1
0 0 0 1
Then the singular values of A are 2; 0; 0; 0 and those of B are 1; 1; 1; 1, while those of
01 0 1 01
B 0 0 0 0 CC
A21 B = B
B@ 0 0 0 0 CA
0 1 0 1
p p
are 2; 2; 0; 0. Thus, the inequality (1.1) fails to hold for the product 21 (and hence also for 22)
for k = 2; 3; 4, and there is no hope of being able to prove (1.1) for these general products except
in the cases p = 1 or q = 1, which are exactly the usual and Hadamard products.
Despite the failure of the products 21 and 22 to satisfy the basic inequalities (1.1), there is
something more that can be said about them. The inequality (1.2) of Cauchy-Schwarz type and the
equivalences in Theorem 1.1 for the usual and Hadamard products have been strongly associated
in the literature with the basic inequality (1.1). Nevertheless, both of these results are correct for
the general products 21 and 22, which do not satisfy (1.1). Before giving a proof of this assertion,
we introduce two preliminary results. For X 2 Mm;n we denote jX j (X X )1=2.
The following result is part of Theorem 2.3 of [11].
X
A = XL M
is positive semidenite, where X 2 Mm;n . Then, L and M are positive semidenite and
k jX jp k2 kLpk kM pk
for all p > 0, all k = 1; : : :; minfm; ng, and all unitarily invariant norms k k.
(3.11)
Lemma 3.4 Let A; B 2 Mn, let n = pq, and let F = [Fij ]pi;j=1, where Fij 2 Mq is the matrix of
all ones if i = j and is the zero matrix if i =
6 j . Then
(a) kA F k kAk for any unitary similarity invariant norm k k.
(b) The block matrix
is positive semidenite.
(AA) F
(A21 B )
7
(A21B )
(B B ) F
(3.12)
Proof: The assertion in (a) is proved in [3]; A F is a convex combination of (diagonal) unitary
similarities of A.
To prove (b), take any vectors x = [xj ]pj=1 and y = [yj ]pj=1 , with xj ; yj 2 C q . Then, using the
Cauchy-Schwarz inequality inequality twice, we have
jy
(A21B )xj2 =
= j
p
X
i;j =1
p
X
i;j =1
p
X
f
f
= f
i;j =1
p
X
i;j =1
p
X
i=1
kAij yik2kBij xj k2 g2
kAij yik22gf
yi (
p
X
j =1
p
X
i;j =1
kBij xj k22 g
p
X
j =1
p
X
xi (
i=1
The assertion (b) now follows from the criterion in [7, Theorem 7.7.7 (a)].
We can now state and prove the analogs of (1.2) and Theorem 1.1 for the partitioned products
21 and 22.
Theorem 3.5 Let n = pq, let denote either of the products 21 or 22, let F 2 Mn be dened as
in Lemma 3.4, let k k be any given unitarily invariant norm on Mn , and let k kQ be any Q-norm
on Mn . Then
(3.13)
and
(3.14)
Proof: To prove (3.13) for 21, apply (3.11) with p = 1 to the positive semidenite matrix (3.12)
and use (a) in Lemma 3.4:
To prove the second inequality in (3.14), let k k be the unitarily invariant norm associated with
k kQ. Because the matrix in (3.12) is positive semidenite we have [1, Lemma 2]
kA21Bk2Q = k(A21B)(A21B)k
= k((B B ) F )1=2C ((AA) F )C ((B B ) F )1=2 k
1(((BB) F )1=2C ) k(AA) F k 1(C ((BB) F )1=2)
1(((BB) F )1=2) k((AA) F )k 1(((BB) F )1=2)
= 1 (((B B ) F )) k((AA) F )k
1(BB)kAAk
= 12(B )kAk2Q :
For the last inequality we have used Lemma 3.4 (a) with the two unitarily invariant norms 1 ()
and k k. To prove the rst inequality in (3.14), apply the preceding argument to (A21B )(A21 B )
and use the fact that, for any unitarily invariant norms on Mn XX has the same norm asX X for
any X 2 Mn since both products have the same singular values. The inequalities (3.13) and (3.14)
for 21 now imply the same inequalities for 22 by using (3.10) and the unitary invariance of k k.
The equivalence of (a) and (b) is the same as in Theorem 1.1. To show that (c) implies (a), let
A = B = E11, the n-by-n matrix whose 1;1 entry is 1 and all other entries are 0, and notice that
E11 E11 = E11. Then (c) gives kE11k kE11k2, so kE11k 1. Now use the fact that a unitarily
invariant norm is a Schur-convex function of the singular values of its argument [7, Corollary 7.4.47]
to conclude that kAk k1 (A)E11k = 1 (A)kE11k 1 (A). Conversely, if (a) holds then we can
use (3.13), Theorem 1.1 for the usual product, and the fact that every unitarily invariant norm is
self adjoint to obtain
that it is clear from the equivalence of (a) and (e) in Theorem 3.1 that at least one on R and RR is
not spectral norm submultiplicative.) By a straightforward computation we have a representation
of 21 in the form (2.3)
p
X
A21B =
P iiAP ji BP jj ;
i;j =1
ij
where P 2 Mpq denotes the p p block matrix with i; j block
(i.e. P ij = Eij
Iq with Eij 2 Mp ). Now using (2.4) we have
ARB =
p
X
i;j =1
P jj AP ii BP ji and ARRB =
p
X
i;j =1
P ji AP jj BP ii :
and hence that AR B and ARR B are block diagonal with diagonal blocks given by
(ARB )ii =
p
X
k=1
p
X
k=1
Aki Bki :
References
[1] T. Ando, R. A. Horn, and C. R. Johnson. The singular values of a Hadamard product: A
basic inequality. Lin. Multilin. Alg., 21:345{65, 1987.
[2] R. Bhatia. Perturbation inequalities for the absolute value map in norm ideals of operators.
J. Operator Theory, 19:129{36, 1988.
[3] C. Davis. Various averaging operations onto subalgebras. Illinois J. Math., 3:523{53, 1959.
[4] K. Fan. Maximum properties and inequalities for the eigenvalues of completely continuous
operators. Proc. Nat. Acad. Sci., 37:760{6, 1951.
10
[5] A. Horn. On the singular values of a product of completely continuous operators. Proc. Nat.
Acad. Sci., 36:374{5, 1950.
[6] R. A. Horn. The Hadamard product. In Matrix Theory and Applications. American Mathematical Society, Providence, to appear.
[7] R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, New York,
1985.
[8] R. A. Horn and C. R. Johnson. Hadamard and conventional submultiplicativity for unitarily
invariant norms on matrices. Lin. Multilin. Alg., 20:91{106, 1987.
[9] R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge University Press, New
York, 1989.
[10] R. A. Horn and R. Mathias. An analog of the Cauchy-Schwarz inequality for Hadamard
products and unitarily invariant norms. to appear in SIAM J. Matrix Anal. Appl.
[11] R. A. Horn and R. Mathias. Cauchy-Schwarz inequalities associated with positive semidenite
matrices. to appear in Linear Algebra Appl.
[12] J. Schur. Bermerkungen zur Theorie der beschrankten Bilinearformen mit unendlich vielen
Veranderlichen. J. fur Reine und Angewandte Mathematik, 140:1{28, 1911.
11