You are on page 1of 11

Inequalities for Unitarily Invariant Norms and Bilinear Matrix

Products
Roger A. Horn and Roy Mathias
The Johns Hopkins University, Baltimore, Maryland 21218, USA
Yoshihiro Nakamura
Research Institute for Applied Electricity, Hokkaido University, Sapporo 060, Japan
May 4, 2001

Abstract

We give several criteria that are equivalent to the basic singular value majorization inequality
(1.1) that is common to both the usual and Hadamard products. We then use these criteria
to give a uni ed proof of the basic majorization inequality for both products. Finally, we
introduce natural generalizations of the usual and Hadamard products and show that although
these generalizations do not satisfy the majorization inequality, they do satisfy an important
weaker inequality that plays a role in establishing their submultiplicativity with respect to every
unitarily invariant norm.

1 Introduction
In this paper we consider three results that are true for both the Hadamard product and the
usual product, and in an attempt to better understand these results, study their generalizations to
families of bilinear products that contain these two familiar products.
Let Mm;n denote the space of m-by-n complex matrices, and de ne Mn  Mn;n . The Hadamard
product of A; B 2 Mm;n is A  B  [aij bij ] 2 Mm;n . We use  : Mm1 ;n1  Mm2 ;n2 ! Mm3 ;n3 to
denote a bilinear function, which we interpret as a `product'. Given A 2 Mm;n let i (A) denote the
ith largest singular value of A for i = 1; : : :; minfm; ng. A norm k  k on Mm;n is unitarily invariant
if kUAV k = kAk for all A 2 Mm;n and all unitary U 2 Mm and V 2 Mn ; a norm k  k on Mn is
unitary similarity invariant if kUAU  k = kAk for all A 2 Mn and all unitary U 2 Mn . A norm
kkQ on Mm;n is said to be a Q-norm if there is some unitarily invariant norm kk on Mn such that
kAkQ = kAAk1=2 for all A 2 Mm;n [2]; notice that a Q-norm is necessarily unitarily invariant. A
matrix P 2 Mm;n is called a partial isometry if its singular values are all either 0 or 1.
After some preliminary general remarks about bilinear products, in Theorem 3.1 we present

several equivalent conditions for a bilinear product  to satisfy the basic inequality
k
X
i=1

i (A  B) 

k
X
i=1

i (A)i (B) for all A 2 Mm1 ;n1 ; B 2 Mm2 ;n2 ; k = 1; : : :; q;

(1.1)

where q  minfm1; m2; m3; n1; n2; n3g. The inequality (1.1) is satis ed by both the Hadamard
product [8, Lemma 1] and the usual product [5]. Although the proofs of (1.1) for the two products
in [8] and [5] are very di erent, using the characterization in Theorem 3.1 one can readily verify
(1.1) for both the usual product and the Hadamard product by similar arguments.
Next we introduce two families of partitioned products, (3.8) and (3.9), that contain the
usual product and Hadamard product as special cases, and show that the usual product and the
Hadamard product are the only members of these two families for which the basic inequality (1.1)
holds.
In [10] (1.1) was used to prove that, for all A and B of appropriate dimensions and all unitarily
invariant norms, both the usual product and the Hadamard product satisfy an inequality of CauchySchwarz type:
kA  Bk2  kAAk kB Bk:
(1.2)
In [8], (1.1) was used to prove

Theorem 1.1 Let k  k be a unitarily invariant norm on Mn, and let  be either the usual product

or the Hadamard product. Then the following are equivalent :

(a) kAk  1(A) for all A 2 Mn


(b) kAk  (A) for all A 2 Mn
(c) kA  B k  kAk kB k for all A; B 2 Mn
We show that both (1.2) and Theorem 1.1 hold for the two families of partitioned products (3.8)
ann (3.9), even though the full set of inequalities (1.1) does not hold for these products.
For simplicity of notation we consider only the square case mi = ni = n in the remainder of
this paper; the general rectangular case follows easily from this by adding suitable zero blocks.
The following result of Ky Fan ([4, Theorem 1] or [9, Chapter 3]) will be useful.

Lemma 1.2 Let A 2 Mn . Then


k
X
i(A) = maxfjtr PAj : P 2 Mn ; rank (P )  k; 1 (P )  1g
i=1

k = 1; : : :; n:

(1.3)

2 Some General Remarks on Bilinear Products


An unknown matrix A = [aij ] 2 Mn is completely determined by the values of tr AX for all
X 2 Mn. Indeed, it suces to know these values only for the n2 matrices X = Eij , i; j = 1; : : :; n,
where all entries of Eij are zero except for an entry 1 at the intersection of row i and j ; one checks
2

that tr AEij = aij . It follows from this observation that a given bilinear product  on Mn is
completely determined by the values of tr (A  B )C for all A; B; C 2 Mn . In particular, if 1 and
2 are two given bilinear products on Mn , then A 1 B = A 2 B for al A; B 2 Mn if and only if
tr A(1B )C = tr (A 2 B )C for all A; B; C 2 Mn .
Associated with any bilinear product  on Mn is another bilinear product R characterized by
the adjoint-like identity
tr (AR B )C  tr A(B  C ) for all A; B; C 2 Mn :

(2.1)

The notation R is intended to remind us that the parentheses and position of the bilinear product
are moved one position to the right in (2.1). The same process associates with R another bilinear
product (R)R, which we denote simply by RR :
tr (ARRB )C  tr A(B R C ) = tr (B R C )A = tr B (C  A) for all A; B; C 2 Mn :

(2.2)

The construction of new bilinear products in this way stops at this point however, as one nds that
tr (A RRR B )C = tr B (C R A) = tr (C RA)B = tr C (A  B ) = tr (A  B )C
for all A; B; C 2 Mn , so RRR = .
It is easy to identity R and RR for several familiar bilinear products . For example, when 
is the usual matrix product, we nd that
tr (ARB )C =; tr A(B  C ) = tr ABC = tr (A  B )C;
so AR B = AB ; it follows that ARRB = AB as well. Thus, when  is the usual matrix product
the R and RR are also the usual matrix product.
When  is the Hadamard product , it is straightforward to verify that tr (A R B )C = tr A(B 
C ) = tr (A  B T )C , and
tr (A RR BC = tr A(B R C ) = tr (B R C )A = tr (BC  A) = tr (AT  B )C:
Thus, A R B = A  B T and A RR B = AT  B .
Another simple bilinear product is given by A  B  AB T . One veri es easily in this case that
AR B = B T AT and ARRB = AT B.
A fourth bilinear product is given by A  B  UAV BW , where U; V; W 2 Mn are given. In this
case,
tr (AR B )C = tr A(B  C ) = tr A(UBV CW ) = tr (WAUBV )C;
so AR B = WAUBV ), a bilinear product of the same form as , but with the three structural
matrices U; V and W shifted cyclically one position to the right. It follows that aRRB = V AWBU .
This last product is of particular interest because any bilinear product  on Mn may be represented as
N
X
A  B = Uk AVk BWk for all A; B 2 Mn
(2.3)
k=1

for some N  n6 and given Uk ; Vk ; Wk 2 Mn , k = 1; : : :; N . This is clear from the fact that
Eij AElm BEpq = ajl bmpEiq . If a representation of the form (2.3) is known for a given bilinear
product , then R and RR are given by

A R B =

N
X

k=1

Wk AUk BVk and A RR B =

N
X

k=1

Vk AWk BUk for all A; B 2 Mn :

(2.4)

3 Main Results
Theorem 3.1 Let  : Mn  Mn ! Mn be given bilinear product. Then the following are equivalent:
P
P
(a) k  (A  B )  k  (A) (B ) for all k = 1; : : :; n, and all A; B 2 M .
i=1 i

i=1 i

(b) jtr P (Q  R)j  minfrank P; rank Q; rank Rg for all partial isometries P; Q; R 2 Mn .
(c) jtr P (Q  R)j  1 for all partial isometries P; Q; R 2 Mn with minfrank P; rank Q; rank Rg =
1.
(d) For all A; B 2 Mn ,
n
X
i=1

and

n
X
i=1

1 (A  B)  1 (A)1(B):
n
X
i (A  B)  1 (A) i (B)
i=1

i (A  B)  1 (B)

n
X
i=1

i(A):

(3.1)
(3.2)

(3.3)

(e) For any A; B 2 Mn

1 (A  B)  1 (A)1(B)
1 (ARB)  1 (A)1(B)

(3.4)
(3.5)

1(ARR B)  1 (A)1(B);

(3.6)

and
where R and RR are de ned by (2.1) and (2.2).

Proof: Lemma 1.2 shows that (a) implies (b). By the linearity of the trace function, and the fact
that any partial isometry is the sum of rank one partial isometries, (c) implies (b). However, (c) is
a special case of (b), so (b) and (c) are equivalent. Now we show that (b) implies (a).
Suppose that  is a bilinear product for which (b) holds, and let A; B be given. De ne

i  i (A) ? i+1 (A)  0 i = 1; : : :; n ? 1


i  i (B) ? i+1 (B)  0 i = 1; : : :; n ? 1
4

and n = n (A)  0, n = n (B )  0. Then


n
X
i=j

i = j (A) and

n
X
i=j

i = j (B):

By the singular value decomposition [7, Theorem 7.3.4], we may write

A=

n
X
i=1

i Qi and B =

n
X
i=1

i Ri;

where Qi and Pi are rank i partial isometries. Now let P be a given rank k partial isometry and
compute
n
n
X
X
jtr P (A  B)j = j tr P [( iQi)  ( j Rj )] j

= j tr P (



=
=

n
X
i;j =1
n
X

i=1
n
X

i;j =1

j =1

i j Qi  Rj ) j

jtr P ( i j Qi  Rj )j

i j minfi; j; kg
i;j =1
k X
n
n
X
X
( i )( j )
l=1 i=l
j =l
k
X
l (A)l(B):
l=1

The inequality (a) now follows from Lemma 1.2. We have shown that (a), (b) and (c) are equivalent.
It is clear that (a) implies (d). Now let us show that (d) implies (e). The inequality (3.4) is the
same as (3.1). To deduce (3.5) we use (3.2) to compute

1 (ARB) = maxfjtr (ARB)C j : C 2 Mn ; rank C = 1; 1 (C )  1g


= maxfjtr A(B  C )j : C 2 Mn ; rank C = 1; 1(C )  1g
n
X
 maxf1(A) i(B  C ) : C 2 Mn; rank C = 1; 1(C )  1g
i=1

 maxf1(A)1(B)

n
X
i=1

i (C ) : C 2 Mn; rank C = 1; 1 (C )  1g

= 1 (A)1(B ):
The inequality (3.6) follows in the same way from (3.3).
Finally we show that (e) implies (c). Let P; Q; R 2 Mn be partial isometries with minfrank P; rank Q; rank Rg =
1. If rank P = 1 then
jtr P (Q  R)j  1(Q  R)  1(Q)1(R) = 1
5

by Lemma 1.2 and (3.4). If rank Q = 1 (respectively, rank R =1) then proceed in the same way
using the de nition of R (respectively, RR) and (3.6) (respectively, (3.5)).
It is now easy to use Theorem 3.1 (e) to verify the inequality (1.1) for the usual product and
the Hadamard product. This is immediate for the usual product since in this case both R and RR
are the usual product and we have the well known inequality 1 (AB )  1 (A)1(B ). Since 1 ()
is also submultiplicative with respect to the Hadamard product  (see [12] for the original proof or
[6] for a variety of di erent proofs), we have 1 (A  B )  1 (A)1(B ),

sigma1(ARB) = 1 (A  B T )  1 (A)1(B T ) = 1 (A)1(B);


and

1 (ARB)

= 1 (AT  B )  1 (AT )1 (B ) = 1 (A)1(B ):

Now that we have a common proof of the basic inequality (1.1) for the usual product and the
Hadamard product it is natural to look for a common generalization of these two results. For this
reason we considered two natural families of products that contain both the usual product and the
Hadamard product.
Let p; q be positive integers, let n = pq , and let A; B 2 Mn be partitioned as

A = [Aij ]pi;j=1 ; B = [Bij ]pi;j=1 with Aij ; Bij 2 Mq :

(3.7)

De ne the products 21 and 22 on Mn by

A21B = [Aij Bij ]pi;j=1


and

A22 B = [

p
X

Aik  Bkj ]pi;j=1 :

k=1
22 is the

(3.8)
(3.9)

If p = 1 then 21 is the usual product and


Hadamard product, while if q = 1 then 21 is
the Hadamard product and 22 is the usual product.
Although 21 and 22 look rather di erent, they are essentially the same in the context of unitary
similarity invariant norms. The following result can be proved by direct computation.

Lemma 3.2 Let p and q be given integers, let n = pq, and let A; B 2 Mn be partitioned as in
(3.7). Then

A22B = V ((V  AV )21(V BV ))V 

where V 2 Mn is the permutation matrix corresponding to the permutation

(p(k ? 1) + j ) = q(j ? 1) + k

j = 1; : : :; p k = 1; : : :; q:

(3.10)

Using the identity (3.10) and the unitary similarity invariance of the singular values, it is clear
that the basic inequality (1.1) holds for the product 21 if and only if it holds for 22 (with respect
to the partitioning pq = n). Consider the following two matrices, where p = q = 2 and n = pq = 4.
01 0 0 11
01 0 0 01
B 0 0 0 0 CC
B
C
A=B
B@ 0 0 0 0 CA ; B = BB@ 00 01 10 00 CCA :
1 0 0 1
0 0 0 1
Then the singular values of A are 2; 0; 0; 0 and those of B are 1; 1; 1; 1, while those of
01 0 1 01
B 0 0 0 0 CC
A21 B = B
B@ 0 0 0 0 CA
0 1 0 1

p p

are 2; 2; 0; 0. Thus, the inequality (1.1) fails to hold for the product 21 (and hence also for 22)
for k = 2; 3; 4, and there is no hope of being able to prove (1.1) for these general products except
in the cases p = 1 or q = 1, which are exactly the usual and Hadamard products.
Despite the failure of the products 21 and 22 to satisfy the basic inequalities (1.1), there is
something more that can be said about them. The inequality (1.2) of Cauchy-Schwarz type and the
equivalences in Theorem 1.1 for the usual and Hadamard products have been strongly associated
in the literature with the basic inequality (1.1). Nevertheless, both of these results are correct for
the general products 21 and 22, which do not satisfy (1.1). Before giving a proof of this assertion,
we introduce two preliminary results. For X 2 Mm;n we denote jX j  (X X )1=2.
The following result is part of Theorem 2.3 of [11].

Lemma 3.3 Suppose that the block matrix


X
A = XL M
is positive semide nite, where X 2 Mm;n . Then, L and M are positive semide nite and
k jX jp k2  kLpk kM pk
for all p > 0, all k = 1; : : :; minfm; ng, and all unitarily invariant norms k  k.

(3.11)

Lemma 3.4 Let A; B 2 Mn, let n = pq, and let F = [Fij ]pi;j=1, where Fij 2 Mq is the matrix of
all ones if i = j and is the zero matrix if i =
6 j . Then
(a) kA  F k  kAk for any unitary similarity invariant norm k  k.
(b) The block matrix
is positive semide nite.

 (AA)  F
(A21 B )
7

(A21B ) 
(B  B )  F

(3.12)

Proof: The assertion in (a) is proved in [3]; A  F is a convex combination of (diagonal) unitary

similarities of A.
To prove (b), take any vectors x = [xj ]pj=1 and y = [yj ]pj=1 , with xj ; yj 2 C q . Then, using the
Cauchy-Schwarz inequality inequality twice, we have

jy

(A21B )xj2 =

= j

p
X

i;j =1
p
X
i;j =1
p
X

 f

 f
= f

yi Aij Bij xj j2


(Aij yi )(Bij xj )j2

i;j =1
p
X
i;j =1
p
X
i=1

kAij yik2kBij xj k2 g2
kAij yik22gf
yi (

p
X

j =1

p
X

i;j =1

kBij xj k22 g

Aij Aij )yi gf

p
X
j =1

= y [(AA )  F ]y  x [(B B )  F ]x:




p
X

xi (

i=1

Bij Bij )xi g

The assertion (b) now follows from the criterion in [7, Theorem 7.7.7 (a)].
We can now state and prove the analogs of (1.2) and Theorem 1.1 for the partitioned products
21 and 22.

Theorem 3.5 Let n = pq, let  denote either of the products 21 or 22, let F 2 Mn be de ned as
in Lemma 3.4, let k  k be any given unitarily invariant norm on Mn , and let k  kQ be any Q-norm
on Mn . Then

kA  Bk2  k(AA)  F k k(BB)  F k  kAAk kBBk for all A; B 2 Mn

(3.13)

and

kA  BkQ  1(A)kBkQ and kA  BkQ  1(B)kAkQ for all A; B 2 Mn :

(3.14)

Furthermore, the following are equivalent

(a) kAk  1(A) for all A 2 Mn


(b) kAk  (A) for all A 2 Mn
(c) kA  B k  kAk kB k for all A; B 2 Mn

Proof: To prove (3.13) for 21, apply (3.11) with p = 1 to the positive semide nite matrix (3.12)
and use (a) in Lemma 3.4:

kA21Bk2  k(AA)  F k k(BB)  kAAk kBBk = kAAk kBBk:


8

To prove the second inequality in (3.14), let k  k be the unitarily invariant norm associated with
k  kQ. Because the matrix in (3.12) is positive semide nite we have [1, Lemma 2]

A21 B = ((AA)  F )1=2C ((B  B)  F )1=2


for some contraction C . Thus
(A21 B ) (A21B ) = ((B  B )  F )1=2 C ((AA)  F )C ((B  B )  F )1=2:
Since kXY Z k  1(X ) kY k 1 (Z ) for any unitarily invariant norm we have

kA21Bk2Q = k(A21B)(A21B)k
= k((B B )  F )1=2C ((AA)  F )C ((B  B )  F )1=2 k
 1(((BB)  F )1=2C ) k(AA)  F k 1(C ((BB)  F )1=2)
 1(((BB)  F )1=2) k((AA)  F )k 1(((BB)  F )1=2)
= 1 (((B B )  F )) k((AA)  F )k
 1(BB)kAAk
= 12(B )kAk2Q :
For the last inequality we have used Lemma 3.4 (a) with the two unitarily invariant norms 1 ()
and k  k. To prove the rst inequality in (3.14), apply the preceding argument to (A21B )(A21 B )
and use the fact that, for any unitarily invariant norms on Mn XX  has the same norm asX X for
any X 2 Mn since both products have the same singular values. The inequalities (3.13) and (3.14)
for 21 now imply the same inequalities for 22 by using (3.10) and the unitary invariance of k  k.
The equivalence of (a) and (b) is the same as in Theorem 1.1. To show that (c) implies (a), let
A = B = E11, the n-by-n matrix whose 1;1 entry is 1 and all other entries are 0, and notice that
E11  E11 = E11. Then (c) gives kE11k  kE11k2, so kE11k  1. Now use the fact that a unitarily
invariant norm is a Schur-convex function of the singular values of its argument [7, Corollary 7.4.47]
to conclude that kAk  k1 (A)E11k = 1 (A)kE11k  1 (A). Conversely, if (a) holds then we can
use (3.13), Theorem 1.1 for the usual product, and the fact that every unitarily invariant norm is
self adjoint to obtain

kA  Bk2  kAAk kB Bk  kAk kAk kBk kBk = kAk2kBk2:


Taking square roots gives (c).
In particular, Theorem 3.5 (a,c) shows that 21and 22 are submultiplicative with respect to the
spectral norm, a general result that includes spectra norm submultiplicativity for both the usual
and Hadamard product.
Although the products 21 and 22 both satisfy the Cauchy-Schwarz inequality (3.13), and so
in particular are submultiplicative with respect to the spectral norm, the products R and RR
associated with them are not even are not even submultiplicative with respect to the spectral
norm. We will only prove this statement for the products R and RR associated with 21. (Note
9

that it is clear from the equivalence of (a) and (e) in Theorem 3.1 that at least one on R and RR is
not spectral norm submultiplicative.) By a straightforward computation we have a representation
of 21 in the form (2.3)
p
X
A21B =
P iiAP ji BP jj ;
i;j =1
ij
where P 2 Mpq denotes the p  p block matrix with i; j block
(i.e. P ij = Eij
Iq with Eij 2 Mp ). Now using (2.4) we have

ARB =

p
X

i;j =1

P jj AP ii BP ji and ARRB =

equal to Iq and all other blocks 0

p
X

i;j =1

P ji AP jj BP ii :

and hence that AR B and ARR B are block diagonal with diagonal blocks given by
(ARB )ii =

p
X

k=1

Aik Bik and (ARRB)ii =

p
X

k=1

Aki Bki :

Using these formulae with p = q = 2 one can see that for


01 0 0 11
01 0 0 01
B 0 0 0 0 CC
BB 0 0 1 0 CC
A=B
and
B
=
B
B@ 0 0 0 0 CA
C
@0 0 0 0A
0 0 0 0
0 0 0 0
we have
02 0 0 01
B 0 0 0 0 CC
AR B = B T RRAT = B
B@ 0 0 0 0 CA ;
0 0 0 0
from which it is evident that neither R nor RR are submultiplicative with respect to the spectral
norm.
Acknowledgement We are grateful to Prof. T. Ando for pointing out that the inequality (1.1) is
false for 21 .

References
[1] T. Ando, R. A. Horn, and C. R. Johnson. The singular values of a Hadamard product: A
basic inequality. Lin. Multilin. Alg., 21:345{65, 1987.
[2] R. Bhatia. Perturbation inequalities for the absolute value map in norm ideals of operators.
J. Operator Theory, 19:129{36, 1988.
[3] C. Davis. Various averaging operations onto subalgebras. Illinois J. Math., 3:523{53, 1959.
[4] K. Fan. Maximum properties and inequalities for the eigenvalues of completely continuous
operators. Proc. Nat. Acad. Sci., 37:760{6, 1951.
10

[5] A. Horn. On the singular values of a product of completely continuous operators. Proc. Nat.
Acad. Sci., 36:374{5, 1950.
[6] R. A. Horn. The Hadamard product. In Matrix Theory and Applications. American Mathematical Society, Providence, to appear.
[7] R. A. Horn and C. R. Johnson. Matrix Analysis. Cambridge University Press, New York,
1985.
[8] R. A. Horn and C. R. Johnson. Hadamard and conventional submultiplicativity for unitarily
invariant norms on matrices. Lin. Multilin. Alg., 20:91{106, 1987.
[9] R. A. Horn and C. R. Johnson. Topics in Matrix Analysis. Cambridge University Press, New
York, 1989.
[10] R. A. Horn and R. Mathias. An analog of the Cauchy-Schwarz inequality for Hadamard
products and unitarily invariant norms. to appear in SIAM J. Matrix Anal. Appl.
[11] R. A. Horn and R. Mathias. Cauchy-Schwarz inequalities associated with positive semide nite
matrices. to appear in Linear Algebra Appl.
[12] J. Schur. Bermerkungen zur Theorie der beschrankten Bilinearformen mit unendlich vielen
Veranderlichen. J. fur Reine und Angewandte Mathematik, 140:1{28, 1911.

11

You might also like