sample function properties of SIRPs.

If P(A = 0) = 0, then the

sample function properties of Gaussian processes carry over di-
rectly to those of SIRPs, and it is impossible to tell the difference
between a Gaussian process and a SIRP on the basis of one
sample function. Sample function properties of Gaussian pro-
cesses have been widely studied (see, for example, [5, chapters
g-131). We will consider a few special cases of interest, and we will
mation theory, IEEE Trans. Inform. Theory, vol. IT-14, pp. 12-16, Jan.
1968.
I31
K. Yao, A representation theorem and its applications to spherically-invariant
random processes, IEEE Trans. Inform. Theory, vol. IT-19, pp. 600-608,
Sept. 1973.
[41
A. N. Kolmogorov, Foundations of the Theory of Probability. New York:
Chelsea, 1956, pp. 27-33.
[51
H. Cramer and M. R. Leadbetter, Stationary and Related Stochastic Pro-
cesses. New York: Wiley, 1967.
state the results in terms of SIRPs. In the sequel, the zero-mean
SIRP X(t) will be assumed to be separable, stationary, second-
order, and mean-square continuous. Also, we will assume that
P(X(t) = 0) = 0. The next theorem follows from the work of Do-
brushin [7].
[61
I. S. Gradshteyn and I. W. Ryzhik, Table oflntegrals, Series, and Products.
New York: Academic Press, 1965, p. 341, Sect. 3.472-l.
[71
R. L. Dobrushin, Properties of sample functions of a stationary Gaussian
process, Theory of Prob. and Its Appl., vol. 5, pp. 120-122,196O.
[81
Yu. K. Belyaev, Local properties of the sample functions of stationary
Gaussian processes, Theory of Prob. and Its Appl., vol. 5, pp. 117-120,
1960.
Theorem 4: Either 1) the sample functions of X(t) are con-
191
G. A. Hunt, Random Fourier transforms, Trans. Amer. Math. Sot., vol. 71,
tinuous with probability one or 2) with probability one, the
pp. 38-69,195l.
sample functions of X(t) have discontinuities of the second kind
1101
N. D. Ylvisaker, The expected number of zeros of a stationary Gaussian
at every point.
process, Ann. Math. Statist., vol. 36, pp. 1043-1046,1965.
1111
B. Picinbono, Spherically invariant and compound Gaussian stochastic
The following corollary is a result of the further work of Be-
processes, IEEE Trans. Inform. Theory, vol. IT-16, pp. 77-79, Jan. 1970.
1121
B. Picinbono and G. Vezzosi, Detection dun signal certain dan un bruit non
lyaev [8]. stationaire et non Gaussien, Ann. des Telecommunications. vol. 25, pp.
__
Corollary 1: If the sample functions of X(t) are not almost
433-439,197o.
surely continuous, then they are almost surely unbounded in any
1131
G. Vezzosi and B. Picinbono, Detection dun signal certain dans un bruit
interval.
spheriquement invariant, structure et caracteriques des receptions, Ann.
The next theorem follows from the work of Hunt [9].
des Telecommunications, vol. 27, pp. 95-110,1972.
Theorem 5: Let S(w) denote the spectral distribution function
of x(t). If
S
o [log (I+ w,] dS(w) < m
On the Entropy of Continuous Probability Distributions
for some a > 1, then X(t) has continuous sample functions with
probability one. Also, if AIDA C. G. VERDUGO LAZO AND PUSHPA N. RATHIE
S 0
m d [log (1 + ,)]a dS(w) < co Abstract-A table is given of differential entropies for various
continuous probability distributions. The formulas, some of which
for some a > 1, then X(t) has continuous sample function de- are new, are of use in the calculation of rate-distortion functions
rivatives with probability one. and in some statistical applications.
The expected rate of zero crossings of a Gaussian random
process has been studied by many investigators. The following
theorem is based on the work of Ylvisaker [lo].
Shannons differential entropy of an absolutely continuous
random variable X with probability density function f(x) is de-
Theorem 6: Let R(T) denote the autocorrelation function of
fined by
X(t). The expected number of zero crossings of X(t) in an in-
terval of length T is given by
h(X) = -s-f(x) logf(x) dx.
-m
T -R(O) 112
- All logarithms are taken to the base e, and-the entropy h(X) is
a [ 1 R(O)
thus measured in nats thoughout this paper.
if R (7) has a finite second derivative at the origin. If R (7) does not
The proplem of finding which probability density function,
have a finite second derivative at the origin, then the expected
among those satisfying certain conditions, has maximum entropy
number of zero crossings in any interval is infinite.
has been considered extensively in the literature. The solutions
of some basic maximum entropy problems can be found in Kagan,
III. CONCLUSION
Linnik, and Lao [8] and in Lisman and van Zuylen [9]. These are
all univariate density results; some results for multivariate
In this correspondence, we have established the useful repre-
densities can be found in the paper of Gokhale [4].
sentation for SIRPs given in Theorem 3. This theorem was then
In the course of the analysis of the maximum entropy problem,
employed to establish several properties of SIRPs.
use is being made of the following obvious fact. If
Consider a random process of the form AY(t), where Y(t) is
a zero-mean Gaussian process and A is an independent random
f(x) = exp (UO + a&l(x) + - -- + anfnb)l (2)
variable. It is easy to show that such a random process is spheri- then
cally invariant. This approach was explored by Picinbono and
Vezzosi [ll]-[13] (see [3, p. 6041 for comments regarding [ll]).
h(X) = -(a0 + alEhI +a.. + u,Eh,(X)}
(3)
However, the essential result of Theorem 3 is that any SIRP can
always be represented by an equivalent random process having
where
this form. This result illustrates the essential role played by
Gaussian processes in the representation of SIRPs.
Ehi(X) 0 J+- hi(x)f(x) dx.
-cm
REFERENCES Manuscript received March 29,1976, revised May 3,1977. This work was sup-
ported by FAPESP, under Grant 76/1111 and CNP,, under Grant 2222.0042/77.
[l] A. M. Vershik, Some characteristic properties of Gaussian stochastic pro- The authors are with the Departamento de Estatistica, Institute de Matemltica,
cesses, Theory of Prob. and Its Appl., vol. 9, pp. 353-356,1964. Estat;stica e Ciencias da Computacao, Universidade Estadual de Campinas,
[z] I. F. Blake and J. B. Thomas, On a class of processes arising in linear esti- Campinas, SBo Paulo, Brasil.
00189448/78/0100-0120$00.75 0 1978 IEEE
120 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-24, NO. 1, JANUARY 1978
ONDENCE
121
TABLE I
TABLE OFDIFFERENTIALENTROPIES
t
7
DISTRIBUTIONS
E"tP.?py
Name
Density Function
(in nats)
f(x) = xp-I(1 - x)q-l/B(P.q) i 0 5 x 5 1 log E3(p,q) - (p-l) IIJI(P)-*(P+q)l
Beta
Where B(p,q) = r(p)r(q)/r(P+q) : P. 4 ' 0
- (q-l) [$ (q)-a(p+q)]
Cauchy
f(X) = (i/n) (h2+x2)-l i -m < x < m, i > 0 log(4nh)
Chi
f(x) ={2/[2"'2 0" r(n/2)]Ix"-l e-x2'(202);x>0 log[or(n/2)/JZ] _
y $(n/2) + n/2
and n is a positive integer
Chi-square
f(X) = .(42)-l .-x/(202),[2n/2 ?r(n/Z)]; x,0 loq[202
r(n/2)] + (l-n/2) yw2) + n/2
and n is a positive integer
Erlang
f(x) = [p/(*-l,!] 2-l e-BX ; x, 6 > 0
(1 - n) $'(n) + loqCr(n,L3] + n
and n is a positive integer
Exponential
f(X) = o-1 e-x/a : x, '3 > 0
.J /2
" z/2
F
f(X) =
"I ' 2
,(v,/2)-1
log[(",/V*,B(V,/2.V2/2~l+(l-~~/2)*~~,/2)
B(V,/2.Vz/2) (" +v x) ("1+"2)'2
;x>o --==-A
and ", , "2 are positi& integer
-(l+"~/2)$(v,/2)+[(v,+v*~/~~[~vlfvz)/2]
Gamma f(x) = xa-l e-X'@/[fi"r(a)] i x, a, B ) 0 log@r(u)] + (1-a) $(a) + a
Laplace f(x) = (l/2) q-1 .-lx-ei'!J ; -m < x < m, @.o
1 + log(2 v )
Logistic
f(x) = e-x (1 + e-X)-2 ; -m < x < -
2
Lognormal f(x) = [,xsrzll)]
-1 e-uogx-m)2/(202) : x > 0
m + (l/2) log(2neo2)
I
Maxwell-Bolt7.ma
ftx) = [4n-1/283/2] x2 e-flX2
i x. 6 > 0 (l/2) log (n/B) + y - l/2
/
Normal
f(x) = Lomi,,,-l e-x2/(2o2) -m < x < ml a>0
(1/2)loq(2neo2)
I
Gsnerallzed-Noml,f(x) = [2&2/r(a/2,] xa-l e-6x2: x,a,
6>0 log~(u/2)/(261'2 )] - [(a-1)/2]$W2) +a/2
Pare to f(x) = a ka/xa+' ; x z k > 0, a > 0
loq(k/a) + 1 +1/a
Rayleigh f(x) = (x/b') e-x2'(2b2' ; x, b > 0
1 + log(B/J2) + y/2
f(x) = (1 + x2/")-("Cl)'2
;--m<X<m
Student-t
J(v) 8(1/2,v/2)
[(v + 1)/2]I$[(V + 1)/2] - $(V/2)1
and v is a positive integer
+ log [J(V)B(1/2,V/2)]
Triangular f(x) = 2x/a ; ojxca
= 2(1 - X)/(1 - a) ;-
l/2 - log2
a<x<1
Uniform
f(X) = l/(6 - a) : a~x<6
log(B - cl)
Weibull f(x) = (c/a) x
c-1 e -xc/a i x,
c, a ) 0 cc - l)y/c + lclg(P/c) + 1
122
IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. IT-24, NO. 1, JANUARY 1978
This reduces the problem of calculating h(X) to that of calcu-
lating Ehi (X).
Shannon [ll] was the first to compute the entropy of the uni-
form, the exponential and the normal distributions. Golomb [5]
computed the entropy for the Pareto distribution; the entropy
for the squared Cauchy distribution appeared in Bergers book
[l]. For the Laplace, logistic, lognormal, and Weibull distribu-
tions, the Eh;(X)s can be found in the books of Johnson and
Kotz [6], [7] expressed in terms of elementary functions. We add
to this list the following distributions: Cauchy, Maxwell-
Boltzmann, Rayleigh, Triangular.
For each of the remaining distributions listed in this paper
(beta, chi, &i-square, F, gamma, generalized-normal, student-t),
at least one of the terms Ehi(X) is not expressible in terms of
elementary functions. The main object of this correspondence
is to point out that a single special function, the logarithmic de-
rivative # of the gamma function, may be used to specify h(X)
for all these distributions.
The definitions of the #-function and of Eulers constant y (to
enough precision for numerical calculations) are as follows:
Hz) = $log I%) = -y + (2 - l)$(k + l)(z + iz)]-1
(4)
y = 0.5772156649 -. - . (5)
For more details on the #-function, see [lo].
The entropy h(X), given by (3),can be easily obtained by
using
a)
b)
4
4
[2, p. 314, (S)] and [2, p. 316, (22)] for the beta distribu-
tion,
[3, p. 233, (8)], [2, p. 314, (8)], and [2,316, (23)] for the F-
distribution,
[2, p. 315, (9)] and [2, p. 312, (l)] for the gamma distribu-
tion,
[2, p. 312, (l)], [2, p. 315, (9)], and [2, p. 133, (3)] for the
generalized normal distribution.
The results are summarized in Table I.
ACKNOWLEDGMENT
The authors would like to thank Prof. T. Berger and the ref-
erees for their helpful comments and suggestions.
Ill
121
[31
141
[51
[3
[71
is1
PI
IlO1
Iill
REFERENCES
T. Berger, Rate Distortion Theory. Englewood Cliffs, NJ, Prentice-Hall,
1971.
A. ErdBlyi et al., Tables of Integral Transforms, vol. I. New York:
McGraw-Hill, 1954.
A. Erdblyi et al., Tables of Integral Transforms, vol. II. New York:
McGraw-Hill. 1954.
D. V. Gokhale, Maximum entropy characterization of some distributions,
in Statistical Distributions in Scientific Work, vol. 3, Patil, Katz, and Ord,
Eds. Boston, MA: Reidel, 1975, pp. 299-304.
S. W. Golomb, The information generating function of a probability distri-
bution, IEEE Trans. Inform. Theory, vol. IT-12, pp. 75-77, Jan. 1966.
N. L. Johnson and S. Katz, Distributions in Statistics: Continuous Uniuariate
Distributions--I. New York: Wiley, 1970.
N. L. Johnson and S. Katz, Distributions in Statistics: Continuous Uniuariate
Distributions-2. New York: Wiley, 1970.
A. M. Kagan, Ju. V. Linnik, and C. R. Rao, Characterization Problems in
Mathematical Statistics. New York: Wilev. 1973.
J. H. C. Lisman and M. C. A. van Zuylen, %Jote on the generation of most
probable frequency distributions, Statistica Neerlandica, vol. 26, pp. 19-23,
1972.
W. Magnus, F. Oberhettinger, and R. P. Soni, Formulas and Theorems for
the Special Functions of Mathematical Physics. New York: Springer-Verlag,
1966.
C. E. Shannon, A mathematical theory of comunication (concluded), Bell
Syst. Tech. J., vol. 27, pp. 629-631,1948.
Advent of Nonregularity in Photon-Pulse Delay
Estimation
ISRAELBAR-DAVID, MEMBER,IEEE,AND MOSHELEVY
Abstract-The mean-square error of the delay estimate of a
photon pulse is known to decrease as Q- if the pulse envelope is
smooth, but as Qe2 if it has sharp edges, thereby belonging to
nonregular estimation cases (Q denotes the expected photon
count). The transition from the Q-l to the Qm2 law is investigated
for trapezoidal pulse models, and is found to occur in the region
where the standard deviation of the error is on the order of the
width of the pulse slopes. Thus for values of Q below this region,
practical pulses are effectively rectangular, and their estimation
problem may be qualified as nonregular.
This note relates to one facet of the problem of estimation of
photon-pulse delay with direct-detection receivers, which has
been exposed in a previous contribution [l] to which we refer the
reader for appropriate background. It has been shown that,
asymptotically with the expected photon count Q in the pulse,
the mean-square error (mse) of the delay estimate decreases only
as Q-l with pulses that have smooth envelopes, i.e., regular
estimation cases [2], but as Q-z with pulses that have sharp edges.
The latter envelopes pertain to nonregular estimation problems
and, apparently, the nonregularity changes the dependence from
the Q-l to the Qe2 law. As many practical pulse envelopes are
more readily associated with a rectangular model than with a
smooth one, the way in which the signal shape affects the tran-
sition between these laws has been suggested for further inves-
tigation [l, item (3) of Section II-C]. Reformulated, the question
is how inclined need a slope be in order that the pulse may qualify
as a rectangular one?
Herein, we analyze the effect by modeling real-life pulse en-
velopes by trapezoids of varying slope as illustrated in Fig. 1. Let
the pulse envelope X(t) be given by
-(l + a)D I t I -D
-DItID
[(l + a)D - t]X,
D 5 t I (1 + a)D
a-D
0,
otherwise.
(1)
Then the expected photon count in the pulse is
Q = I:::&:,
X(t) dt = (a + 2)0X,,
the received envelope is X(t - T), and the observables are the
instants tk, k = 1,2, . . - , L, of the photoelectron emissions at the
direct detection receivers output. The minimum m.s.e. estimate
of the delay r for such an envelope is not expressible explicitly
in terms of tk, as it fortunately is for rectangular ones [I]. We
therefore adopt here the estimate which is optimum for the
rectangular envelope (with (Y = 0 in (1)) and is given [ 11, for L I
Manuscript received March 2.1977; revised May 6.1977. This correspondence
incorporates results from a Master of Science theses submitted by M. Levy to the
Senate of the Technion-Israel Institute of Technology, Haifa, Israel, in October
1976.
The authors are with the Faculty of Electrical Engineering, Technion-Israel
Institute of Technology, Haifa, Israel.
0018-9448/78/0100-0122$00.75 0 1978 IEEE