Audio Classification Using Features Derived From The Hartley Transform
I. Paraskevas, E. Chilton,
Centre for Vision Speech and Signal Processing (CVSSP)
School of Electronics and Physical Sciences, University of Surrey
Guildford, GU2 7XH, Surrey, UK
E-mail: eeplip@ee.surrey.ac.uk, e.Chilton@ee.surrey.ac.uk
Keywords: Audio recordings, audio databases, content-based retrieval, classification, phase spectrum, Hartley
transform.
Abstract - The increasing use of audio databases has led to
the need for automatic content-based retrieval and
classification of audio signals. In this paper, a feature
extraction method is presented for classification of audio
recordings. The proposed method employs magnitude and
phase-related information based on the Hartley transform.
Up to now, the features extracted from an audio recording
have been temporal and magnitude spectrum related, since
the phase spectrum could not be used due to the
discontinuities it exhibits. In the proposed method, the
sources of phase discontinuities are detected and overcome,
resulting in a phase spectrum in which the number of
discontinuities is significantly reduced. Experimental results
show that the classification performance is improved, using
magnitude and phase related information together, compared
to the case where only magnitude is used.
1. INTRODUCTION
Recent research on automatic content-based
classification of audio recordings employs magnitude
spectrum and temporal related features to classify different
acoustic events retrieved from audiovisual databases, [1].
In most cases, the task is to classify sounds that are
acoustically different and yet belong to the same family,
such as sports sounds, [2]. In other cases, existing research
work attempts to discriminate between various TV events,
such as news, advertisements, etc. that are acoustically
dissimilar, [3].
Some of the features employed for speech recognition,
such as Mel-scale cepstral coefficients or Perceptual
Linear Prediction coefficients, are not always effective for
audio classification, as speech signals constitute a special
group within the family of audio signals. Magnitude
spectrum features convey energy-related information that
is essential for the classification process. Hence, for audio
classification, most of the frequency domain feature
extraction techniques are based on the spectral magnitude
content of the signal, [3]. The calculation of the magnitude
spectrum of a signal preserves information related to the
absolute value of its real and imaginary Fourier spectrum
components, but does not preserve information related to
the signs/change of signs of these components. The aim of
this work is to show that, for certain classes of audio
signals, the feature vectors extracted from the phase
content of a signal perform better, in terms of
classification, as compared to the feature vectors extracted
from the magnitude content of a signal, in the frequency
domain. Consequently, phase-related feature vectors can
be used for frequency based feature extraction, towards
M. Rangoussi
Department of Electronics
Technological Education Institute of Piraeus
250, Thivon str., Aigaleo-Athens, GR-12244, GREECE
Phone/Fax: +302105381222,6 E-mail: mariar@teipir.gr
audio classification. In this paper, the phase and magnitude
information are combined in order to discriminate sounds
that spectrally belong to the same family.
Most researchers do not use phase information in the
classification process, although it conveys essential
information, because of the difficulties in processing the
discontinuities in the phase spectrum. In our previous work
[7], the Fourier transform was used in order to implement
the phase spectrograms. In this paper, the Hartley
transform is used instead. As will be explained in section
5, the major advantage of the Hartley over the Fourier
transform is that the Hartley-derived phase related
spectrum conveys fewer discontinuities and also
encapsulates the phase related content of the signal in an
improved manner.
The discontinuities appearing in the phase spectrum are
unwanted features that affect the classification rate. There
exist two kinds of phase discontinuities: the “extrinsic”
type arises from the computation of the phase, [4], whereas
the “intrinsic” type is due to the structure of the signal, [5].
Note that, the proposed method for the process of the
phase spectrum is non-parametric so it does not use any a-
priori information [6].
The novelty of the proposed method is the use of phase
related information together with magnitude information,
for frequency based statistical feature extraction for audio
classification. Feature vectors, containing phase and
magnitude spectrogram statistical values, are
independently passed to a Mahalanobis metric classifier.
The experimental results obtained from a database of
spectrally similar sounds, indicate that for certain classes
of audio signals the use of phase-related feature streams
together with magnitude feature streams (section 4)
increase the classification rate as compared to the only-
magnitude feature streams case.
2. PHASE SPECTRUM DISCONTINUITIES
2.1. Phase spectrum computation via the DTFT
The phase spectrum calculation is first carried out in the
Fourier transform domain, using the Discrete-Time Fourier
Transform (DTFT), in order to investigate the signal
properties in that domain. After establishing the
relationship between the Fourier and the Hartley
transforms, the analysis will be extended to the Hartley
transform domain.
The phase of the DTFT is defined as:

⎛ S (ω ) ⎞ (1)
f (ω ) = arctan ⎜⎜ I ⎟⎟,0 ≤ f (ω ) ≤ 2π ,
⎝ S R (ω ) ⎠
where S (ω ) is the complex Fourier spectrum of the data
and S I (ω ) , S R (ω ) are its imaginary and real
components respectively.
When the phase information is extracted from S (ω ) ,
two problems arise. The first one is related to the
discontinuities that the inverse tangent function presents
(‘extrinsic’ discontinuities). The computation of this
function results in phase values modulo 2π . In order to
overcome these ambiguities, the phase has to be
‘unwrapped,’ [4].
The second kind of discontinuity is caused by ‘intrinsic’
characteristics of the system that generates the data under
analysis. The ‘intrinsic’ discontinuities appear at
frequencies ω (‘critical points’) where both the imaginary
and the real part of the Fourier spectrum become
simultaneously zero, during the phase evaluation process.
This is equivalent to the existence of a ‘pole’ or a ‘zero’ of
the signal on the circumference of the unit circle in the z-
domain, [5], [8]. This latter kind of discontinuity causes π
‘jumps’ in the phase spectrum, [5]. In order to overcome
the ‘intrinsic’ discontinuities, the ‘critical points’ of the
signal have to be detected first. Then the spectrum is
scanned from lower (zero) to higher frequencies and π is
added to the rest of the phase spectrum values if the value
before the critical point is higher or subtracted, if it is
lower.
The techniques used to detect and overcome ‘extrinsic’
and ‘intrinsic’ discontinuities have their drawbacks. For
the ‘extrinsic’ discontinuities, based on [4], whenever a
phase ‘jump’ occurs, 2π is added or subtracted
accordingly. The disadvantage is that phase ‘jumps’ can be
caused either by rapidly changing angles or due to the
‘wrapping’ ambiguity; yet, conventional ‘unwrapping’
algorithms cannot discriminate the cause of these ‘jumps’.
On the other hand, ‘intrinsic’ discontinuities may appear
when both the real and imaginary parts of the complex
representation of the spectrum assume values very close to
- but not exactly - zero, due to the precision limitations of
discrete computation, [13].
Although the ‘intrinsic’ discontinuities are due to the
structure of the signal and not to computational artifacts,
their existence still reduces the classification rate. The
classification score is maximized when both sources of
discontinuities are detected and removed, [7].
2.2. Phase spectrum computation via the z-transform
The alternative approach for the removal of the
discontinuities appearing in the phase spectrum is based on
the z-transform. Any signal can be modeled as an all-
‘zero’ filter by taking its roots, [8]. The ‘zeros’ (roots of
the polynomial formed by the signal) lying on the unit
circle are causing π or near- π ‘jumps’ in the phase
spectrum, [5]. These phase ‘jumps’ are the ‘intrinsic’
discontinuities.
In the proposed method, [7], all the zeros lying on the
unit circle are removed and the phase spectrum is re-
constructed from the remaining zeros, based on the
‘geometric’ evaluation of phase, [8]. In practice, however,
due to computational errors in the evaluation of the roots
of the polynomial, zeros lying not exactly on but within a
certain distance (‘ring’) of the unit circle have to be
removed as well. The width of the ‘ring’ has to be kept
within certain limits so that the information loss is limited
(see Section 5).
The advantage of this method over the DTFT-based
method is that, although each phase component still comes
out of the inverse tangent function, its value is constrained
within ± π radians. Therefore, ‘wrapping’ ambiguities do
not arise (phase is not ‘wrapped’ around zero) and
‘extrinsic’ discontinuities do not appear at all.
2.3. Fourier transform and Hartley transform relation
In this subsection, the Hartley spectrum magnitude and
phase are defined, based on the relationship between the
Fourier and the Hartley transforms. Let H (ω ) denote the
Hartley spectrum; then, [9],:
H (ω ) = S R (ω ) − S I (ω ) (2)
On the other hand, the Fourier spectrum S (ω ) can be
written as
S (ω ) = M (ω )(cos ϕ (ω )) − j sin(ϕ (ω ))) (3)
in terms of its magnitude M (ω ) , (4), and phase ϕ (ω ) ,
(5), respectively:
M (ω ) = S R (ω ) + S I (ω )
2 2
⎛ S (ω ) ⎞
ϕ (ω ) = arctan⎜⎜ I ⎟⎟
⎝ S R (ω ) ⎠
The real and imaginary parts of S (ω ) become:
S R (ω ) = M (ω ) cos(ϕ (ω )) (6)
S I (ω ) = − M (ω ) sin(ϕ (ω )) (7)
Then, the Hartley spectrum H (ω ) and its ‘conjugate’
H ∗ (ω ) = S R (ω ) + S I (ω ) take on the forms:
H (ω ) = M (ω )(cos(ϕ (ω )) + sin(ϕ (ω ))) (8)
H ∗ (ω ) = M (ω )(cos(ϕ (ω )) − sin(ϕ (ω ))) (9)
The Hartley spectrum magnitude, N (ω ) , is defined as:
N (ω ) = H (ω ) H * (ω ) = M (ω ) cos(2ϕ (ω )) (10)
The Hartley spectrum magnitude combines in a single
quantity both Fourier spectrum magnitude and phase;
hence, it captures all Fourier spectrum-related information
of the signal.
Furthermore, the ‘whitened’ Hartley spectrum or
Hartley phase spectrum, Y (ω ) , is defined as:
Y (ω ) = H (ω ) / M (ω ) = cos(ϕ (ω )) + sin(ϕ (ω )) (11)
Y (ω ) is a function of the Fourier phase only, [14]; as
such, it inherits the ‘intrinsic’ discontinuities. Also Y (ω )
values are bounded within ± 2 , from (11).
2.4. Hartley phase spectrum computation
 (1) If we choose to employ the DTFT-based phase
computation method, (Subsection 2.1), ‘intrinsic’
discontinuities can be removed as follows:
Let ω denote a critical frequency point, where π
∗
should be either added or subtracted to the Fourier phase
for compensation, and let b = ϕ (ω ) . Then, by (11), the
∗
compensated Y (ω ) is given by (12) in either case:
∗
cos(b ± π ) + sin(b ± π ) = − (cos(b) + sin(b)) (12)
Thus, compensation for a ‘intrinsic’ discontinuity is
achieved by multiplication of Y (ω ) by (-1).
Consequently, the method of Subsection 2.1 can now be
simplified as follows (computation of Y (ω ) via the
Discrete-time Hartley Transform – DTHT): After the
‘critical points’ of the signal have been detected, the
spectrum is scanned from lower to higher frequencies.
Whenever a critical frequency point ω is met, all Y (ω )
∗
values for ω ≥ ω are multiplied by (-1).
∗
(2) If we choose to employ the z-transform based phase
computation method (Subsection 2.2), this method can also
be modified to employ Y (ω ) (computation of Y (ω ) via
the z-transform). Y (ω ) can be evaluated geometrically,
(see Appendix).
3. STATISTICAL ANALYSIS & CLASSIFICATION
In order exploit the Hartley phase properties mentioned
earlier in an audio signal classification problem, eight (8)
different statistical features are extracted by analysis of
each spectrogram (Hartley phase spectrum), namely:
variance, skewness, kurtosis, entropy, inter-quartile range,
range, median, mean absolute deviation, [10], [11]. The
value range of the Hartley phase spectrum is not used as a
feature for either of the ‘whitened’ Hartley spectrograms,
because Y (ω ) is always bounded within ± 2 .
As the focus of our work is on the statistical feature
extraction rather than the classifier, a simple Mahalanobis
distance classifier is employed in order to classify the
acoustic patterns. The Mahalanobis distance is defined as:
d ( xt , xr ) = ( xr − xt )Cr−1 ( xr − xt )T , (13)
where xt and xr are the test and the reference feature
vectors, respectively and Cr is the covariance matrix of
the reference data, [12].
A codebook is derived from the mean values of each
class of audio signals and represents the class. The
distance between a test pattern and the codeword of each
class is calculated and the test pattern is assigned to the
class that yields the minimum distance among all classes.
4. AUDIO DATABASE & EXPERIMENTAL SET UP
An audio signal database containing gunshot recordings
(10 classes of 10 recordings each, on average) is used to
test the performance of the proposed Hartley spectrum-
based feature set. The ten classes contain firings of:
i) revolver, ii) .22 caliber handgun, iii) M-1 rifle, iv)
World War II German rifle, v) cannon, vi) 30-30 rifle, vii)
0.38 calibre semi-automatic pistol, viii) lever action
Winchester rifle, ix) 37mm anti-tank gun and x) pistol.
The particular database is chosen as a demanding
classification task, because all ten classes belong
acoustically to the same family. Each recording has
different length from the rest, as it contains real field data.
Seven (7) recordings of each class are used as training data
and the rest as test data.
For the spectrogram computation, each signal is
segmented into frames of equal length, with zero padding.
Frame length is set to 256 samples, to limit the round-off
error in the calculation of the polynomial roots (Subsection
2.2.) from each frame.
Five (5) different spectrograms are calculated from each
recording of the database, and feature vectors are extracted
from each of them:
1. Hartley magnitude spectrogram, based on (10),
2. Hartley transform spectrogram, based on (2),
3. Fourier magnitude spectrogram,
4. ‘whitened’ Hartley spectrogram (via the DTHT), and
5. ‘whitened’ Hartley spectrogram (via the z-transform).
5. RESULTS AND DISCUSSION
The correct classification rates (average across ten
classes) obtained from the gunshot database with the
Mahalanobis classifier are as follows:
• Hartley transform spectr.: 82.9%,
• Hartley magnitude spectr. 91.4%,
• whitened Hartley spectr. (via the DTHT): 81.4%,
• whitened Hartley spectr. (via the z-transf.): 82.9%.
These results compare favorably to their Fourier
spectrum counterparts, reported in [7] under the same
experimental set up. In fact, the whitened Hartley
spectrogram (via the DTHT) outperforms the Fourier
spectrogram (via the DTFT) by 5.7% while via the z-
transform by 4.8%. This significant improvement is
explained by the ‘immunity’ of the whitened Hartley
spectrum to the discontinuities, relative to its Fourier
counterpart.
For the computation of the whitened Hartley
spectrogram via the z-transform, the choice of the width of
the exclusion ‘ring’ around the unit circle is studied as to
its impact on the classification rate. The width of the ‘ring’
is varied between 0 (when ‘zeros’ are excluded only if they
lie on the unit circle – not possible in practical calculations
of roots) and 0.001. Further increase would yield
unreliable results, as the ring would exclude too many
‘zeros’. Classification scores vary accordingly between
78.6% and 68.6%. In general, classification scores
decrease as the ring width increases, due to the information
loss incurred; they peak for a ‘ring’ width of 0.00003
(82.9%) – a similar performance and at the same ring
width is reported in [7] for the Fourier spectrogram via the
z-transform.
A closer examination of correct classification scores per
class reveals that the 2nd, 4th, 5th and 10th classes yield the
poorer results and lower the average scores across classes.
In order to assist the decision as to these classes, three out
of the five ‘experts’ or streams are combined in a fusion
scheme shown in Fig.1:
1. Fourier magnitude spectrogram: it encapsulates only [2] Z. Xiong et. al., “Audio events detection based highlights
the magnitude content of the signal, extraction from baseball, golf and soccer games in a
2. Whitened Hartley spectrogram (via the z-transform): it unified framework,” IEEE Intl. Conf. on Acoustics, Speech
conveys the phase-related content of the signal, and and Signal Processing, vol. 5, pp. 632-635, 2003.
3. Hartley magnitude spectrogram: it encapsulates both [3] Y. Wang, Z. Liu, “Multimedia Content Analysis using both
signal magnitude and phase spectral content. Audio and Visual Cues,” IEEE Signal Processing
Magazine, November 2000.
Fourier Mahal. [4] J.M. Tribolet, “A new phase unwrapping algorithm,” IEEE
magn. - classifier Trans. on Acoustics, Speech and Signal Processing, vol.
stream 1
25, pp. 170-177, April 1977.
[5] H. Al-Nashi, “Phase Unwrapping of Digital Signals,” IEEE
Trans. on Acoustics, Speech and Audio Processing, vol. 37,
‘whitened’ Mahal. Major no. 11, November 1989.
Hart. via the classifier vote
z - stream 2
[6] C.L. Nikias, A.P. Petropulu, Higher-order spectra
analysis: a nonlinear signal processing framework,
Prentice Hall - Signal Processing Series, 1993, Ch. 6.
[7] I. Paraskevas, E. Chilton, “Combination of Magnitude and
Hartley Mahal.
magnitude - classifier Phase Statistical Features for Audio Classification,”
stream 3 Acoustics Research Letters Online, vol. 5, (3), July 2004.
[8] J.G. Proakis, D.G. Manolakis, Digital Signal Processing
Fig. 1. Majority vote decision rule: A recording is classified to a Principles, Algorithms, and Applications, Macmillan
certain class if two or more streams agree. Publishing Company, 1992, Ch. 4 and 5.
[9] R.N. Bracewell, The Fourier Transform and Its
An incoming sound is ‘misclassified’ when two or more Applications, McGraw-Hill Book Company, 1986, Ch.19.
out of the three streams assign it to the same ‘false’ class, [10] E. Mansfield, Basic Statistics with Applications, W.W.
and ‘unclassified’ when each one of the three streams Norton and Co., 1986.
assigns it to a different class. [11] A. Papoulis, Probability and Statistics, Prentice-Hall, Inc.,
For these four classes, classification rates per single 1990, Ch. 12.
‘expert’ or stream are tabulated in Table 1. When the [12] P.C. Mahalanobis, “On the generalized distance in
majority vote decision rule of Fig.1 is employed, however, statistics,” Proceedings of the National Institute of Science
the classification score reaches 90.5%, with 9.5%
of India, vol. 12, pp. 49-55, 1936.
unclassified recordings (no misclassifications).
[13] G.E. Forsythe, M.A. Malcom, C.B. Moler, Computer
Methods for Math. Comput, Prentice-Hall, 1977, Sect. 7.
Fourier Whitened Hartley
[14] E. Chilton, “An 8kb/s speech coder based on the Hartley
Magnitude Hartley (z) Magnitude
transform,” ICCS ‘90 Communication Systems: Towards
Class 2 83.3 83.3 83.3
Class 4 63.6 63.6 77.9 Global Integration, vol. 1, pp. 13.5.1-13.5.5, 1990.
Class 5 81.8 91.7 91.7
Class 10 91.7 85.1 91.7
All 4 classes 80.4 81.2 86.5 APPENDIX
Table 1. Correct classification scores in (%) per ‘expert’.
Imaginary
B
z
6. CONCLUSION
A ϕ
ϕ C
We have proposed a novel approach to phase extraction,
based on the Hartley rather than the Fourier transform, ωω
with application to an audio signal classification problem. O

The experimental results obtained indicate that, for certain Real

classes of audio signals, the combination of magnitude and
phase-related information provides an improved
performance, as compared to the independent use of each
information stream. Moreover, the experimental results
indicate that the ‘whitened’ Hartley spectrograms perform
on average better than their Fourier phase counterparts.
The contribution of a ‘zero’ z to the phase spectrum is evaluated
REFERENCES with respect to a single frequency point C, as an example. For the
Fourier spectrum, ϕ (ω ) = arctan(BC / AB ) , while for the
[1] T. Zhang, C.C.J. Kuo, “Audio content analysis for online
Hartley spectrum,
audiovisual data segmentation and classification,” IEEE
Trans. on Acoustics, Speech, and Audio Proc., vol. 9, no.
Y (ω ) = cos(ϕ (ω )) + sin(ϕ (ω )) = ( AB + BC ) /( AC )
4, 2001. Repeat for all frequency points of interest and all ‘zeros’.