Theory: Convolution .

convolution is a mathematical operation on two functions f and g, producing a th ird function that is typically viewed as a modified version of one of the origin al functions, giving the area overlap between the two functions as a function of the amount that one of the original functions is translated. compute a sliding, weighted-average of function f(T), where the weighting functi on is g(-T).

Correlation . Correlation is the statistical relationship between two random variables or two sets of data. It indicates a predictive relationship that can be exploited in pr actice. correlation coefficients, measures the degree of correlation or linear relations hip between two variables "Pearson's correlation." is obtained by dividing the covariance of the two varia bles by the product of their standard deviations. 1. The Pearson correlation is +1 in the case of a perfect positive (increasing) linear relationship (correlation), 2. -1 in the case of a perfect decreasing (negative) linear relationship (antico rrelation), 3. and some value between -1 and 1 in all other cases, indicating the degree of linear dependence between the variables. 4. zero there is less of a relationship (closer to uncorrelated). 5. The closer the coefficient is to either -1 or 1, the stronger the correlation between the variables.

Cross-correlation cross-correlation is a measure of similarity of two waveforms as a function of a time-lag applied to one of them. It is commonly used for searching a long-signa l for a shorter, known feature. It also has applications in pattern recognition. 1. The cross-correlation is similar in nature to the convolution of two function s. 2. Correlation is always used to include a standardising factor in such a way th at correlations have values between -1 and +1, and the term cross-correlation is used for referring to the correlation corr(X, Y) between two random variables X and Y, while the "correlation" of a random vector X is considered to be the cor relation matrix (matrix of correlations) between the scalar elements of X. 3. consider two real valued functions f and g differing only by an unknown shift along the x-axis. One can use the cross-correlation to find how much g must be shifted along the x-axis to make it identical to f. The formula essentially slid es the g function along the x-axis, calculating the integral of their product at each position. When the functions match, the value of conjugate of f and g is m aximized. This is because when peaks (positive areas) are aligned, they make a l arge contribution to the integral. Similarly, when troughs (negative areas) alig n, they also make a positive contribution to the integral because the product of

two negative numbers is positive. 4. With complex-valued functions f and g, taking the conjugate of f ensures that aligned peaks (or aligned troughs) with imaginary components will contribute po sitively to the integral. (f \star g)(t)\ \stackrel{\mathrm{def}}{=} \int_{-\infty}^{\infty} f^*(\tau) \ g(t+\tau)\,d\tau, (f \star g)[n]\ \stackrel{\mathrm{def}}{=} \sum_{m=-\infty}^{\infty} f^*[m]\ g[n+m].

Autocorrelation Autocorrelation is the cross-correlation of a signal with itself. It is the simi larity between observations as a function of the time separation between them. I t is a mathematical tool for finding repeating patterns, such as the presence of a periodic signal which has been buried under noise, or identifying the missing fundamental frequency in a signal implied by its harmonic frequencies. Given a signal f(t), the continuous autocorrelation R_{ff}(\tau) is most often d efined as the continuous cross-correlation integral of f(t) with itself, at lag \tau. When the autocorrelation function is normalized by mean and variance,, that is, without subtracting the mean and dividing by the variance, it is sometimes refer red to as the autocorrelation coefficient.

Fourier transform The Fourier transform, is a mathematical transform, transforming a mathematical function of time domain, f(t), into a new function, in frequency domain with uni ts of cycles/sec (hertz) or radians per second. The new function is then known a s the Fourier transform and/or the frequency spectrum of the function f. The Fou rier transform is also a reversible operation. In the case of a periodic function (for example, a continuous but not necessaril y sinusoidal musical sound), the Fourier transform can be simplified to the calc ulation of a discrete set of complex amplitudes, called Fourier series coefficie nts. Also, when a time-domain function is sampled to facilitate storage or compu ter-processing, it is still possible to recreate a version of the original Fouri er transform according to the Poisson summation formula, also known as discretetime Fourier transform. Periodic functions are written as the sum of simple waves mathematically represe nted by sines and cosines. The Fourier transform is an extension of the Fourier series that results when the period of the represented function is lengthened an d allowed to approach infinity. Due to the properties of sine and cosine, it is possible to recover the amplitud e of each wave in a Fourier series using an integral. In many cases it is desira ble to use Euler's formula, which states that e2pi?= cos(2p?) + isin(2p?), to wr ite Fourier series in terms of the basic waves e2pi?. This has the advantage of simplifying many of the formulas involved, and provides a formulation for Fourie r series.

Discrete Fourier transform It convert the sampled function from its original domain (often time or position along a line) to the frequency domain. The input samples are complex numbers (in practice, usually real numbers), and t he output coefficients are complex too. The frequencies of the output sinusoids are integer multiples of a fundamental frequency, whose corresponding period is the length of the sampling interval. The combination of sinusoids obtained throu gh the DFT is therefore periodic with that same period. The DFT differs from the discrete-time Fourier transform (DTFT) in that its input and output sequences a re both finite; it is therefore said to be the Fourier analysis of finite-domain (or periodic) discrete-time functions. X_k = \sum_{n=0}^{N-1} x_n \cdot e^{-i 2 \pi k n / N}.

Discrete-time Fourier transform the discrete-time Fourier transform (DTFT) is one of the specific forms of Fouri er analysis that transforms one function into another, which is called the frequ ency domain representation, of the original function (which is often a function in the time-domain). The DTFT requires an input function that is discrete ie sam pling a continuous function, like a person's voice. The DTFT frequency-domain representation is always a periodic function. Since on e period of the function contains all of the unique information, it is sometimes convenient to say that the DTFT is a transform to a "finite" frequency-domain ( the length of one period), rather than to the entire real line. X(\omega) = \sum_{n=-\infty}^{\infty} x[n] \,e^{-i \omega n}. Often the x[n] sequence represents the values (aka samples) of a continuous-time function, x(t), at discrete moments in time: t = nT, where T is the sampling in terval (in seconds), and fs = 1/T is the sampling rate (samples per second). The n the DTFT provides an approximation of the continuous-time Fourier transform: X(f) = \int_{-\infty}^\infty x(t)\cdot e^{- i 2\pi f t} dt.

Fast Fourier transform A fast Fourier transform (FFT) is an algorithm to compute the discrete Fourier t ransform (DFT) and its inverse in a fast (computationally efficient) way. By making use of periodicities in the sines that are multiplied to do the transf orms, the FFT greatly reduces the amount of calculation required. 1. Functionally, the FFT decomposes the set of data to be transformed into a ser ies of smaller data sets to be transformed. 2. Then, it decomposes those smaller sets into even smaller sets. 3. At each stage of processing, the results of the previous stage are combined i n special way. 4. Finally, it calculates the DFT of each small data set.

It turns out that it is possible to take the DFT of the first N/2 points and com bine them in a special way with the DFT of the second N/2 points to produce a si ngle N-point DFT. Each of these N/2-point DFTs can be calculated using smaller D FTs in the same way. These are combined to form N/4 4-point DFTs. The next stage produces N/8 8-point DFTs, and so on, until a single N-point DFT is produced. efficient is the FFT? The DFT takes N^2 operations for N points. Since at any stage the computation re quired to combine smaller DFTs into larger DFTs is proportional to N, and there are log2(N) stages (for radix 2), the total computation is proportional to N * l og2(N). Therefore, the ratio between a DFT computation and an FFT computation fo r the same N is proportional to N / log2(n). In cases where N is small this rati o is not very significant, but when N becomes large, this ratio gets very large. (Every time you double N, the numerator doubles, but the denominator only incre ases by 1.) Are FFTs limited to sizes that are powers of 2? No. The most common and familiar FFTs are "radix 2". However, other radices are sometimes used, which are usually small numbers less than 10. For example, radix -4 is especially attractive because the "twiddle factors" are all 1, -1, j, or j, which can be applied without any multiplications at all. What is an FFT "radix"? The "radix" is the size of an FFT decomposition. In the example above, the r adix was 2. For single-radix FFTs, the transform size must be a power of the rad ix. In the example above, the size was 32, which is 2 to the 5th power. What are "twiddle factors"? "Twiddle factors" are the coefficients used to combine results from a previo us stage to form inputs to the next stage. What is an "in place" FFT? An "in place" FFT is simply an FFT that is calculated entirely inside its or iginal sample memory. In other words, calculating an "in place" FFT does not req uire additional buffer memory (as some FFTs do.) What is "bit reversal"? "Bit reversal" is just what it sounds like: reversing the bits in a binary w ord from left to right. Therefore the MSBs become LSBs and the LSBs become MSBs. But what does that have to do with FFTs? Well, the data ordering required by ra dix-2 FFTs turns out to be in "bit reversed" order, so bit-reversed indexes are used to combine FFT stages. It is possible (but slow) to calculate these bit-rev ersed indices in software; however, bit reversals are trivial when implemented i n hardware. Therefore, almost all DSP processors include a hardware bit-reversal indexing capability (which is one of the things that distinguishes them from ot her microprocessors.) What is "decimation-in-time" versus "decimation-in-frequency"? FFTs can be decomposed using DFTs of even and odd points, which is called a

Decimation-In-Time (DIT) FFT, or they can be decomposed using a first-half/secon d-half approach, which is called a "Decimation-In-Frequency" (DIF) FFT. Generall y, the user does not need to worry which type is being used. How do I implement an FFT? Many good FFT implementations are available in C, Fortran and other language s, and microprocessor manufacturers generally provide free optimized FFT impleme ntations in their processors' assembly code, Therefore, it is not so important t o understand how the FFT really works, as it is to understand how to use it.

Laplace transform The Laplace transform is a generalized Fourier transform. It allows a transform of any system or signal because it is a transform into the complex plane instead of just the j? line like the Fourier transform. The major difference is that th e Laplace transform has a region of convergence for which the transform is valid . This implies that a signal in frequency may have more than one signal in time; the correct time signal for the transform is determined by the region of conver gence. If the region of convergence includes the j? axis, j? can be substituted into the Laplace transform for s and it's the same as the Fourier transform. The Laplace transform is: X(s) = \int^\infty_{0^-} x(t)e^{-s t}\, dt and the inverse Laplace transform, if all the singularities of X(s) are in the l eft half of the complex plane, is: x(t)=\frac{1}{2\pi}\int^\infty_{-\infty} X(s )e^{s t}\, d s

Bode plots Bode plots are plots of magnitude vs. frequency and phase vs. frequency for a sy stem. The magnitude axis is in Decibel (dB). The phase axis is in either degrees or radians. The frequency axes are in a logarithmic scale. These are useful bec ause for sinusoidal inputs, the output is the input multiplied by the value of t he magnitude plot at the frequency and shifted by the value of the phase plot at the frequency.

Radius of convergence The radius of convergence of a power series is the radius of the largest disk in which the series converges. It is either a non-negative real number or 8. When it is positive, the power series converges absolutely and uniformly on compact s ets inside the open disk of radius equal to the radius of convergence.

Z Transform

Z-transform converts a discrete time-domain signal, which is a sequence of real or complex numbers, into a complex frequency-domain representation. It can be considered as a discrete-time equivalent of the Laplace transform The Z Transform has a strong relationship to the DTFT, and is incredibly useful in transforming, analyzing, and manipulating discrete calculus equations.

Stochastic process a stochastic process (pronunciation: /sto?'kst?k/), or sometimes random process ( widely used) is a collection of random variables; this is often used to represen t the evolution of some random value, or system, over time.In the simple case of discrete time, a stochastic process amounts to a sequence of random variables k nown as a time series. Another basic type of a stochastic process is a random fi eld, whose domain is a region of space, in other words, a random function whose arguments are drawn from a range of continuously changing values. One approach t o stochastic processes treats them as functions of one or several deterministic arguments (inputs, in most cases regarded as time) whose values (outputs) are ra ndom variables: non-deterministic (single) quantities which have certain probabi lity distributions.