Professional Documents
Culture Documents
is a branch of statistics and signal processing that deals with estimating the values of
parameters based on measured/empirical data that has a random component. The
parameters describe an underlying physical setting in such a way that the value of the
parameters affects the distribution of the measured data. An estimator attempts to
approximate the unknown parameters using the measurements.
For example, it is desired to estimate the proportion of a population of voters who will
vote for a particular candidate. That proportion is the unobservable parameter; the
estimate is based on a small random sample of voters.
Or, for example, in radar the goal is to estimate the range of objects (airplanes, boats,
etc.) by analyzing the two-way transit timing of received echoes of transmitted pulses.
Since the reflected pulses are unavoidably embedded in electrical noise, their measured
values are randomly distributed, so that the transit time must be estimated.
Estimation process
The entire purpose of estimation theory is to arrive at an estimator, and preferably an
implementable one that could actually be used. The estimator takes the measured data as
input and produces an estimate of the parameters.
1
• After deciding upon a probabilistic model, it is helpful to find the limitations
placed upon an estimator. This limitation, for example, can be found through the
Cramér–Rao bound.
• Finally, experiments or simulations can be run using the estimator to test its
performance.
After arriving at an estimator, real data might show that the model used to derive the
estimator is incorrect, which may require repeating these steps to find a new estimator. A
non-implementable or infeasible estimator may need to be scrapped and the process
started anew.
Estimators
Commonly-used estimators and estimation methods, and topics related to them:
2
Applications
Numerous fields require the use of estimation theory. Some of these fields include (but
are by no means limited to):
Measured data are likely to be subject to noise or uncertainty and it is through statistical
probability that optimal solutions are sought to extract as much information from the data
as possible.
3
HYPOTHESIS TEST
A statistical hypothesis test is a method of making decisions using data, whether from a
controlled experiment or an observational study (not controlled). In statistics, a result is
called statistically significant if it is unlikely to have occurred by chance alone, according
to a pre-determined threshold probability, the significance level. The phrase "test of
significance" was coined by Ronald Fisher: "Critical tests of this kind may be called tests
of significance, and when such tests are available we may discover whether a second
sample is or is not significantly different from the first."[1]
A result that was found to be statistically significant is also called a positive result;
conversely, a result that is not unlikely under the null hypothesis is called a negative
result or a null result.
The critical region of a hypothesis test is the set of all outcomes which, if they occur, will
lead us to decide that there is a difference. That is, cause the null hypothesis to be
rejected in favor of the alternative hypothesis. The critical region is usually denoted by
the letter C.
4
The following examples should solidify these ideas.
In the start of the procedure, there are two hypotheses H0: "the defendant is not guilty",
and H1: "the defendant is guilty". The first one is called null hypothesis, and is for the
time being accepted. The second one is called alternative (hypothesis). It is the
hypothesis one tries to prove.
The hypothesis of innocence is only rejected when an error is very unlikely, because one
doesn't want to convict an innocent defendant. Such an error is called error of the first
kind (i.e. the conviction of an innocent person), and the occurrence of this error is
controlled to be rare. As a consequence of this asymmetric behaviour, the error of the
second kind (acquitting a person who committed the crime), is often rather large.
5
3. The second step is to consider the statistical assumptions being made about the
sample in doing the test; for example, assumptions about the statistical
independence or about the form of the distributions of the observations. This is
equally important as invalid assumptions will mean that the results of the test are
invalid.
4. Decide which test is appropriate, and stating the relevant test statistic T.
5. Derive the distribution of the test statistic under the null hypothesis from the
assumptions. In standard cases this will be a well-known result. For example the
test statistics may follow a Student's t distribution or a normal distribution.
6. The distribution of the test statistic partitions the possible values of T into those
for which the null-hypothesis is rejected, the so called critical region, and those
for which it is not.
7. Compute from the observations the observed value tobs of the test statistic T.
8. Decide to either fail to reject the null hypothesis or reject it in favor of the
alternative. The decision rule is to reject the null hypothesis H0 if the observed
value tobs is in the critical region, and to accept or "fail to reject" the hypothesis
otherwise.
It is important to note the philosophical difference between accepting the null hypothesis
and simply failing to reject it. The "fail to reject" terminology highlights the fact that the
null hypothesis is assumed to be true from the start of the test; if there is a lack of
evidence against it, it simply continues to be assumed true. The phrase "accept the null
hypothesis" may suggest it has been proved simply because it has not been disproved, a
logical fallacy known as the argument from ignorance. Unless a test with particularly
high power is used, the idea of "accepting" the null hypothesis may be dangerous.
Nonetheless the terminology is prevalent throughout statistics, where its meaning is well
understood.
Alternatively, if the testing procedure forces us to reject the null hypothesis (H-null), we
can accept the alternative hypothesis (H-alt)and we conclude that the research hypothesis
is supported by the data. This fact expresses that our procedure is based on probabilistic
considerations in the sense we accept that using another set could lead us to a different
conclusion.
Definition of terms
The following definitions are mainly based on the exposition in the book by Lehmann
and Romano:
Simple hypothesis
Any hypothesis which specifies the population distribution completely.
Composite hypothesis
Any hypothesis which does not specify the population distribution completely.
Statistical test
A decision function that takes its values in the set of hypotheses.
Region of acceptance
6
The set of values for which we fail to reject the null hypothesis.
Region of rejection / Critical region
The set of values of the test statistic for which the null hypothesis is rejected.
Power of a test (1 − β)
The test's probability of correctly rejecting the null hypothesis. The complement
of the false negative rate, β.
Size / Significance level of a test (α)
For simple hypotheses, this is the test's probability of incorrectly rejecting the null
hypothesis. The false positive rate. For composite hypotheses this is the upper
bound of the probability of rejecting the null hypothesis over all cases covered by
the null hypothesis.
Most powerful test
For a given size or significance level, the test with the greatest power.
Uniformly most powerful test (UMP)
A test with the greatest power for all values of the parameter being tested.
Consistent test
When considering the properties of a test as the sample size grows, a test is said to
be consistent if, for a fixed size of test, the power against any fixed alternative
approaches 1 in the limit.[9]
Unbiased test
For a specific alternative hypothesis, a test is said to be unbiased when the
probability of rejecting the null hypothesis is not less than the significance level
when the alternative is true and is less than or equal to the significance level when
the null hypothesis is true.
Conservative test
A test is conservative if, when constructed for a given nominal significance level,
the true probability of incorrectly rejecting the null hypothesis is never greater
than the nominal level.
Uniformly most powerful unbiased (UMPU)
A test which is UMP in the set of all unbiased tests.
p-value
The probability, assuming the null hypothesis is true, of observing a result at least
as extreme as the test statistic.