About Hypothesis Testing

TABLE OF CONTENTS
About Hypothesis Testing................................................................................................... 1 What is a HYPOTHESIS TEST?.................................................................................... 1 Hypothesis Testing.............................................................................................................. 1 Hypothesis Testing.......................................................................................................... 1 Steps in Hypothesis Testing................................................................................................ 2 Steps in Hypothesis Testing............................................................................................ 2 Step 1: State the Hypotheses........................................................................................... 2 Step 2: Set the Critical Region........................................................................................ 3 Step 3: Collect Data and Compute Statistics .................................................................. 3 Step 4: Make a Decision ................................................................................................. 4 Why do we reject the null hypothesis rather than accept the alternate hypothesis? ... 4 Directional Testing.............................................................................................................. 4 Directional Testing.......................................................................................................... 4 One-tailed Testing........................................................................................................... 5 Errors in Hypothesis Testing .............................................................................................. 6 Errors in Hypothesis Testing .......................................................................................... 6 Directional Testing.......................................................................................................... 7 A Note on Effect Size ..................................................................................................... 8 Glossary .............................................................................................................................. 9 References......................................................................................................................... 11

About Hypothesis Testing

What is a HYPOTHESIS TEST? In hypothesis testing, researchers use sample data to draw logical conclusions on the results of a research study and to make inferences on a population of interest. Hypothesis tests are one of the most commonly used statistical procedures and they can be performed with a wide range of statistics.

Hypothesis Testing
Hypothesis Testing When researchers conduct studies they want to see if a specific variable of interest will have an effect on a specific population. It is unpractical to test an entire population, as the number of individuals that constitute a population can be quite large, or the study may have effects to which it would be unethical to expose a large number of people (for example, experimental drug research that is done on rats). Hypothesis testing allows researchers to use sample data and to infer the results from the sample to the population. The goal of hypothesis testing is to decide whether the results of a study indicate a real relationship between variables, or if the results simply show the random fluctuation that would be the result of chance. Hypothesis tests are one of the most commonly used statistical procedures. They can be performed with a wide range of statistics (i.e. z-tests, t-tests, correlations, ANOVAs, etc.).

Steps in Hypothesis Testing

Steps in Hypothesis Testing The way that a hypothesis test is applied may differ slightly depending on the type of statistic used; however, every hypothesis test has the same basic procedure. There are four main steps: 1) State the hypotheses. 2) Define the critical regions. 3) Collect the data and compute the statistics. 4) Make a decision. Step 1: State the Hypotheses There are 2 opposing hypotheses that need to be stated. The first is the null hypothesis (H0): Null means nothing. The null hypothesis states that there is no change/relationship/difference as a result of the study. This hypothesis predicts that the independent variable will have no effect on the dependent variable. In other words, the population mean will stay the same. The null hypothesis is what we are trying to disprove. The second is the alternate hypothesis (H1): It predicts that there will be some sort of change in the population mean as a result of the study. The alternate hypothesis predicts that the independent variable will have an effect on the dependent variable.

Typically, the alternate hypothesis does not predict a direction of change, simply that there will be a change. However, an alternate hypothesis can predict a positive or negative change in the population mean. Step 2: Set the Critical Region Setting the criteria for the decision means defining the alpha level and critical region. The critical region is composed of one or two extreme scores that are very unlikely to be obtained if the null hypothesis is true. The result has to be beyond these scores in order to be statistically significant.

The alpha level is a probability value. It defines the critical region and tells us the probability that a result beyond the critical region occurred by chance. The alpha level is often set at = 0.05. This means that there is a 5% chance that a result in the critical region occurred by chance.

Step 3: Collect Data and Compute Statistics This is where the researcher conducts the study, collects the data and manipulates the data with statistical tests. The results from these tests are used to determine if the null hypothesis should be rejected or accepted.

Hypothesis testing can be done with a variety of statistics.

Step 4: Make a Decision There are two possible decisions: Reject the null hypothesis. This means that there was a significant change in the population mean. Although it is never stated this way in formal write-ups, rejecting the null hypothesis is the same as accepting the alternate hypothesis. Fail to reject the null hypothesis. This means that there was no change in the population mean as a result of the manipulation.
Why do we reject the null hypothesis rather than accept the alternate hypothesis?

Experimentation in the social sciences follows the scientific method. One of the tenets of the scientific method is falsifiability, meaning that the results of one study can be rendered false through further research. We reject the null hypothesis for the same reason. It is easier to say that something is false rather than claim that something is universally true.

Directional Testing
Directional Testing Hypothesis tests can be conducted as either one-tailed tests, or as twotailed tests. Usually, hypothesis tests are done with two-tailed tests. This means that a significant result can be present in either of the critical regions of the distribution. H0 = H1

Researchers can also choose to predict a direction of change in the population mean. This is a one-tailed test, meaning that a significant result can only occur in the specified critical region.

One-tailed Testing A positive one-tailed test predicts that there will be an increase in the population mean as a result of the manipulation. H0 H1 >

A negative one-tailed test predicts that there will be a decrease in the population mean as a result of the manipulation. H0 H1 <

Errors in Hypothesis Testing

Errors in Hypothesis Testing There are 2 main errors that can be made in hypothesis testing: Type I error Type II error Type I error (alpha error): This occurs when the null hypothesis is rejected although it is true. In other words, this means that it was concluded that there was a significant effect when really there was not. Making a Type I error does not mean that the researcher was careless in their research. Rather, it indicates that the information from the sample was misleading.

Because each individual is different, there could be differences that exist in the sample that are not as prominent in the population (or vice versa). Type II error (beta error): This occurs when the researcher fails to reject the null hypothesis, even though it is false. In other words, the researcher concludes that there was no significant effect of the manipulation, when there actually was. This occurs when the sample mean is not in the critical region, even though the treatment had an effect. The consequences of a Type II error are not as severe as they are for a Type I error. Directional Testing Consider the following table, which conceptualizes the types of errors and when they are made. True in Population Ho False Reject Ho Type I Fail to reject Ho Power

Conclusion In Experiment

Ho True Correct Type II

Power is the probability that the test will correctly reject the null hypothesis. It refers to the probability that the test will detect a significant treatment effect. Power is affected by 3 factors: Alpha level: reducing the alpha level will reduce the power. Number of tails: A one-tailed test has more power than a twotailed test. Sample size: the larger the sample size the greater the power.

A Note on Effect Size A criticism of hypothesis testing is that you do not know the meaningfulness of a significant result. You know there is an effect, but how big is the effect? This is determined by calculating the effect size with a statistic called Cohens d:

The value of d is a proportion indicating the change in the population mean for which the treatment is responsible. Generally: d < 0.2 small effect size 0.2 d 0.8 medium effect size d > 0.8 large effect size

Glossary Alpha level:

sets the probability of making a Type I error. Usually the alpha level is set at 0.05, meaning that there is a 5% chance that the result occurred by chance. predicts that there is a change in the population mean, as a result of the manipulation. defines the boundaries of a significant result. Any score beyond the critical region is significant. refers to the meaningfulness of a significant result. The larger the effect size the greater the significance. predicts that there is no change in the population mean, as a result of the manipulation. predicts the direction of the change. the probability of correctly rejecting a false null hypothesis and concluding that there was an effect from the treatment. a result is statistically significant if it is such an extreme value that it is unlikely to have occurred by chance. does not predict the direction of change. Occurs when the null hypothesis is rejected even though it is true. A result is said to be significant even though it is not.

Alternate hypothesis:

Critical region:

Effect size:

Null hypothesis:

One-tailed test: Power:

Statistically significant:

Two-tailed test: Type I error:

Type II error:

Occurs when the null hypothesis is retained even though it is false. A result is said to be non-significant even though it is.

10

References
Gravetter, F. J. & Wallnau, L. B. (2004). Statistics for the behavioural sciences. (6th ed.) Belmont: Wadsworth/Thomson Learning. Howell, D. C. (2001). Statistical methods for psychology. (5th ed.). California: Duxbury Thomson Learning.

11