Laboratory in Psychology: Writing a Research Protocol

Purpose 1) To serve as a catalyst for specifying, refining and developing your research question. 2) To provide a forum in which your experiment and the steps necessary for completing it can be defined in very specific terms. If, after writing this protocol, you do not know the next steps in running your experiment, you havent been complete enough. 3) To provide a framework for managing the experimental design and execution in a group after writing the proposal, every member of your group should know exactly what needs to happen at each step. 4) To help you understand the relationship between experimental design and research questions, hypotheses and explanations.

What is a Research Protocol

Definition: A succinct description or snapshot of an experiment. Includes what question(s) you are asking, and why, the methods and design of experiment and the hypotheses and predicted results.

Components of a Research Protocol

1) Project Title (1-2 lines) May be similar to (or the same as) the titles we created and proposed in small groups. 2) Research Question (1-2 pages) A precise formulation of the major question being asked. Examples of bad research questions are: what happens when we give subjects X?, how do subjects perceive Y?. Good examples include Do subjects show categorical perception along dimension X?, Does memory for the scenes change in response to Y?. This may be more than a sentence and may require some background before it can be asked. In addition to your specific question you must also address: a) Why ask your question? Why should we ask your question. It hasnt been done before is rarely a good enough reason to invest the time and money in an experiment. Good reasons might include: To assess some broader issue (modularity, domain-specificity, innateness, parallel vs. serial processing) within your experimental context. To determine if findings from two different experimental paradigms yield the same result. To understand the nature of the computation involved in a given perception (e.g. is it processed in serial or parallel, what types of information are used in the computation).

 To relate psychological theory to other disciplines. These might include relations to a) clinical applications, b) neuroscientific findings, c) computer or statistical models, d) linguistic theories. Even these reasons alone are not always justification. Your question must be novel. Moreover, the answer must be not entirely obvious, and it must be interesting in the context of cognitive psychology as well as specific theories of cognition and perception. b) What do we already know about your question? This does not need to be a complete literature review, although you should look at some. Critically, you need to discuss what is known, and why are there gaps in our knowledge. In short, given all the work that has been done, why is your question still important? 3) Methodological overview: The Logic of your design (<1 page) In a very broad sense, what methods will your experiment use? This should cover The nature of the subjects task (e.g. discrimination, identification) The type of stimuli you will use. The outlines of the design (e.g. a familiarization period followed by a testing period). The conditions (e.g. two groups of subjects will be usedmusically trained and untrained, category boundaries will be compared across three conditionsafter a context which is expected to bias subjects towards major chords, after a minor-biasing context and after no context). This section should critically cover the Logic: Why use this task, stimuli and design? Why did you decide to use this particular methodology to answer your question. How do the particular stimuli, design and task answer the research question? Does the task in question actually tap the psychological process you are interested in? Are the stimuli you are using representative of the kind of stimuli subjects may see in the real world? Are they representative of the kind of processing you want to study (for example: do studies using synthetic speech really tap natural language processes)? Why study this question in this domain? What else might justify your methods? (e.g. It was used in prior research successfully, It is better than the task used by X because). What are the right control conditions? In sum: how do the tasks, stimuli and procedures you have selected map onto core cognitive and perceptual processes? How do they allow you to answer your question? 4) Specific Predictions (<1 page) This should describe the broad predictions that you will make in specific terms with reference to the methodology you described above. Hypotheses should be specific and refer to your specific conditions, stimuli and task. Bad predictions might include Musically trained subjects will perform better than non-musically trained or subjects wont be able to perceive the stimuli in condition X. An example of a good prediction might be Overall, discrimination will be better for musically trained subjects than non musically trained subjects, however both groups will show improved discrimination at stimuli adjacent to their category boundary, consistent with categorical perception.

In particular you should describe: Your Hypothesis: what you expect to happen. Alternative Hypotheses: what might happen if the other guy is right. The Im dead hypothesis: what is most likely to happen if the experiment is a failure. (note that none of these correspond to the classic experimental/null hypothesis you learned in statisticswell talk about that).

Moreover you must discuss the implications of these predictions. a) Why are these predictions interesting? What do they mean? For each hypothesis, what would it mean if it was supported by your experiment? What answer to your research question would that imply. Would it provide a satisfactory answer at all? Very important: would such a finding be consistent with alternative explanations or theories? If so, how could you rule out such a theory (or how have you)? 5) Specific Methods (at least 2-3 pages) This section forms the exact recipe for running your experiment. It should be the longest and the most detailedan outsider should be able to construct your experiment using only this protocol. Note that in determining many of these details, you may want to refer to the papers we have read (or other related papers)why make up your own number of continuum steps for a face perception in experiment when we already know how many Beale & Keil used? In particular you must describe in detail: a) Subjects Who are your subjects? How many? Are there particular types of subjects you will have to exclude? Are there particular criteria for inclusion? b) Stimuli This should describe the visual, music, linguistic, text or gustatory stimuli to which you will be exposing your subjects on each trial. What is the nature of the stimuli? How will they be constructed? What are the specific stimuli (e.g. a continuum between Sylvester Stallone and Bob McMurray)? How will they be obtained (e.g. digital photography, recordings)? Are there challenges to constructing the stimuli? What experimental variables describe the stimuli and how many levels do they have? For example, We will use 2 different continua, A-major to A-minor and C-major to C-minor. Each continuum will have 9 steps created by varying the frequency of the middle note as described in table X. How will the stimuli be presented to the stimuli (e.g. over headphones in quiet room, on a normal computer screen)? Are there problems with the stimuli? Uncontrolled factors that you will just have to live with? How does this set of stimuli apply to your research question and hypotheses? Are these the right stimuli? Are they the sorts of things subjects will experience in the real world? Are they really representative of the kinds of processes you want to study? 3

c) Task(s) This describes in detail the specific tasks that subjects will perform. If there are multiple types of trials (e.g. identification and discrimination), you may need to answer these questions for each type). This needs to be super-specific: What are the specific instructions you will give to the subjects (write them out)? When will they be presented? How will they be presented (on paper, on a computer screen)? How big are visual stimuli? How long are auditory stimuli? On each trial what happens? What does the subject do? If multiple stimuli are presented how? If simultaneous, how big are each one and where are they located? If they are separated by time (e.g. in an ABX task), how much time? Is exposure limited? That is, do subjects get a specific period of time in which to view the stimuli, or can they view it as much as they want? What data will be recorded (e.g. subjects identification responses)? How will it be recorded (e.g. pen & paper on a response sheet; automatically by the computer)? What else is recorded, if anything (e.g. musical experience of each subject, gender, reaction time)? Do subjects have unlimited time to make their responses, or are they timelimited? Will reaction time be recorded? If so, why? How are trials controlled (e.g. the experimenter will select a stimulus and give it to the subject to taste; a computer will present the visual stimuli using psyscope)? Does this task tap the natural perceptual/cognitive processes you want to study? d) Trial Structure/Design This describes how the trials will be structured over the course of the experiment. How many repetitions will you have for each of your stimuli? Typically a single repetition of each stimulus may not be enough statistical power to see your effects. McMurray et al, for example, used 24 repetitions at each of the 9 stimulus steps for a total of 216 trials. How many total trials will there be? How will the trials be ordered? Will they be in some specific order? In a random order? How will randomization be achieved? Will trials be grouped in to blocks? How many trials per block? How many blocks? Why did you block the trials (or why not)? Are there limits to what types of trials appear in a block (e.g. each block will consist of 1 repetition of each stimulusstimuli are not repeated within a block; or each block will assess discrimination on a single continuum). What happens in between blocks or trials (if anything)? Here you should include a table indicating the factors, number of levels and the total number of trials this will require? e) Post Experiment What happens immediately after the experiment has been completed. Are subjects debriefed? Are subjects compensated? Is their performance reported to them? How? 6) Results (~2 pages + figures)

This describes what sorts of analyses you will do, what you will compute and what you expect to find (specifically). You may want to create a spreadsheet of sample data to make sure you understand what you are expecting. You know how many trials and what the responses are, so it should be easy to generate this data. You can then analyze it to make sure you are reporting your computations properly and to create your figures. a) Computation What exactly will be computed (e.g. mean percentage correct for each subject for each step in the continuum)? How will computations be used (e.g. subjects percentage correct during familiarization will be used to determine whether their testing performance will be analyzedsubjects failing to reach 90% correct will be excluded, subjects discrimination performance during testing will be compared across the condition X and condition Y.). b) Comparisons / Analyses How will your computations be used to assess your hypotheses? What will be compared to what? It is not necessary to know the specific statistical analyses (e.g. ANOVA, T-test) yet, but if you are familiar enough to report them, even better). c) Expected Results How do each of your hypotheses (experimental, null and alternative) relate to these specific comparisons or analyses. For example, if the mean discrimination of subjects in condition X differs from condition Y that would support the experimental hypothesis because if they do not differ, this would support the null hypothesis. You must include figures illustrating these comparisons (include at least one figure for each hypotheses)what should your data look like if each hypothesis is true? What needs to be computed to generate these figures? 7) Conclusion (1 paragraph) In the last (very brief) section you should sum up What your question is and why it is important. How you propose to answer your question. What the answer might mean (more broadly).

Directions for this assignment

1) Each group must turn in a single protocol with all of your names on it. 2) Each individual must turn in a 1-page response similar to the response papers we wrote after reading our research articles. This is your opportunity to criticize your own groups projects, to talk about ideas you had that may not have been included, or to suggest alternative interpretations to what your group comes up with. 3) DO NOT use the above document as a fill in the blanks or mad-lib protocol. Your protocol should include narrative style text (with different sections) that may refer tables describing stimuli, conditions or possible results. You will lose points if you simply run through the bullet points and answer them one by one. 4) Some of these questions will be relevant to your experiment. Others will not. These questions are not an exhaustive list of detailsDO NOT let this handout substitute for your own thinking. What information would you need to have to create and run an experiment? Your protocol will serve as a recipe for your experiment. If, after writing this protocol, you do not know the next steps in running your experiment, you havent been complete enough. 5) Dont make things too complicatedremember you have to run this experiment in under 3 months. It might be tempting to say that you are going to present your stimuli in a soundproof booth, but where are you going to find one? You might want to have 32 continuum steps to make sure that you can really see the differences, but can subjects complete that many trials? In many cases, it is sufficient to state what the ideal design would be, and what your design was (and what factors such as cost or time limited it). 6) Dont forget to include your figure(s) in your results section. These figures must have labeled axes and datapoints or curves. You must have a figure for each predicted result (although if a single figure illustrates more than one result that is also acceptable). Figures must have detailed captions describing how they illustrate your results, that is, the caption must direct the readers attention to the parts of the figure that illustrate your crucial comparisons. Figures may be handdrawn (neatly) or drawn using powerpoint, photoshop, etc. 7) Dont reinvent the wheel! Good perception experiments have been done most of the 20th centurylook at prior (related work) to help choose the types of stimuli, tasks, and analyses you will use. The papers on reserve in the library (as well as the categorical perception book) were selected because they contain techniques you may want to use. 8) Your PsycInfo project is not due until after your protocols are turned in. However, you may find it extremely helpful to do it concurrently. While this will be discussed in a separate handout, you will likely find that the papers and abstracts you locate contain lots of useful tips for your own experiment.