Professional Documents
Culture Documents
Instruments
tools researchers use to collect data for research studies (alternatively called tests)
1. Cognitive instruments...
Measure an individuals attainment in academic areas typically used to diagnose strengths and weaknesses
achievement tests
provide information about how well the test takers have learned what they have been taught in school achievement is determined by comparing it to the norm, the performance of a national group of similar students who have taken the same test
aptitude tests
measure the intellect and abilities not normally taught and often are used to predict future performance typically provide an overall score, a verbal score, and a quantitative score
2. Affective instruments...
Measure characteristics of individuals along a number of dimensions and to assess feelings, values, and attitudes toward self, others, and a variety of other activities, institutions, and situations
attitude scales
self-reports of an individuals beliefs, perceptions, or feelings about self, others, and a variety of activities, institutions, and situations frequently use Likert, semantic differential, Thurstone , or Guttman scales
values tests
measure the relative strength of an individuals valuing of theoretical, economic, aesthetic, social, political, and religious values
personality inventories
an individuals self-report measuring how behaviors characteristic of defined personality traits describe that individual
3. Projective instruments...
associational tests participants react to a stimulus such as a picture, inkblot or word onto which they project a description
Selecting an instrument...
1. determine precisely the type of instrument needed 2. identify and locate appropriate instruments 3. compare and analyze instruments 4. select best instrument
Instrument sources
Burros Mental Measurements Yearbook Tests in Print PRO-ED Publications Test Critiques Compendium ETS Test Collection Database ERIC/AE Test Review Locator ERIC/Burros Test Publisher Directory
Types of validity...
1. Content validity 2. Criterion-related validity 3. Construct validity
1. Content validity: the degree to which an instrument measures an intended content area
forms of content validity sampling validity: does the instrument reflect the total content area? item validity: are the items included on the instrument relevant to the measurement of the intended content area?
2. Criterion-related validity: an individual takes two forms of an instrument which are then correlated to discriminate between those individuals who possess a certain characteristic from those who do not
forms of criterion-related validity concurrent validity: the degree to which scores on one test correlate to scores on another test when both tests are administered in the same time frame predictive validity: the degree to which a test can predict how well individual will do in a future situation
3. Construct validity: a series of studies validate that the instrument really measures what it purports to measure
Types of reliability...
1. Stability 2. Equivalence 3. Internal consistency
1. Stability (test-retest): the degree to which two scores on the same instrument are consistent over time
2. Equivalence (equivalent forms): the degree to which identical instruments (except for the actual items included) yield identical scores
3. Internal consistency (split-half reliability with Spearman-Brown correction formula , KuderRichardson and Cronbacks Alpha reliabilities, scorer/rater reliability): the degree to which one instrument yields consistent results
Data the pieces of information researchers collect through instruments to examine a topic or hypothesis
Constructs abstractions of behavioral factors that cannot be observed directly and which researchers invent to explain behavior
Measurement scales...
Qualitative (categorical) 1. nominal variables Quantitative (continuous) 2. ordinal variables 3. interval variables 4. ratio variables
2. ordinal (order): classifies persons or objects and ranks them in terms of the degree to which those persons or objects possess a characteristic of interest
3. interval: ranks, orders, and classifies persons or objects according to equal differences with no true zero point
4. ratio: ranks, orders, classifies persons or objects according to equal differences with a true zero point
Norm reference provides an indication about how one individual performed on an instrument compared to the other students performing on the same instrument
Self reference involves measuring how an individuals performance changes over time
Standard error of measurement an estimate of how often a researcher can expect errors of a given size on an instrument
Mini-Quiz
True or false
True or false the primary source of test information for educational researchers is the Burros Mental Measurements Yearbook
true
True or false similar to a Thurstone scale, a Guttman scale attempts to determine whether an attitude is unidimensional
true
True or false validity requires the collection of evidence to support the desired interpretation
true
True or false researchers should first consider developing an instrument rather than utilizing a published instrument
false
True or false predictive validity is extremely important for instruments that are used to classify or select individuals
true
True or false norm reference and criterion reference are synonymous terms
false
True or false criterion related refers to correlating one instrument with a second instrument; the second instrument is the criterion against with the validity of the second instrument is judged
false
True or false a valid test is always reliable but a reliable test is not always valid
true
True or false it is difficult to state appropriate reliability coefficients because reliability, like validity, is dependent upon the group being tested, i.e., groups with different characteristics will produce different reliabilities
true
True or false content validity is not compromised if the instrument covers topics not taught
false
experimental or causal-comparative
Fill in the blank data collection methods including multiple-choice, true-false, and matching
selection
Fill in the blank data collection methods in which students fill in the blank, provide a short answer, or write an essay
supply
Fill in the blank an instrument administered, scored, and interpreted in the same way no matter where or when it is administered
standardized
Fill in the blank the term that includes the general process of collecting, synthesizing, and interpreting information, whether formal or informal
assessment
Fill in the blank a formal, systematic, usually paper-and-pencil procedure for gathering information about peoples cognitive and affective characteristics
test
Fill in the blank the degree to which individuals seek out or participate in particular activities, objects, and ideas
interests
Fill in the blank also called temperament, the characteristics representing an individuals typical behaviors and describes what individual do in their natural life circumstances
personality
Fill in the blank things individuals feel favorable or unfavorable about; the tendency to accept or reject groups, ideas, or objects
attitudes
Fill in the blank deeply held beliefs about ideas, persons, or objects
values
Fill in the blank requires administering the predictor instruments to a different sample from the same population and developing a new equation
cross-validation
Which type of validity compares the content of the test to the domain being measured
content
Which type of validity correlates scores from one instrument to scores on a criterion measure, either at the same or different time
criterion-related
Which type of validity amasses convergent, divergent, and content-related evidence to determine that the presumed construct is what is being measured
construct
Which type of reliability scores on one instrument are consistent over time
stability (test-retest)
Which type of reliability the extent to which independent scorers or a single scorer over time agree on the scoring of an open-ended instrument
scorer/rater
Which type of reliability scores correlate between similar version of an instrument given at different times
equivalence and stability
Which type of reliability scores correlate between two versions of a test that are intended to be equivalent
equivalence (alternate forms)
Which type of reliability the extent to which items included on an instrument are similar to one another in content
internal consistency
Which type of response scale an individual gives a quantitative rating to a topic where each position on the continuum has an associated score value
semantic differential
Which type of response scale value points are assigned to a participants responses to a series of statements
Likert
Which type of response scale participants select from a list of statements that represent differing points of view from those which participations agree
Thurstone