Characteristics of Good Measurement Instruments Measurement tools can be judged on a variety of merits.

These include practical issues as well as technical ones. All instruments have strengths and weaknessesno instrument is perfect for every task. Some of the practical issues that need to be considered include: Cost Availability Training required Ease of administration, scoring, analysis Time and effort required for respondent to complete measure Along with the practical issues, measurement tools (especially surveys, tests and scales) may be judged on the following technical characteristics: 1) Consistency (Reliability): A good instrument will produce consistent scores. An instruments reliability is estimated using a correlation coefficient of one type or another. For purposes of service learning research, the major characteristics of good scales include: Temporal consistencythe ability of an instrument to give accurate scores from one time to another. Also known as test-retest reliability. Coherencethe consistency of items within a test. There are two types of item coherence: split-half reliability assesses the consistency of items in one-half of a scale to the other half. Internal consistency reliability estimates the consistency among all items in the instrument (typically measured using Chronbachs alpha coefficient.) Scoring agreementthe degree to which different observers or raters give consistent scores using the same instrument, rating scale, or rubric. Also called inter-rater reliability.

For more information about reliability please refer to the following: http://www.socialresearchmethods.net/kb/reltypes.php http://www2.chass.ncsu.edu/garson/pa765/reliab.htm

2.) Meaningfulness (Validity): A valid measurement tool does a good job of measuring the concept that it purports to measure. It is important to remember that the validity of an instrument only applies to a specific purpose with a specific group of people. For example, a scale is not considered simply valid or invalidbut it might be considered valid for measuring social responsibility outcomes with college freshmen. There are three main classes of validity, each having several subtypes, described briefly below.

Construct validityThe concept (e.g., intelligence, moral development, content knowledge) that is being measured is sometimes called the construct. Construct validity establishes that the instrument is truly measuring the desired construct. This is the most important form of validity, because it really subsumes all of the other forms of validity. o Convergent validityComparison and correlation of scores on an instrument with other variables or scores that should theoretically be similar. (For example, two measures of verbal intelligence should yield similar scores and therefore be highly correlated.) o Discriminant validityComparison of scores on an instrument with other variables or scores from which it should theoretically differ. (For example, a measure of verbal ability should not be highly correlated with arithmetic skills.) o Factor structureA statistical look (using factor analysis) at the internal consistency of an instrument, usually one that has subscales or multiple parts. The items that are theoretically supposed to be measuring one concept (i.e., a subscale) should correlate highly with each other, but have low correlations with items measuring a theoretically different concept. Content validityEstablishes that the instrument includes items that comprise the relevant content domain. (For example, a test of English grammar might include questions on subject-verb agreement, but should not include items that test algebra skills.) o Face validityA subjective judgment about whether or not on the face of it the tool seems to be measuring what you want it to measure. Criterion-related validityThe instrument behaves the way it should given your theory about the construct. o Concurrent validityComparison of scores on some instrument with current scores on another instrument. If the two instruments are theoretically related in some manner, the scores should reflect the theorized relationship (For example, a measure of verbal intelligence should be highly correlated with a reading achievement test given at the same time.) o Predictive validityComparison of scores on some instrument with some future behavior or future scores on another instrument. The instrument scores should do a reasonable job of predicting the future performance. (For example, SAT scores are a fairly good predictor of future college academic achievement.)

For more specific information about test validity: http://www2.chass.ncsu.edu/garson/pa765/validity.htm http://www.gifted.uconn.edu/siegle/research/Instrument%20Reliability%20and%20Valid ity/Validity.htm

http://en.wikipedia.org/wiki/Validity_(statistics)

For information on the relationship between reliability and validity: http://www.socialresearchmethods.net/kb/relandval.php