Intangible Measurement

231804-P

Social scientists have great need for the development of valid measures, e.g., of the quantity and quality of education services and of the outcomes of those services. Many researchers are frustrated when existing instruments are not well tailored to the task, since they then cannot expect sensitive, accurate, or valid findings. This workshop presents the theory and practice of classical test theory, the traditional approach. It then provides an overview of "modern" measurement as practiced using item response theory with a focus on Rasch measurement. Rasch analysis provides the social sciences with the kind of measurement that characterizes measurement in the natural sciences. Since item response theory focuses on the items and the persons rather than the test score, the synthesis of quantitative analysis with qualitative issues is experienced in a way that is rare in social science. Ultimately, Rasch measurement can facilitate more efficient, reliable, and valid assessment while improving privacy and convenience to users. The workshop is useful for anyone who wants to understand the role of modern measurement in research.

231804-P

Measurement..
:I N S T R U M E N T A T I O N -The Science of Metrology

Measurement....
: Instrument Usage

Measurement....
: Fatalities

Mohd Saidfudin Masodi; IRCA(Lon.) Lead Assessor

FU N DAM E N TAL
PRINCIPLES OF MEASUREMENT

7:01 AM

PRESENTATION STRUCTURE
Measurement Overview

A Scale Construct Development of Instrument Essentials of Measurement Q & A Session

7:01 AM

Measurement Overview -Any measurement call for an instrument.

7:01 AM

Instrument Properties:
- Construct Validity -Person free - Equal interval - Reliable

7:01 AM

Measurement Overview:
- Q & A Session
: Are you measuring what you want to measure ?

7:01 AM

10

Measurement of Physical Attributes

In physical measurement; a metric system has been devised and concept developed; e.g. Length 1. Instrument: Ruler, Tape measure, digital 2. Unit: inches / feet, meter / km 3. Standard definition: The length of path travelled by light in an interval of 1/299 792 458 seconds Can we name others..?
7:01 AM 11

Measurement Units
Two major units: 1. Fundamental units Length : metre Mass : kilogram Time : second Temperature: kelvin Electricity: Ampere

m kg s oK A

7:01 AM

12

Measurement of Units
cont.: 2. Derived units Area : square metre Volume : cubic meter Density : kg per m3 Velocity : metres per sec Acceleration: metres per sec sq Force : Newton
7:01 AM

m2 m3 kg/m3 m/s m/s2 kg m/s2

13

Measurement in Psychology

Recent research has been done in the area of psychometry and in principle has agreed on the approach and criteria of measurement
( Thorndike et al., 1991)

7:01 AM

14

Psychometry Measurement Overview:

- Q & A Session
: So, how do we measure ability or beauty ?

7:01 AM

15

Measurement of Psychological Attributes

HOW DO WE CONSTRUCT THE INSTRUMENT IN SOCIAL SCIENCE STUDY ?

7:01 AM

16

A PERFORMER

BEAUTIFUL

GLORIOUS

Measurement Construct Currently, there are five (5) fundamental issues

.

1. Sample / Item dependency 2. Use of Raw scores or Observed counts to develop interval measure 3. Response Validity 4. Item Validity 5. Construct Validity
7:01 AM 21

Measurement Overview:
- Q & A Session
: What is a measurement construct ? Discuss.

7:01 AM

22

Measurement of Psychological Construct 1. Sample/Item Dependency

a. Items; as agent of measurement, shall be independent of the person; the sample
object used in the calibration. b. Object measures shall be independent of the particular agents used to obtain them. Have you ever come across a Medical Doctor keeping 10 thermometers in his clinic; one for different gender and size ? Discuss.
7:01 AM 23

Measurement of Psychological Construct 1a. Specific Objectivity

a. Specifivity and objectivity is the cornerstone of all measurement

Test Free / Sample Free Measures

e.g. Speed test for typist. Result independent of number of candidates and stand on its own merit unless you dont know how to use the instrument. Discuss.
7:01 AM 24

Measurement of Psychological Construct 1b. Sample Dependent

a. More Correct Responses; ? therefore an Easy test .

High Achievers ? How does an item dependent on the sample objects; person Low Achievers ?

?
7:01 AM

b. Less Correct Responses; therefore a Hard test .

25

Measurement of Psychological Construct 2. Interval Measure

Linearity is a requirement for an item to be measurable which can be developed by transforming raw scores into interval scale. a. Raw scores are observed counts and are not measures b. Percentages are only numerical data summary but does not give any measure

7:01 AM

26

Measurement of Psychological Construct 2b. Interval Measure

Limitations of Raw Score
a. Its a concrete set of nominal answers to an item. b. Its a count of person responses pointing to knowledge & might be taken as ordinal correct answers. c. The quantitative meaning of an ordinal raw score, however, varies with test length; whilst percentage correct varies with item difficulty. d. Raw score is a description of a bygone event. It does not imply a person ability to take on a given task; item difficulty. e. Neither set of raw score nor % correct are linear.
7:01 AM 27

2a. Equal Interval

7:01 AM

28

Measurement of Psychological Construct 2c. Linearity of Measure

In order to compare persons ability for a given cohort, the measure must be interval data on a linear scale to enable due statistical analysis done.
Excellent 6 V good 5 4 Good 3 Mediocre 2 Poor 1 Poorest 1 2 3 4 5 6

Perception assessment => Ordinal; mathematically = axn Linearised, transformed by ratio. = log (axn) = log a + n logx
Linear regression y = mx + c, n = m; gradient thus, ratio scale

7:01 AM

29

Measurement of Psychological Construct 2d. Linearity of Measure

Numerical series are only orders with unequal separation which expand exponentially but log maintain equal interval. Proof:
Numerical series 1 2 5 10 20 50 100
7:01 AM

log10 0.000 0.301 0.699 1.000 1.302 1.699 2.000

loge 0.000 0.694 1.609 2.303 2.997 3.912 4.606

30

Measurement of Psychological Construct Construct Validity

What is the underlying theory ? 1. Sampling Frame: Sampling Unit 2. Item representativeness: Accuracy 3. Focus: Content Validity 4. Item-dependent: Construct Validity

7:01 AM

31

Measurement of Psychological Construct

Response Validity: Observe the pattern
EASY ITEMS SMART 11111011111111111 11111111111111111 ZONE 1 11111111111111111 11111111111111111 11010111101111011 11111111111111111 POOR 11111111111111101 11111111111111111 11111111111111111 DIFFICULT ITEMS 11111111111100 = 48 11111001001000 = 43

11111111111100100 01100010000000 = 33 ZONE 0 11111111111010100 00110100000000 = 33 10101111101001101 11110111001000100 11011011001000100 01110110101101 = 33 01000000000000 = 27 00000000001000 = 25 ?

7:01 AM

32

Measurement of Psychological Construct

Item Validity : Measurement Direction
EASY ITEMS 11111011111111111 11111111111111111 11111111111111111 11111111111111111 11110111111111111 11111111111111111 11111111111111101 11111111111111111 11111111111111111 11111111111100101 11111111111010100 01111111101001100 10110111011000100 11011011001000100 DIFFICULT ITEMS 111111111111100 = 48 11011001001000 = 42 01000010000000 = 33 10110100000000 = 33 00110100101100 = 33 01000000000100 = 27 00000000000100 = 25

BI-DIRECTIONAL ITEM
7:01 AM 33

CONCLUSION
A valid measurement in a psychological construct MUST meet five (5) criteria:
1. Linear Scale: AMOUNT 2. Reasonable numerical values: ACCURACY 3. Empirical Coherence: Response, Item and Construct VALIDITY 4. Incorporate parameter separation: REPLICABLE 5. Overcome missing data: PREDICTIVE
(Wright & Mok, 2004)
7:01 AM 34

Measurement Overview:
- Q & A Session
: Are you measuring what you really want to measure ?

7:01 AM

35