You are on page 1of 7

# MEASUREMENT SYSTEM SOFTWARE CALIBRATION FILE

INTRODUCTION

Measurement data are used more often and in more ways than ever before. Analytic
studies are among the most important uses of measurement data because they lead ultimately to
better understanding of processes. The benefit of using a data-based procedure is largely
determined by the quality of the measurement data used. If the data quality is low, the benefit of
the procedure is likely to be low. To ensure that the benefit derived from using measurement data
is great enough to warrant the cost of obtaining it; emphasis must be placed on learning how the
measurement system interacts with its environment so that only data of acceptable quality are
generated.
PURPOSE

The measurement system is the entire process of operations, procedures, gauges and other
equipment, software, and personnel used to assign a number to the characteristic being measured
The purpose of this calibration file is to provide a means to assure the quality of the output
provided by a software program used in the measurement system.
INSTRUCTIONS
The data included on the following worksheets is a standard set of validation data
developed to calibrate the output of measurement system software program(s) for both gauge
R&R and capability indices (Cp/Cpk and Pp/Ppk). Contact your STA site engineer if your

1 of 7

## Required Outputs from Statistical Sofware

Gauge R&R as a percent of tolerance using the ANOVA Method (See
Measurement System Analysis Third Edition)

8.23

## Gauge R&R as a percent of study variation using the ANOVA Method

(See Measurement System Analysis Third Edition)

9.74

## Gauge R&R as a percent of tolerance using the Average and Range

Method (See Measurement System Analysis Third Edition)

8.27

## Gauge R&R as a percent of study variation using the Average and

Range Method (See Measurement System Analysis Third Edition)
Number of distinct categories

10.28
14

## Input Data for Statistical Software

Study Variation (Number of Standard Deviations)
Part specification
Process Tolerance
Measurement
Operator Part
3.63957 Operator1
3.93548 Operator1
3.84455 Operator1
4.1651 Operator1
4.28118 Operator1
3.43333 Operator1
3.80442 Operator1
3.93066 Operator1
4.14554 Operator1
4.22784 Operator1
3.58826 Operator2
3.91847 Operator2
3.85039 Operator2
4.16154 Operator2
4.22682 Operator2
3.40564 Operator2
3.80264 Operator2
3.86563 Operator2
4.14732 Operator2
4.22073 Operator2
3.57734 Operator3
3.88087 Operator3
3.85293 Operator3
4.17678 Operator3
4.26467 Operator3
3.43282 Operator3
3.81052 Operator3
3.85141 Operator3
4.14072 Operator3
4.22352 Operator3
3.57531 Operator1
3.93015 Operator1
3.88189 Operator1
4.22454 Operator1
4.30886 Operator1
3.44552 Operator1
3.84481 Operator1
Ford STA 15 August 2004

6
4.0 1.0
2.0

1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
2 of 7

3.90602
4.15316
4.21844
3.62865
3.90653
3.84887
4.20828
4.23444
3.39446
3.80366
3.84912
4.09804
4.2187
3.58268
3.87173
3.78054
4.16129
4.23876
3.37973
3.79959
3.8857
4.15189
4.21133
3.61748
3.89916
3.79324
4.18314
4.25248
3.36779
3.78816
3.89789
4.19837
4.24714
3.63094
3.94514
3.8509
4.23495
4.27788
3.39599
3.8001
3.87503
4.1237
4.21311
3.62865
3.87351
3.7968
4.1745
4.19888
3.3998
3.78664
3.85217
4.12522
4.22454

Operator1
Operator1
Operator1
Operator2
Operator2
Operator2
Operator2
Operator2
Operator2
Operator2
Operator2
Operator2
Operator2
Operator3
Operator3
Operator3
Operator3
Operator3
Operator3
Operator3
Operator3
Operator3
Operator3
Operator1
Operator1
Operator1
Operator1
Operator1
Operator1
Operator1
Operator1
Operator1
Operator1
Operator2
Operator2
Operator2
Operator2
Operator2
Operator2
Operator2
Operator2
Operator2
Operator2
Operator3
Operator3
Operator3
Operator3
Operator3
Operator3
Operator3
Operator3
Operator3
Operator3

8
9
10
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10

3 of 7

## Required Outputs from Statistical Sofware

AIAG Method (defined on p.80 of the SPC manual 1st edition)
Cpk
Cp
Ppk
Pp

## using R bar divided by d2 to estimate "within"

sigma
using sample standard deviation to estimate
"overall" sigma

1.55
1.67
0.98
1.06

Cpk
Cp
Cpk
Cp
Ppk
Pp

## using "pooled" standard deviation divided by c4

(unbiasing constant) to estimate "within" sigma
using "pooled" standard deviation to estimate
"within" sigma
using sample standard deviation divided by c4
(unbiasing constant) to estimate "overall" sigma

1.60
1.73
1.61
1.73
0.98
1.05

d2 and c4 are constants that depend (respectively) on the number of data points in subgroups and
on the number of data points in the sample.
Values of d2 and c4 can be found in Appendix E of the AIAG SPC manual 1st edition.
d2 is the constant used to estimate population standard deviation from the range of data values
within each sub-group.
c4 is the constant used to correct standard deviation estimate errors that result from using a low
number of data points.

## pooled std dev = s p=

( x ij
xi )
i

( ni 1 )
i

where i = the subgroup number and j = sample number within the subgroup

## Input Data for Statistical Software

Subgroup Size
Upper Specification Limit
Lower Specification Limit
Measurement

Subgroup
37.2
36.9
38.1
37.5
37.7
37.8
37.9
38.2
36.8

## Ford STA 25 August 2004

5
39.0 Part specification: 37.0 2.0
35.0

1
1
1
1
1
2
2
2
2

4 of 7

37.8
37.1
38.2
37.2
37.5
37.7
37.8
38.1
37.2
37.3
37.5
36.2
35.8
35.8
35.5
36.0
36.5
35.9
35.9
36.4
36.4
36.6
36.1
36.4
36.9
36.0
36.3
36.8
36.5
36.7
36.3
36.5
36.9
36.8
36.7
37.5
36.6
37.2
37.1
37.4
37.3
36.2
37.2
36.1
36.3
36.2
36.9
36.8
36.7
36.6
36.3
37.1

2
3
3
3
3
3
4
4
4
4
4
5
5
5
5
5
6
6
6
6
6
7
7
7
7
7
8
8
8
8
8
9
9
9
9
9
10
10
10
10
10
11
11
11
11
11
12
12
12
12
12
13

5 of 7

37.9
38.1
37.6
37.5
37.6
38.2
38.1
37.8
37.2
36.8
37.5
37.1
37.4
36.5
37.3
37.1
36.7
37.5
37.6
36.1
37.2
37.1
36.9
36.2
36.5
36.7
36.8
36.9
36.7
36.3
36.2
36.5
36.5
35.8
35.8
36.5
36.4
36.0
36.0
37.1
37.9
36.5
36.9
36.9
36.8
37.2
37.3
37.4
36.6
35.9
36.1
37.0

13
13
13
13
14
14
14
14
14
15
15
15
15
15
16
16
16
16
16
17
17
17
17
17
18
18
18
18
18
19
19
19
19
19
20
20
20
20
20
21
21
21
21
21
22
22
22
22
22
23
23
23

6 of 7

36.6
36.1
36.3
37.4
36.6
36.3
36.0
36.1
37.1
36.9
36.8
36.7
36.8
36.2
36.6
36.8
37.1

23
23
24
24
24
24
24
25
25
25
25
25
26
26
26
26
26

7 of 7