3 views

Uploaded by LibyaFlower

- Test21.pdf
- Intro to Simulation - Ingalls - WSC 2011
- Math 10 Sample Final
- tabel Statistics.docx
- Marketing Research Workshops Second Take Home Assessment 0
- Statistics in a Nutshell.pdf
- Mb0050 Slm Unit10
- Statistics Finals
- p-value
- Hypothesis Testing
- Encyclopedia of Research Design, 3 Volumes (2010) by Neil J. Salkind.pdf
- usm 2016-vesperyino-marketing-inferencias de una muestra-inglés (1).pptx
- Statistics Review 2 Samples and Populations
- 10.pdf
- Confidence Intervals
- Untitled
- Testing of Hypothesis
- 495-1951-1-PB
- Haha
- water-06-01685

You are on page 1of 45

Questions Addressed

What

two or more system configurations?

What is hypothesis testing?

How do common random numbers work?

9-2

Purpose

Purpose:

Approach: discuss a few of many statistical

methods that can be used to compare two or more

system designs.

differences are due to:

Differences in design, or

The random fluctuation inherent in the models.

3

Outline

For two-system

comparisons:

Independent sampling.

Correlated sampling (common random numbers).

For multiple

system comparisons:

Bonferroni approach: confidence-interval

estimation, screening, and selecting the best.

attributable to actual differences in

performance and not to statistical variation.

is required.

9-5

Hypothesis Testing

Used

precision is required.

A hypothesis is first formulated (e.g. that

methods A and B both result in the same throughput)

results of the simulation lead us to reject

the hypothesis.

9-6

fail to reject a hypothesis?

1. Our hypothesis is correct

OR

2. The variance in the observed outcomes

are too high given the number of

replications to be conclusive (run more

replications or use variance reduction techniques).

9-7

Types of Errors

Type

Type II error: Accept False Hypothesis

True

Accept

Reject

False

II

I

9-8

[------|------]

(A)

(B)

(C)

[------|------]

Fail to

Reject H0

Reject H0

[------|------]

Reject H0

U1-U2=0

[------|------] denotes confidence Interval

9-9

maximize the throughput of the system.

We offer two methods based on the Confidence Interval

approach.

We seek to discover if the mean throughput of the system

due to Strategy 1 and Strategy 2 are significantly different.

We begin by estimating the mean performance of the two

proposed strategies by simulating each strategy for 16 days

past the warm-up period.

The simulation experiment was replicated 10 times for

each strategy.

The throughput achieved by each strategy is shown next.

9-10

(A)

Replication

(C)

(B)

Strategy 2

Strategy 1

Throughput Throughput

54.48

56.01

57.36

54.08

54.81

52.14

56.20

53.49

54.83

55.49

57.69

55.00

58.33

54.88

57.19

54.47

56.84

54.93

10

55.29

55.84

mean

Standard Deviation

56.30

1.37

54.63

1.17

Variance

1.89

1.36

9-11

Requires

from each simulated system be

normally distributed and independent

within a population and between

populations.

9-12

9-13

17.5

= T.INV.2T(0.05,17.5) = 2.10

9-14

9-15

9-16

Like

method requires that the observations drawing

from each population be normally distributed

and independent within a population.

However, the paired-t CI method does not

require that the observations between

populations be independent.

9-17

This

Numbers to force a positive correlation between

the two populations in order to reduce the halfwidth.

Finally, like the Welch method, the paired-t CI

method does not require that the population have

equal variances.

Given n observations (n1=n2=n), we pair the

observations from each population (x1 and x2) to

define a new random variable: x(1-2)j = x1j - x2j

9-18

9-19

Paired-t Comparison

(A)

Replication (j)

(B)

Strategy 1

Throughput x1j

(C)

Strategy 2

Throughput

x2j

(D)

Difference (B C)

X(1-2)j = x1j - x2j

54.48

56.01

-1.53

57.36

54.08

3.28

54.81

52.14

2.67

56.20

53.49

2.71

54.83

55.49

-0.66

57.69

55.00

2.69

58.33

54.88

3.45

57.19

54.47

2.72

56.84

54.93

1.91

10

55.29

55.84

-0.55

mean

Standard Dev.

Variance

1.67

1.85

3.42

9-20

9-21

Bonferroni

approach

ANOVA

Factorial

design and

optimization experiments

9-22

9-23

9-24

9-25

9-26

Factorial Design

Tests system response(s) when

multiple factors are being manipulated.

Input

Output measures are the responses

9-27

Define a high and low level setting for each factor.

Try every combination of factor settings.

Run detailed studies for factors that have the

greatest impact.

9-28

Fractional-factorial Design

Strategically "screen out" factors that have little or no

impact on system performance.

On remaining factors, run a full-factorial experiment.

Run detailed studies for factors that have the greatest

impact.

9-29

Variance Reduction

Common

Evaluates each system under the exact same

circumstances.

Helps ensure that observed differences of

system designs are due to the differences in

the designs and not to differences in

experimental conditions.

Antithetic Random Numbers (ARN)

9-30

common random numbers.

CRN Continued

(A)

Replication

1

2

3

4

5

6

7

8

9

10

(B)

Strategy 1

Throughput

79.05

54.96

(C)

Strategy 2

Throughput

75.09

51.09

(D)

Difference (B-C)

3.96

3.87

51.23

49.09

2.14

88.74

56.43

70.42

88.01

53.34

67.54

0.73

3.09

2.88

35.71

34.87

0.84

58.12

57.77

45.08

54.24

55.03

42.55

3.88

2.74

2.53

X Difference = 2.67

s Difference = 1.16

9-31

System Designs

Goal: compare two possible system configurations, e.g.:

two possible ordering policies in a supply-chain system,

two possible scheduling rules in a job shop.

output data.

qi (i = 1, 2) to obtain point and interval estimates for the

difference in mean performance, namely q1 q2.

32

Vehicle-safety inspection example:

The station performs 3 jobs: (1) brake check, (2) headlight check, and (3)

steering check.

Vehicles arrival: Possion with rate = 9.5/hour.

Present system:

Three stalls in parallel (one attendant makes all 3 inspections at each stall).

Service times for the 3 jobs: normally distributed with means 6.5, 6.0 and 5.5

minutes, respectively.

Alternative system:

Each attendant specializes in a single task, each vehicle will pass through

three work stations in series

Mean service times for each job decreases by 10% (5.85, 5.4, and 4.95

minutes).

Performance measure: mean response time per vehicle (total time from

33

Assuming that the estimators Yir are (at least approximately) unbiased:

q1 = E(Y1r ), r = 1, , R1;

q2 = E(Y2r ), r = 1, , R2

If c.i. is totally to the left of 0, strong evidence for the hypothesis that q1 q2 < 0 (q1 < q2 ).

If c.i. is totally to the right of 0, strong evidence for the hypothesis that q1 q2 > 0 (q1 > q2 ).

If c.i. contains 0, no strong statistical evidence that one system is better than the other

If enough additional data were collected (i.e., Ri increased), the c.i. would

most likely shift, and definitely shrink in length, until conclusion of q1 < q2 or

q1 > q2 would be drawn.

34

A two-sided 100(1-)% c.i. for q1 q2 always takes the

form of:

where Y.i is the sample mean performance measure for system i over all replications,

and is the degress of freedom.

two statistical techniques.

Both techniques assume that the basic data Yir are

approximately normally distributed.

We will discuss these two methods next.

35

Statistically significant versus practically significant

Statistical significance: is the observed difference Y.1 Y.2

Practical significance: is the true difference q1 q2 large

Confidence intervals do not answer the question of

difference within a range.

36

[Comparison of 2 systems]

used to simulate the two systems

All observations of simulated system 1 are statistically

independent of all the observations of simulated system 2.

The variance of the sample mean, Y.i , is:

V Y.i i2

V Y.i

,

Ri

Ri

i 1,2

12

R1

22

R2

37

[Comparison of 2 systems]

If it is reasonable to assume that 21 22 (approximately) or if R1 = R2,

a two-sample-t confidence-interval approach can be used:

The point estimate of the mean performance difference is: Y.1 Y.2

The sample variance for system i is:

The pooled estimate of 2 is:

( R1 1) S12 ( R2 1) S 22

S

,

R1 R2 2

2

p

i

1

2

Si

Yri Y.i

Ri 1 r 1

i

1

Yri 2 RiY.i 2

Ri 1 r 1

1

2

Standard error:

1

1

R1 R2

38

[Comparison of 2 systems]

approximate 100(1-)% C.I. for can be computed as:

S12 S 22

R1 R2

/ R1 S 22 / R2

S 2 / R 2 / R 1 S 2 / R

1

1 1

2 2

2

1

/ R

2

, round to an interger

39

[Comparison of 2 systems]

For each replication, the same random numbers are used to simulate

both systems.

For each replication r, the two estimates, Yr1 and Yr2, are correlated.

However, independent streams of random numbers are used on different

reduce variance in the point estimator of Y.1 Y.2 .

12

R

22

R

2 12 1 2

R

12 is positive

Variance of Y.1 Y.2 arising from CRN is less than that of independent

40

[Comparison of 2 systems]

shorter confidence interval for the difference.

S D2

Dr D

R 1 r 1

where Dr Yr1-Yr 2

Standard error:

D Y.1 Y.2 :

1

2

2

Dr RD

R 1 r 1

1

and D

R

D ,

r

r 1

SD

R

41

[Comparison of 2 systems]

random-number generator(s):

The random numbers must be synchronized: each

random number used in one model for some purpose

should be used for the same purpose in the other

model.

e.g., if the ith random number is used to generate a

service time at work station 2 for the 5th arrival in

model 1, the ith random number should be used for

the very same purpose in model 2.

42

To compare K alternative system designs based on some specific

performance measure, qi, of system i , for i = 1, 2, , K.

Procedures are classified as:

Fixed-sample-size procedures: predetermined sample size is used to

draw inferences via hypothesis tests of confidence intervals.

Sequential sampling (multistage): more and more data are collected

until an estimator with a pre-specified precision is achieved or until one

of several alternative hypotheses is selected.

Estimation of each parameter q.

Comparison of each performance measure qi, to control q1.

All pairwise comparisons, qi - qj, for all i not equal to j

Selection of the best qi.

43

Bonferroni Approach

[Multiple Comparisons]

(where all statements are true simultaneously).

Bonferroni inequality:

C

j 1

probability of a false conclusion

The smaller j is, the wider the jth confidence interval will be.

independent sampling or CRN

Major disadvantage: width of each individual interval increases

as the number of comparisons increases.

44

Bonferroni Approach

[Multiple Comparisons]

Practical upper limit: about 10 comparisons

3 possible applications:

Individual c.i.s: Construct a 100(1- j)% c.i. for parameter qi,

where # of comparisons = K.

Comparison to an existing system: Construct a 100(1- j)%

c.i. for parameter qi- q1 (i = 2,3, K), where # of comparisons

= K 1.

All pairwise: For any 2 different system designs, construct a

100(1- j)% c.i. for parameter qi- qj. Hence, total # of

comparisons = K(K 1)/2.

45

- Test21.pdfUploaded bySelvi Tagore
- Intro to Simulation - Ingalls - WSC 2011Uploaded bybasemfallatah
- Math 10 Sample FinalUploaded byngbuck
- tabel Statistics.docxUploaded byOla Dwi Nanda
- Marketing Research Workshops Second Take Home Assessment 0Uploaded byHemu Jain
- Statistics in a Nutshell.pdfUploaded bynanaa02
- Mb0050 Slm Unit10Uploaded byMargabandhu Narasimhan
- Statistics FinalsUploaded byelle
- p-valueUploaded byKamal Anchalia
- Hypothesis TestingUploaded byAshish Kumar Annepu
- Encyclopedia of Research Design, 3 Volumes (2010) by Neil J. Salkind.pdfUploaded bySidney GM
- usm 2016-vesperyino-marketing-inferencias de una muestra-inglés (1).pptxUploaded byVictor Sebastian Flores Oñate
- Statistics Review 2 Samples and PopulationsUploaded byManish Chandra Prabhakar
- 10.pdfUploaded byShahista Shaikh
- Confidence IntervalsUploaded byMuaiena Khatun Runa
- UntitledUploaded byEvan Hernandez
- Testing of HypothesisUploaded byK.Prasanth Kumar
- 495-1951-1-PBUploaded byBRO HAFIZ
- HahaUploaded byapi-3813174
- water-06-01685Uploaded byhaerul84
- Self Assessment ExamUploaded byAbodi
- ~NeedToKnowsUploaded bySierra Minty
- Second Edition - Geography, Economics And Economic Geography PART-1Uploaded byDr Swaamee Aprtemaanandaa Jee
- Exer 1 Analytical BalanceUploaded bySeamus Alaric
- Effectiveness of e–Learning Investigation Model on Students’ Understanding of Classification of Organisms in School BiologyUploaded byAlexander Decker
- CT Causal Stat AnalogyUploaded byHector
- BQT ProjectUploaded byAbdullah221790
- 233767863-Newbold-Chapter-7.pptUploaded byAshutosh
- spi1991.pdfUploaded byaspipol pol
- Investigation Into Machining Characteristics of TrialumaneUploaded byInternational Journal of Research in Engineering and Technology

- 319222728-Key-and-Explanation-for-Cambridge-IELTS-Reading-Tests-ZIM-vn.pdfUploaded byLibyaFlower
- Introduction_methodology_2013.pdfUploaded byLibyaFlower
- manual-PK-TAC-feb102.pdfUploaded byLibyaFlower
- Introduction to Clinical Research for Residents (16.9.14) Hani Tamim (FC1)Uploaded byLibyaFlower
- Introduction to Clinical Research for Residents (16.9.14) Hani Tamim (FC1)Uploaded byLibyaFlower
- 16 Lovell March 2014 (1)Uploaded byRoland Duncombe
- tv_22_2015_6_1447_1456.pdfUploaded byMuhammad Izzamir Firdaus
- fulltext (1)Uploaded byLibyaFlower
- PIIS0169500215001622.pdfUploaded byLibyaFlower
- dining-application-2016.pdfUploaded byLibyaFlower
- Ielts Reading 2016 by Ngoc Bach_part 1Uploaded byNguyễn Yolo
- art%3A10.1007%2Fs10182-013-0221-1.pdfUploaded byLibyaFlower
- art%3A10.1007%2Fs11222-005-3393-z.pdfUploaded byLibyaFlower
- jcph464.pdfUploaded byLibyaFlower
- yhyUploaded byLibyaFlower
- Accurate ARL Calculation for EWMA Control Charts Monitoring Simultaneously Normal Mean and VarianceUploaded byHafiedz Pradana
- Operation NorthwoodsUploaded byandroline
- Operation NorthwoodsUploaded byandroline
- Operation NorthwoodsUploaded byandroline
- Six Sigma TutorialUploaded byarnabmoitra11
- Operations ManagementUploaded byrockon605
- GE Six SigmaUploaded byimran27pk
- svy.pdfUploaded byLibyaFlower
- ewmacharts.pdfUploaded byLibyaFlower
- Categorical-Stata.pdfUploaded byLibyaFlower
- A0250109.pdfUploaded byLibyaFlower
- 7-1.xlsxUploaded byLibyaFlower
- Stanford STATA BasicsUploaded byJason Pan
- Operation WindigoUploaded byFuckencioPerez
- 8-2.xlsxUploaded byLibyaFlower

- Multiple Regression AnalysisUploaded byfollow_your_dreams
- Full Stat TablesUploaded byfusilier256013
- sst4e_tif_10.pdfUploaded byFatima Abubakar
- Quiz 1- PSW135 Game Theory in the Social Sciences.pdfUploaded byFanzil Feroz
- An Interactive Minkowski DiagramUploaded byPnfp Tucuman Trancas Primaria
- Quantum Field TheoryUploaded byTeka Kam
- Ujian EBMUploaded byRizha Zhetira
- Alex Lang NSF GRFP Research ProposalUploaded bySilas Wilkinson
- Advanced Mechanics problemsUploaded byPaul Ryan
- Mission Hospital Case StudyUploaded byAbhishekKumar
- 2018_mes_DianaTaschetto.pdfUploaded byAnonymous s3LTiHpc8
- BQT ProjectUploaded byAbdullah221790
- USEQIP BellInequalities ReschUploaded byChristina Ria Ernawati
- Simpson's Paradox - Stats gone Wrong.docxUploaded bykasula
- Theory X and Theory Y WikiUploaded bysagigajjar
- LECT-4-Hypthesis-Test.pdfUploaded byShah Rukh
- FinQuiz - Item-set Answers, Study Session 3, Reading 9.pdfUploaded bygauravroongta
- Peskin-&-Schroesder-An-Introduction-to-Quantum-Field-Theory-Ï°Ìâ´ð°¸-Ch02.pdfUploaded bychen
- Copula RegressionUploaded byDanna Lesley Cruz Reyes
- Statistical vs ClinicalUploaded byTeng Ker Sheng
- Answer of Revision Eqt271 Sem 1 2015.PDFUploaded byMiss A
- SemUploaded byAyesha Kanwal
- QED and Particle Physics.Uploaded bysid_senadheera
- Zeh h d Basic Concepts and Their InterpretationUploaded byMaria Guryeva
- Ch1 7 SlidesUploaded bypratik bhagat
- Big bangUploaded byRahulKumar
- Chem-01-Atoms ElectronicStructure Lecture NotesUploaded byXimena
- Assign 1Uploaded bydarkmanhi
- STAT 200 Final Exam Fall 2016Uploaded byLiam
- Reading Bohr - Physics and PhilosophyUploaded byfmori50