17 views

Uploaded by Songya Pan

Stochastic Models I (for first-year doctoral students)

- Laying System eBook
- Kauda Game_A Case Study
- Expected Value Example-Rolling Dice
- Ibps Po Syllabus
- GMAT Math - Tough Problems
- Stochastic I CE
- description: tags: bass-hyman
- PolyM1_English
- Math Problem of the Day #408
- Kelly Option Talk 1
- algebra ii syllabus
- Untitled
- Binom_Poisson
- Mat 126 week 4 quiz
- Maths Probability Expectation and Conditional Expectation lec6/8.pdf
- A Level Answer Key for Year 2009 (1)
- se_fe
- C3 Solomon A
- M_1.3_RM
- Identifiability of Biological Systems

You are on page 1of 5

Problem 4.1 Let Dn be the random demand of time period n. Clearly Dn is i.i.d. and independent of all Xk for k < n. Then we can represent Xn + 1 by Xn+1 = max{0, Xn 1[s,) (Xn ) + S 1[0,s) (Xn ) Dn+1 } which depends only on Xn since Dn+1 is independent of all history. Hence {Xn , n 1} is a Markov chain. It is easy to see assuming k = 0 for k < 0, S j if i < s, j > 0 if i < s, j = 0 k=S k Pij = ij if i s, j > 0 if i s, j = 0 k=i k The following three problems (4.2, 4.4, 4.5) needs a fact: P(A B |C ) = P(A|B C )P(B |C ) which requires a proof to use. Try to prove it by yourself.

Problem 4.2 Let S be the state space. First we show that P(Xnk +1 = j |Xn1 = i1 , , Xnk = ik ) = P(Xnk +1 = j |Xnk = ik ) by the following : Let A = {Xnk +1 = j }, B = {Xn1 = i1 , , Xnk = ik } and Bb , b I are elements of {(Xl , l nk , l = n1 , , l = nk ) : Xl S}. P(A|B ) =

bI

bI

= =

bI

bI

= P(A|Xnk = ik )

We consider the mathematical induction on l n m. For l = 1, we just showed. Now assume that the statement is true for all l l and consider l = l + 1: P(Xn = j |Xn1 = i1 , , Xnk = ik ) =

iS

P(Xn = j, Xn1 = i|Xn1 = i1 , , Xnk = ik ) P(Xn = j |Xn1 = i, Xn1 = i1 , , Xnk = ik )P(Xn1 = i|Xn1 = i1 , , Xnk = ik )

iS

= =

iS

P(Xn = j |Xn1 = i)P(Xn1 = i|Xnk = ik ) By l l cases P(Xn = j |Xn1 = i, Xnk = ik )P(Xn1 = i|Xnk = ik )

iS

= =

iS

= P(Xn = j |Xnk = ik ) which completes the proof for l = l + 1 case. Problem 4.3 Simply by Pigeon hole principle which saying that if n pigeons return to their m(< n) home (through hole), then at least one home contains more than one pigeon. Consider any path of states i0 = i, i1 , , in = j such that Pik ,ik+1 > 0. Call this a path from i to j . If j can be reached from i, then there must be a path from i to j . Let i0 . , in be such a path. If all of values i0 , , in are not distinct, then there must be a subpath from i to j having fewer elements (for instance, if i, 1, 2, 4, 1, 3, j is a path, then so is i, 1, 3, j ). Hence, if a path exists, there must be one with all distinct states. Problem 4.4 Let Y be the rst passage time to the state j starting the state i at time 0.

n Pij = P(Xn = j |X0 = i) n

=

k=0 n

k=0 n

= =

k=0 n

n k k Pjj fij k=0

Problem 4.5 (a) The probability that the chain, starting in state i, will be in state j at time n without ever having made a transition into state k . 2

(b) Let Y be the last time leaving the state i before rst reaching to the state j starting the state i at time 0.

n Pij = P(Xn = j |X0 = i) n

=

k=0 n

k=0 n

= =

k=0 n

k P(Xn = j, Y = k |Xk = i)Pii k=0 n

= =

k=0 n

=

k=0

nk k Pij/i Pii

Problem 4.7 (a) Here is an argument: Let x be the expected number of steps required to return to the initial state (the origin). Let y be the expected number of steps to move to the left 2 steps, which is the same as the expected number of steps required to move to the right 2 steps. Note that the expected number of steps required to go to the left 4 steps is clearly 2y , because you rst need to go to the left 2 steps, and from there you need to go to the left 2 steps again. Then, consider what happens in successive pairs of steps: Using symmetry, we get x = 2 + (0 (1/2) + y (1/2) = 2 + y/2 and y = 2 + (0 (1/4) + y (1/2) + (2 y ) (1/4) If we subtract y from both sides, this last equation yields 2=0. Hence there is no nite solution. The quantity y must be innite; a nite value cannot solve the equation. (b) Note that the expected number of returns in 2n steps is the sum of the probabilities of returning in 2k steps for k from 1 to n, each term of which is binomial. Thus, we have

n

E [N2n ] =

k=1

(2k )! (1/2)2k , k !k !

which can be shown to be equal to the given expression by mathematical induction. (c) We say that f (n) g (n) as n if f (n)/g (n) 1 as n . By Stirlings approximation, (2n + 1) so that E [Nn ] Problem 4.8 (a) Pij = 2n/ as n . (2n)! (1/2)2 n 2 n/ , n!n!

j k=i+1 k

j>i

(b) {Ti , i 1} is not a Markov chain - the distribution of Ti does depend on Ri . {(Ri+1 , Ti ), i 1} is a Markov chain. P(Ri+1 = j, Ti = n|Ri = l, Ti1 = m) = j

l l n1

k=l+1 k

k

k=0 n1 k=l+1

= j

k=0

j>l

(c) If Sn = j then the (n + 1)st record occurred at time j . However, knowledge of when these n + 1 records occurred does not yield any information about the set of values {X1 , , Xj }. Hence, the probability that the next record occurs at time k , k > j , is the probability that both max{X1 , , Xj } = max{X1 , , Xk1 } and that Xk = max{X1 , , Xk }. Therefore, we see that {Sn } is a Markov chain with Pjk = j 1 , k1 k k>j .

n Pij = E[number of visits to j |X0 = i] n=1

= E[number of visits to j |ever visit j, X0 = i]fij = (1 + E[number of visits to j |X0 = j ])fij fij = <. 1 fjj since 1 + number of visits to j |X0 = j is geometric with mean 4

1 1fjj .

= 1+

n=1

n Pjj .

Problem 4.12 If we add the irreducibility of P, it is easy to see that = unique) limiting probability.

1 n1

is a (and the

- Laying System eBookUploaded byhorace_hrc
- Kauda Game_A Case StudyUploaded bynavin9849
- Expected Value Example-Rolling DiceUploaded bypazrie
- Ibps Po SyllabusUploaded bysudheer
- GMAT Math - Tough ProblemsUploaded bySabina Ahuja
- Stochastic I CEUploaded byBilly Yates
- description: tags: bass-hymanUploaded byanon-83857
- PolyM1_EnglishUploaded byapi-3773410
- Math Problem of the Day #408Uploaded byMPOTD
- Kelly Option Talk 1Uploaded byJ.c. Mason
- algebra ii syllabusUploaded byapi-323031305
- UntitledUploaded byapi-200605484
- Binom_PoissonUploaded byLiana Putri Maharani
- Mat 126 week 4 quizUploaded byproffbit
- Maths Probability Expectation and Conditional Expectation lec6/8.pdfUploaded byeeshgarg
- A Level Answer Key for Year 2009 (1)Uploaded byJonathan Lee
- se_feUploaded byEduardo
- C3 Solomon AUploaded byJosh Zngr
- M_1.3_RMUploaded byB S Praveen Bsp
- Identifiability of Biological SystemsUploaded byJoan Albiol
- LAB2pUploaded byJavier Medina
- Subject-Topics for CE Boards (Prepare)Uploaded bymickozara
- Concept ListUploaded byAshima Mishra
- TRAN-2017Uploaded byDalipKumar
- ANA201-300Uploaded byAnonymous bmahyY
- FFrateProbs1Uploaded byJustin
- syllabus math 1 specialUploaded byapi-62785081
- Functions, Graphs, Algebra –CAT past questions-finalUploaded byapi-3756296
- Linear Equation With Two VariablesUploaded byJagabandhu Kar
- Math TricksUploaded byMunish kumar

- midtermTwo6711F12solsUploaded bySongya Pan
- midtermOne6711F10solsUploaded bySongya Pan
- midtermOne6711F08solsUploaded bySongya Pan
- midtermOne6711F12solsUploaded bySongya Pan
- Midterm Two 6711 f 10Uploaded bySongya Pan
- Midterm One 6711 f 10Uploaded bytabrupt
- Midterm One 6711 f 08Uploaded bySongya Pan
- Midterm Two 6711 f 12Uploaded bySongya Pan
- Lect 0910Uploaded bySongya Pan
- Midterm Two f 07Uploaded bySongya Pan
- Midterm One 6711 f 12Uploaded bySongya Pan
- Lect 0903Uploaded bySongya Pan
- Lect 0905Uploaded bySongya Pan
- midtermTwoF07solsUploaded bySongya Pan
- midtermTwo6711F10solsUploaded bySongya Pan
- homewk11SolsUploaded bySongya Pan
- homewkLTUploaded bySongya Pan
- homewk13Uploaded bySongya Pan
- homewk10Uploaded bySongya Pan
- homewk12Uploaded bySongya Pan
- homewk10SolsUploaded bySongya Pan
- homewk8Uploaded bySongya Pan
- homewk11Uploaded bySongya Pan
- homewk13SolsUploaded bySongya Pan
- homewk8SolsUploaded bySongya Pan
- homewk7SolsUploaded bySongya Pan
- homewk7Uploaded bySongya Pan
- homewk9Uploaded bySongya Pan
- homewk12SolsUploaded bySongya Pan

- Performance & Reward Management (P&RM)Uploaded byTahseen Zakaria
- Transportation Statistics: entireUploaded byBTS
- The_Struggle_for_Germany.pdfUploaded byTony
- David Regal - Enough With the Tricks AlreadyUploaded bynirmal23
- Cardiac Function TestUploaded byramanrajesh83
- Tetrahedron 2004 60 10575Uploaded byabder
- incoldbloodUploaded byapi-321434139
- Web Technologies NotesUploaded byShri Kishore
- Celon ENTUploaded byAndromed44
- Financial-and-Managerial-Accounting-The-Basis-for-Business-Decisions-18th-Edition-Williams-Test-Bank.pdfUploaded bya257194639
- Tom SawyerUploaded byraswashere
- 6Uploaded byAna Paula Romeira
- Terminal Radar Data DisplayUploaded byEmely Ambay
- 04 n Wave InterfereUploaded byVina Permatasari
- nullUploaded byapi-25916603
- 20130918 PC Jeweller Ltd IER InitiationReport 2Uploaded byHimanshu Jain
- Road Accident Information Management SystemUploaded byPadma Shahi
- QualiQode v. BizFlowUploaded byPatentBlast
- Who Leaves? Teacher Attrition and Student Achievement (2007)Uploaded byThe Research Alliance for NYC Schools
- moline-roberts articleUploaded byapi-244230664
- biorUploaded byKarn Angurala
- Seven Roman Statesmen of the Later Roman RepublicUploaded byAsterflames Cas Ber
- FP007 CM Eng TrabajoUploaded byricardoaudivision
- Power and InfluenceUploaded byManuela Botis
- Valenzuela v. Maricopa, Ariz. Ct. App. (2015)Uploaded byScribd Government Docs
- Andrew Arato - Part IV IntroductionUploaded bylmzds
- MBA Project Dealer Satisfaction RootsUploaded bySelvaKumarMba
- (eBook) - LaTeX - LaTeX for Word Processer UsersUploaded byLuiz Goulart
- ccr 9 syllabus 2016-2017Uploaded byapi-326096867
- 149939-topic-based-vocabulary-list.pdfUploaded byAnonymous xuyRYU3