First Links in the Markov Chain
Homework for the first week:
Required reading: from page 1 up to and including the statement of Theorem 1.7, page 15.
Optional reading: up to the proof of Lemma 1.9, page 17.
Exercises 1.1–1.43 are optional but highly recommended.
Ask questions in class.
Quiz 1 on Tuesday January 23: 20 minutes in class (at the end of the lecture). The quiz will test your basic understanding of Markov chains and will use working knowledge of probability and linear algebra.
Homework for the second week:
Required reading: Chapter 1 (up to page 28), omitting proofs.
Optional reading: proofs in Chapter 1 (up to page 28).
Exercises are optional but recommended.
Ask questions in class.
No quiz during the third week.
Homework for the third week:
Required reading: Chapter 1 (up to the middle of page 35), omitting proofs.
Optional reading: proofs in Chapter 1 (up to page 35).
Exercises are optional but recommended.
Ask questions in class.
No quiz during the third week.
Homework for the 4th week:
Required reading: Chapter 1 (up to the middle of page 40), omitting proofs.
Optional reading: proofs in Chapter 1 (up to page 41).
Ask questions in class.
Quiz 2 on Tuesday February 6: 20 minutes in class (at the end of the lecture). Main topics: stationary distributions; detailed balanced condition; reversibility; Kolmogorov cycle condition; the period of a state.
Homework for the 5th week:
Required reading: Chapter 1 (up to the middle of page 43), omitting proofs.
Also read in the Chapter Summary (pages 61-63):
Recurrence and transience, Stationary distributions, Convergence theorems.
Optional reading: proofs in Chapter 1 (up to page 43).
Ask questions in class.
Quiz 3 on Tuesday February 20: 20 minutes in class (at the end of the lecture). Main topics: Chapter 1 HW given above
Homework for the 6th week:
Required reading: Chapter 1 omitting proofs.
Optional reading: proofs in Chapter 1.
Ask questions in class.
Homework for the 7th and 8th weeks:
Required reading: Chapter 2, Sections 1, 2, 3 omitting proofs.
Optional reading: proofs in Chapter 2, Sections 1, 2, 3.
Ask questions in class.
Quiz 4 on Tuesday March 5: 20 minutes in class (at the end of the lecture).
material discussed in class (expected exit times, exit probabilities, Poisson processes)
Test on Tuesday March 26
Optional reading: Chapters 3, 4.
Required reading: Chapter 5.
Quiz 5 on Tuesday April 16: 20 minutes in class (at the end of the lecture).
material discussed in class (Martingales in Chapter 5)
Quiz 6 on Tuesday April 23: 20 minutes in class (at the end of the lecture).
material discussed in class (Chapter 6)