First Links in the Markov Chain

**Homework for the first week:**

Required reading: from page 1 up to and including the statement of Theorem 1.7, page 15.

Optional reading: up to the proof of Lemma 1.9, page 17.

Exercises 1.1–1.43 are optional but highly recommended.

Ask questions in class.

**Quiz 1 on Tuesday January 23:** 20 minutes in class (at the end of the lecture). *The quiz will test your basic understanding of Markov chains and will use working knowledge of probability and linear algebra.*

**Homework for the second week:**

Required reading: Chapter 1 (*up to page 28*), omitting proofs.

Optional reading: proofs in Chapter 1 (*up to page 28*).

Exercises are optional but recommended.

Ask questions in class.

*No quiz during the third week*.

**Homework for the third week:**

Required reading: Chapter 1 (*up to the middle of page 35*), omitting proofs.

Optional reading: proofs in Chapter 1 (*up to page 35*).

Exercises are optional but recommended.

Ask questions in class.

*No quiz during the third week*.

**Homework for the 4th week:**

Required reading: Chapter 1 (*up to the middle of page 40*), omitting proofs.

Optional reading: proofs in Chapter 1 (*up to page 41*).

Ask questions in class.

**Quiz 2 on Tuesday February 6:** 20 minutes in class (at the end of the lecture). *Main topics: stationary distributions; detailed balanced condition; reversibility; Kolmogorov cycle condition; the period of a state. *

**Homework for the 5th week:**

Required reading: Chapter 1 (*up to the middle of page 43*), omitting proofs.

Also read in the *Chapter Summary* (pages 61-63):

**Recurrence and transience, Stationary distributions, Convergence theorems**.

Optional reading: proofs in Chapter 1 (*up to page 43*).

Ask questions in class.

**Quiz 3 on Tuesday February 20:** 20 minutes in class (at the end of the lecture). *Main topics: Chapter 1 HW given above *

**Homework for the 6th week:**

Required reading: Chapter 1 omitting proofs.

Optional reading: proofs in Chapter 1.

Ask questions in class.

**Homework for the 7th and 8th weeks:**

Required reading: Chapter 2, Sections 1, 2, 3 omitting proofs.

Optional reading: proofs in Chapter 2, Sections 1, 2, 3.

Ask questions in class.

**Quiz 4 on Tuesday March 5:** 20 minutes in class (at the end of the lecture).

material discussed in class (expected exit times, exit probabilities, Poisson processes)

**Test on Tuesday March 26**

Optional reading: Chapters 3, 4.

Required reading: Chapter 5.

**Quiz 5 on Tuesday April 16:** 20 minutes in class (at the end of the lecture).

material discussed in class (Martingales in Chapter 5)

**Quiz 6 on Tuesday April 23:** 20 minutes in class (at the end of the lecture).

material discussed in class (Chapter 6)