Originally I planned to spend 2 weeks talking about Markov chains with the goal of learning about Markov Chain Monte Carlo (MCMC) methods used in Bayesian statistics. Unfortunately, we lost one week due to coronavirus, and I think most of you would rather have some time to catch up on old homework. So all of this week's material is optional. If you are interested in spending this week learning a little about Markov Chains, keep reading!
Both our current textbook Probability and Statistics - The Science of Uncertainty, 2nd edition, by Evans and Rosenthal and last semester's textbook Introduction to Probability, 8th edition, by Grinstead & Snell have chapters about Markov chains. Of the two, I think that Grinstead & Snell's chapter is a little easier to read and has better exercises.
Optional. If you want to learn about Markov chains, then
Step 1. Read Section 11.1 in Grinstead & Snell. Notice that they use row vectors instead of column vectors, which is a little weird, but for some reason that is common when people work with Markov chains.
Step 2. Try Exercises 2 - 7 and 13 at the end of the section. I recommend using Octave on the SageCell server for the matrix calculations.
Step 3. Let me know if you have any questions.
On Wednesday and Friday, I'll suggest additional exercises from Sections 11.2 and 11.3 in Grinstead & Snell.
Optional Topic: Absorbing Markov chains.
Step 1. Briefly skim Section 11.2 in Grinstead & Snell.
Step 2. Try Exercises 6, 8, 9, and 12 at the end of the section. You'll probably need to flip back to read parts of Section 11.2 more carefully to figure these out, but I'll be happy to help if you get stuck.
Optional Topic: Irreducible (aka Ergodic) Markov chains.
Step 1. Skim Section 11.3 in Grinstead & Snell.
Step 2. Try Exercises 5, 8, 9 and 12 at the end of the section. Go back and re-read the relevant parts of the section as needed to complete the exercises.