Intro
Preface
Contents
Preliminaries
1 Markov Chains
1.1 Markov Kernels
1.2 Markov Chains
1.3 The Canonical Chain
1.4 Markov and Strong Markov Properties
1.5 Continuous Time: Markov Processes
2 Countable Markov Chains
2.1 Recurrence and Transience
2.1.1 Positive Recurrence
2.1.2 Null Recurrence
2.2 Subsets of Recurrent Sets
2.3 Recurrence and Lyapunov Functions
2.4 Aperiodic Chains
2.5 The Convergence Theorem
2.6 Application to Renewal Theory
2.6.1 Coupling of Renewal Processes
2.7 Convergence Rates for Positive Recurrent Chains

Notes
3 Random Dynamical Systems
3.1 General Definitions
3.2 Representation of Markov Chains by RDS
Notes
4 Invariant and Ergodic Probability Measures
4.1 Weak Convergence of Probability Measures
4.1.1 Tightness and Prohorov's Theorem
A Tightness Criterion
4.2 Invariant Measures
4.2.1 Tightness Criteria for Empirical Occupation Measures
4.3 Excessive Measures
4.4 Ergodic Measures
4.5 Unique Ergodicity
4.5.1 Unique Ergodicity of Random Contractions
4.6 Classical Results from Ergodic Theory
4.6.1 Poincaré, Birkhoff, and Ergodic Decomposition Theorems

6.1.1 Continuous Time: Doeblin Points for Markov Processes
6.2 Random Dynamical Systems
6.3 Random Switching Between Vector Fields
6.3.1 The Weak Bracket Condition
6.4 Piecewise Deterministic Markov Processes
6.4.1 Invariant Measures
6.4.2 The Strong Bracket Condition
6.5 Stochastic Differential Equations
6.5.1 Accessibility
6.5.2 Hörmander Conditions
Notes
7 Harris and Positive Recurrence
7.1 Stability and Positive Recurrence
7.2 Harris Recurrence
7.2.1 Petite Sets and Harris Recurrence
7.3 Recurrence Criteria and Lyapunov Functions

7.4 Subsets of Recurrent Sets
7.5 Petite Sets and Positive Recurrence
7.6 Positive Recurrence for Feller Chains
7.6.1 Application to PDMPs
7.6.2 Application to SDEs
8 Harris Ergodic Theorem
8.1 Total Variation Distance
8.1.1 Coupling
8.2 Harris Convergence Theorems
8.2.1 Geometric Convergence
Aperiodic Small Sets
8.2.2 Continuous Time: Exponential Convergence
8.2.3 Coupling, Splitting, and Polynomial Convergence
8.3 Convergence in Wasserstein Distance
A Monotone Class and Martingales
A.1 Monotone Class Theorem
A.2 Conditional Expectation