Efnisyfirlit

• Cover
• Title
• Copyright
• Dedication
• Preface
• Introduction
• 1. Discrete-time Markov chains
• 1.1 Definition and basic properties
• 1.2 Class structure
• 1.3 Hitting times and absorption probabilities
• 1.4 Strong Markov property
• 1.5 Recurrence and transience
• 1.6 Recurrence and transience of random walks
• 1.7 Invariant distributions
• 1.8 Convergence to equilibrium
• 1.9 Time reversal
• 1.10 Ergodic theorem
• 1.11 Appendix: recurrence relations
• 1.12 Appendix: asymptotics for n!
• 2. Continuous-time Markov chains I
• 2.1 Q-matrices and their exponentials
• 2.2 Continuous-time random processes
• 2.3 Some properties of the exponential distribution
• 2.4 Poisson processes
• 2.5 Birth processes
• 2.6 Jump chain and holding times
• 2.7 Explosion
• 2.8 Forward and backward equations
• 2.9 Non-minimal chains
• 2.10 Appendix: matrix exponentials
• 3. Continuous-time Markov chains II
• 3.1 Basic properties
• 3.2 Class structure
• 3.3 Hitting times and absorption probabilities
• 3.4 Recurrence and transience
• 3.5 Invariant distributions
• 3.6 Convergence to equilibrium
• 3.7 Time reversal
• 3.8 Ergodic theorem
• 4. Further theory
• 4.1 Martingales
• 4.2 Potential theory
• 4.3 Electrical networks
• 4.4 Brownian motion
• 5. Applications
• 5.1 Markov chains in biology
• 5.2 Queues and queueing networks
• 5.3 Markov chains in resource management
• 5.4 Markov decision processes
• 5.5 Markov chain Monte Carlo
• 6. Appendix: probability and measure
• 6.1 Countable sets and countable sums
• 6.2 Basic facts of measure theory
• 6.3 Probability spaces and expectation
• 6.4 Monotone convergence and Fubini’s theorem
• 6.5 Stopping times and the strong Markov property
• 6.6 Uniqueness of probabilities and independence of σ-algebras
• Further reading
• Index