Preface
Applied Stochastic Processes
Preface
Notes
Some Functions for Working with Discrete Time Finite State Markov Chains
Introduction to Markov Chains
Markov Chains: Joint, Conditional, and Marginal Distributions
Stationary Distributions
Steady State Distributions
Discrete Time Markov Chains: Long Run Behavior
Absorbing States and First Step Analysis
Miscellaneous Markov Math
Inference for Discrete Time Markov Chains
Hidden Markov Models
Introduction to Markov Chain Monte Carlo (MCMC) Methods
Metropolis-Hastings Algorithm
Some Applications of MCMC
Exponential Distributions
Poisson Processes
Further Properties of Poisson Processes
Spatial Poisson Processes, and Relatives of Poisson Processes
Introduction to Continuous Time Markov Chains
Continuous Time Markov Chains: Transition Probabilities and Kolmogorov Equations
Stationary Distributions and Long Run Behavior of CTMCs
Birth and Death Chains
Practice Problems
Conditional Probability, Independence, and Conditional Independence
Random Variables: Joint, Conditional, and Marginal Distributions
Introduction to Stochastic Processes
Introduction to Markov Chains
Discrete Time Markov Chains: Joint, Conditional, and Marginal Distributions
Discrete Time Markov Chains: Stationary Distributions
Discrete Time Markov Chains: Steady State Distributions
Discrete Time Markov Chains: Long Run Behavior
Discrete Time Markov Chains: Absorbing States and First Step Analysis
Inference for Discrete Time Markov Chains
Introduction to Markov Chain Monte Carlo (MCMC) Methods
Metropolis-Hastings Algorithm
Some Applications of Markov Chain Monte Carlo (MCMC) Methods
Some Diagnostics for MCMC
Exponential distributions
Poisson Processes
Further Properties of Poisson Processes
Spatial Poisson Processes, and Relatives of Poisson Processes
Introduction to Continuous Time Markov Chains
Continuous Time Markov Chains: Transition Probabilities and Kolmogorov Equations
Stationary Distributions and Long Run Behavior of CTMCs
Birth and Death Chains
Gaussian Distributions and Processes
Brownian Motion
Relatives of Brownian Motion
Applications
Simple Random Walk
Markov Chains
More Markov Chains
Chutes and Ladders and MCMC
Costco Time Markov Chains
Project
References
Table of contents
Preface
Applied Stochastic Processes
Author
Kevin Ross
Published
Mar 14, 2024
Preface
Selected examples and assignments from Cal Poly STAT 545.
Some Functions for Working with Discrete Time Finite State Markov Chains