MA 262: Introduction to Stochastic Processes

Instructor: Arvind Ayyer
Office: X-15 (new wing)
Office hours: TBA
Phone number: (2293) 3215
Email: (First name) at iisc dot ac dot in
Class Timings: Tuesdays and Thursdays, 11:30am — 1pm
Classroom: G-21, OPB
To join the course on MS Teams, use the code gsi5m13
Textbooks:
  • Sheldon Ross, Stochastic Processes, Wiley; 2nd edition, 2008.
  • James Norris, Markov chains, Cambridge University Press, 1997
  • Karlin and Taylor, A first course in Stochastic Processes, Academic Press; 2nd edition, 1975.
  • Bhattacharya and Waymire, Stochastic Processes and Applications, Society for Industrial and Applied Mathematics, 2009.
Tutorials: Wed, 5:15-6:15pm, LH-1.
TAs and office hours:

Email ids are in paranthesis and end with at iisc dot ac dot in, and offices are in the maths department.

  • Arghyadeep Chatterjee (arghyadeepc), X30
  • Sourish Maniyar (sourishparag), X21

Course Description

  1. Discrete parameter Markov Chains: Chapman-Kolmogorov equations, Classification of states, Limit Theorems,
    Examples: Random Walks, Gambler's Ruin, Branching processes. Time reversible Markov chains. Simulations and MCMC
  2. Poisson processes, Definitions, and properties: interarrival and waiting time distributions,
    superposition and thinning, Nonhomogeneous Poisson process, Compound Poisson process. Simulation.
  3. Continuous time Markov Chains: Definition, Birth-Death processes, Kolmogorov backward and forward equations,
    Limiting probabilities, Time reversibility. Queueing Theory, Simulation.
  4. Renewal Theory
  5. Brownian Motion

Exams

All exams will be closed book, closed notes, and
no calculators or electronic devices are allowed (no cell/smart phones).
No communication among the students will be tolerated.
There will be no make up exams.

The date for the midterms and final will be announced later.

Grading

Here are the weights for the homework and exams.
All marks will be posted online on Teams.

Tentative Class Plan

Tutorials are marked in green.

week date sections material covered homework and other notes
0 1/1 (N) 1.1 Introduction to Markov chains N: 1.1.1, 1.1.2;
R: 1.1, 1.2, 1.8
1 6/1 (N) 1.1 Chapman-Kolmogorov equation N: 1.1.3, 1.1.5, 1.1.7;
R: 4.1, 4.2
7/1 -

No quiz

 -
8/1 (N) 1.2, 1.3 Communication classes
hitting times		 N: 1.2.1, 1.2.2, 1.3.1;
R: 4.3, 4.5, 4.23, 4.31
2 13/1 4.3 Recurrence and transience Chap. 4:
14/1 -

Quiz 1

 -
15/1 4.4 Long run proportions Chap. 4:
3 20/1 4.4 Stationary distribution Chap. 4:
21/1 -

No quiz

 -
22/1 4.6 Aperiodicity Chap. 4:
4 27/1 4.7 Branching processes Chap. 4:
28/1 -

Quiz 2

 -
29/1 4.8 Reversible Markov chains Chap. 4:
5 3/2 4.9 Markov chain Monte Carlo Chap. 4:
4/2 -

No quiz

 -
5/2 5.1-5.2 Exponential distribution Chap. 5:
6 10/2 5.3 Poisson process Chap. 5:
11/2 -

Quiz 3

 -
12/2 5.3 Interarrival and waiting times Chap. 5:
7 17/2

No class (midterm week)
18/2 -

No quiz

 -
19/2

No class (midterm week)
8 24/2 6.1-6.3 Continuous-time Markov chains Chap 6:
25/2 -

No quiz

 -
26/2 6.4 Kolmogorov's equations Chap 6:
9 3/3 6.5 Limiting probabilities Chap 6:
4/3 -

Quiz 4

 -
5/3 6.6 Time reversibility Chap 6:
10 10/3 6.9 Matrix formulation Chap 6:
11/3 -

No quiz

 -
12/3 8.1-8.2 Introduction to queues Chap. 8:
11 17/3 8.3 Exponential queueing servers Chap. 8:
18/3 -

Quiz 5

 -
19/3 -

Holiday
12 24/3 7.1-7.2 Renewal processes Chap. 7:
25/3 -

No quiz

 -
26/3 7.3 Renewal limit theorems Chap. 7:
13 31/3 -

Holiday
1/4 -

Quiz 6

 -
2/4 10.1 Brownian motion Chap. 10:
14 7/4 10.2 Hitting times Chap. 10:
8/4 -

No quiz

 -
9/4 10.5 Brownian motion with drift Chap. 10:
15 14/4 10.6-10.7 White noise and Gaussian processes Chap. 10:
15/4 -

Quiz 7

 -
19 TBD -

Final Exam