MATH 396 Lecture Notes - Lecture 8: Random Variable, Random Walk, Poisson Point Process
1
Overview of Markov processes
Stochastic processes
(i) Processes occur as a sequence of events, or outcomes, in time (or space).
(ii) Time (or space) may be discrete or continuous.
(iii) The state of the process at time k is summarized as a random variable Yk.
(iv) The state Yk can be discrete or continuous, but we consider only discrete random variables Yk. (v) The
probabiities of future events can depend on past history. That is the probabilities for Yk may depend on the
outcomes Y0,Y1,....Yk−1.
The Markov property and examples
(i) If P(Yk+1 = y|Yk = yk,Yk−1 = yk−1,...,Y1 = y1,Y0 = y0) = P(Yk+1 = y|Yk = yk) for all (y0,y1,....,yk,y) then the process has a
(first-order) Markov property.
(ii) Example: Branching processes
Yk is the number of individuals at generation k. Each individual generates a number of offspring in the next
generation, each independently with the same distribution.
That is, where the Xk,i are non-negative integer random variables, independent, and all with
the same “family-size” or “offspring” distribution P(X = x), x = 0,1,2,....,
P(Yk+1 = y|Yk,Yk−1,....Y2,Y1,Y0) = P(Yk+1 = y | Yk)
(iii) Example: Random walk
Yk is (discrete integer) position at time k. Yk+1 = Yk + Xk, where the Xk are (integer) random variables, independent
and with the “step-size” distribution, P(X = x|Y = y), x = 0,±1, ± 2, ± 3.....
That is, the step distribution may depend on the current state Yk but not on any other past history, so
P(Yk+1 = y|Yk,Yk−1,....Y2,Y1,Y0) = P(Yk+1 = y | Yk)
Classification of some stochastic processes in MATH/STAT 396
Discrete time
Continuous time
Discrete outcomes
Discrete events
Independent events
Bernoulli process
Poisson Process
sequence of
independent
events occurring randomly
0/1 outcomes
and independently in time
Independent
reproduction
Branching process
Birth and death process
Offspring distribution X
births/deaths indep with some rate
Independent increments
Random walk
Renewal process
Document Summary
That is the probabilities for yk may depend on the. Yk is the number of individuals at generation k. each individual generates a number of offspring in the next generation, each independently with the same distribution. where the xk,i are non-negative integer random variables, independent, and all with. Yk is (discrete integer) position at time k. yk+1 = yk + xk, where the xk are (integer) random variables, independent. That is, the same (cid:498)family-size(cid:499) or (cid:498)offspring(cid:499) distribution p(x = x), x = 0,1,2,, and with the (cid:498)step-size(cid:499) distribution, p(x = x|y = y), x = 0, 1, 2, 3 P(yk+1 = y|yk,yk 1,y2,y1,y0) = p(yk+1 = y | yk) (iii) example: random walk. P(yk+1 = y|yk,yk 1,y2,y1,y0) = p(yk+1 = y | yk) That is, the step distribution may depend on the current state yk but not on any other past history, so. Classification of some stochastic processes in math/stat 396. Independent reproduction sequence of independent events occurring randomly.
