A Markov chain on {0,1,2,3} has transition probabilities:
p(1,0)=31,p(1,2)=32,p(2,1)=21,p(2,3)=21.
State 0 is absorbing and state 3 is reflecting: p(3,2)=1.
Let T=inf{n≥0:Xn=0}.
(a) Find a function f:{0,1,2,3}→R and a constant c>0 such that Mn=f(Xn∧T)−(n∧T)⋅c is a martingale. Use this to compute E[T∣X0=2].
(b) Find a function g:{0,1,2,3}→R such that Nn=g(Xn∧T)−(n∧T)⋅d is a martingale for an appropriate constant d, and use the optional stopping theorem to compute E[T2∣X0=2]. Hence find Var(T∣X0=2).