Dynamic programming and Bellman Equation

We consider the following maximization problem with log utility function

subject to and as in 0517.

We have the same following two parameters

on probability of p11 from state 1 to state 1 and p12 from state 2 to state 1

and

on probability of p21 from state 1 to state 2 and p22 from state 2 to state 2

where we suppose that they are given and constant. Here, p12=1-p11 and p22=1-p21.

Then, the corresponding Bellman equations are going to be

and

We just guess

and

That is

FOC’s w.r.t. are

(*)

and

(**)

Plugging (*) and (**) into Bellman equation at optimum, we get

They are

Comparing coefficients in each kind, we get

They are

So

We could simplify further as

where they are the same as before.

The rest is

Then

Plugging the latter into the former, we get

We solve out and we can say that the guess is right.

From (*) and (**), we have

and

in terms of F and H. We may simplify them further. Actually, they are

and

where they do not vary even though we have different probabilities.