Dynamic programming and Bellman Equation

We consider the following maximization problem with log utility function

subject to and without labor as in 0522.

We have the same following four parameters

for state1

for state 2

for state 3

for state 4

Then, the corresponding Bellman equations are going to be

where

and each probability lies between 0 and 1.

We just guess

That is

FOC’s w.r.t. are

Since

for the first one

they are

(*)

(**)

(***)

(****)

Plugging (*) and (**) into Bellman equation at optimum, we get

Arranging the first terms on the right hand side, they are

Comparing coefficients for each kind, we get

As we have in the case of two value functions,

The policy functions are now

since .

Thus

for state 1

for state 2

for state 3

for state 4

The rest of constants in value functions are

They can be simplified as

E’s can be solved if we set

and

as in ME=b. That is

.

We have solved out and we can say that the guess is right.