Dynamic programming and Bellman Equation

[Today’s model cannot be solved by guess and verify.]

We consider the following maximization problem

where

on probability of p11 from state 1 to state 1 and p12 from state 2 to state 1

on probability of p21 from state 1 to state 2 and p22 from state 2 to state 2

Here, p21=1-p11 and p22=1-p12. They are exactly the same as yesterday’s.

However, p11 and p12 are functions of labor. Let us describe them as

p11(lt) and p12(lt).

The corresponding Bellman equations are

and

We guess

and

That is

and

FOC’s w.r.t. are

(*)

and

(**)

They are almost the same as yesterday’s but in terms of probability functions.

On the other hand, FOC’s w.r.t. are

For the first one

(***)

For the second one, in the same way,

(****)

The rest is almost the same as yesterday’s but we cannot get constant labors.

That is the problem. Both E and G in value functions are no longer solvable

in terms of parameters and constants. They depend on by (***) and (****)

that also depend on k. That is a contradiction. We fail to guess and verify.

We could solve as FOC’s as they are but we cannot apply the guess and verify.