Dynamic programming and Bellman Equation

We consider the following maximization problem

where

for state 1

for state 2

.

.

.

for state n

for the same log utility functions with weights.

The corresponding n Bellman equations are

.

.

.

where

.

.

.

and each probability lies between 0 and 1.

We guess

.

.

.

That is

.

.

.

FOC’s w.r.t. are

.

.

.

They are

.

.

.

On the other hand, FOC’s w.r.t. are

.

.

.

For the first one

In the same way,

.

.

.

Plugging FOC’s of k’ with the solutions of ’ s at optimum, we get

.

.

.

.

.

.

.

.

.

.

.

.

Arranging the first terms each, we have

.

.

.

.

.

.

.

.

.

.

.

.

Comparing coefficients in each kind, we get

.

.

.

They are

From FOC’s with respect to l and solving as l*

Since

So

for state 1

In the same way

for state 2

for state 3

.

.

.

for state n

They are constant. After solving these l*’s, we get

Since

That is

for state 1

In the same way

for state 2

for state 3

.

.

.

for state n

The rest is

.

.

.

.

.

.

.

.

.

.

.

.

The can be simplified as

.

.

.

We can express them as ME=b where

and

That is

We have solved out and we can say that the guess is right.