Dynamic programming and Bellman Equation

 

We consider the following optimization problem

 

          subject to

 

where we assume

 

and

 

We convert it into the following Bellman equation

 

      at time t

 

We guess .

 

Then we get

 

or

 

Taking the derivative with respect to ct, we get

 

 

Set

and

for simplicity.

 

Substituting c together with D1 and D2 back into Bellman equation, we get

 

 

since

Comparing the same kinds on both sides,

 

From the second,

From the first, we get the following temporary result

 

 

Here, we can also simplify D1 and D2.

 

We can simplify in the following


So from

we get

 

Value function would be

 

From

we also get

Thus