Dynamic programming and Bellman Equation

We consider the following maximization problem

subject to

where both R and Q are negative.

The negative function of minimization is going to be maximization.

Yesterday’s minimization is a special case when R=-1, Q=-1, C=2 and D=1.

We convert it into the following Bellman equation

subject to at time t

We guess for A<0.

Then we get

Taking the derivative with respect to v_t, we get

So

To find A, we plug v_t back into Bellman equation and get

Factoring out , we get

Thus we get

(*)

We cannot solve out A explicitly but we can get A by some nonlinear optimization.

The value function we get is

using the solution of A that is negative.