Howard’s policy iteration(policy improvement algorithm)

We continue to consider the following optimization problem

subject to

where we assume

and

as in 0504. The corresponding Bellman equation is

The initial value function is

We can regard or in the limit.

Then, the initial value function is

where

If we do not like it, we have to calculate the second part one by one by

shifting one period back.

The parameter here appears in value function though it does not in policy functions.