Value function iteration

We consider the following minimization problem

subject to

as in 0505.

Here, we have two kinds of costs, x and v. The corresponding Bellman equation is

subject to

We start from V0=0 and v=0.

When j=0

That is

When j=1

FOC w.r.t. v is

That is

Plugging this v to Bellman equation at optimum, we get

That is

We can regard it as

When j=2

FOC w.r.t. v is

That is

Plugging this v to Bellman equation at optimum, we get

That is

We can regard it as

.

.

.

We go on and we find that

when j=n

where

that is not changed at all and it starts from .

All we have to do is find the convergence of in the limit and plug it into policy

function and value function.

Notice that initial value does not have to be 1 but we could start any point.