Value function iteration

(1) Assume initials such as .

(2) Calculate the next function

(3) Calculate the next function subject to

(4) Repeat the calculation subject to

(5) Stop calculation if V converges or if we find its infinite form.

Let us set

x=k, u=k’, r( )=u( )=ln( ), and

as in the maximization problem

subject to

where

and

as in 0502.

Practically, we have the following steps:

(1) Set current value function and k’ to Bellman equation to calculate the next.

(2) Calculate FOC w.r.t. k’ and solve it for k’.

(3) Plug k’ into Bellman equation at optimum and arrange the terms.

(4) Repeat (1) through (3) until it converges.

We start with and k’=0.

When j=0

Since and k’=0

When j=1

That is

FOC w.r.t. k’ is

So

Plugging it into Bellman equation at optimum when j=1

When j=2

That is

FOC w.r.t. k’ is

Plugging it into Bellman equation at optimum when j=2

.

.

.

We repeat this and we get

and

by induction when j=n.

Since when 0<a<1

Thus

and in the same way we get

where we got the same things in the method of guess and verify.

Here, E’s are some constants we will calculate tomorrow.