Value function iteration

We consider the following optimization problem

subject to

where we assume that and .

The corresponding Bellman equation is

We start with and k’=0 as in what we did the day before yesterday.

When j=0

Since and k’=0

On the other hand, FOC w.r.t. l is

When k’=0

We have to solve nonlinear equation for l*.

We then express

in terms of l*. We set it as

When j=1

That is

FOC w.r.t. k’ is

So

Plugging it into Bellman equation at optimum when j=1

On the other hand, FOC w.r.t. l is

that is unchanged but k’ is changing. Plugging k’, we get

We have to solve the nonlinear equation for l* again.

We can express as

in terms of l*. We can set

When j=2

That is

FOC w.r.t. k’ is

Plugging it into Bellman equation at optimum when j=2

On the other hand, FOC w.r.t. l is

Plugging k’, we get

We have to solve the nonlinear equation above for l*.

The left hand side is going to change.

We can express

in terms of l*. We can set

.

.

.

We repeat this and we get

We have to solve the nonlinear equation above for l*.

by induction when j=n.

Since when 0<a<1

Thus

On the other hand,

Therefore

We have to solve this nonlinear equation for l*.

In the same way we get

where we got the same things in the method of guess and verify.

The calculation of E is similar to yesterday’s but using .

When j=0

When j=1

When j=2

When j=3

.

.

.

When j=n+1

Since for 0<a<1 when we consider infinitely many terms

That is

Here, we can regard and it converges so we rewrite it as

Thus

where l* also approaches some constant value.