Howardfs policy iteration(policy improvement algorithm)
We consider the following maximization problem with log utility function
subject to 
adding labor. We have the same random processes of Q and A.
and
where 
and 
are error terms
with mean zero.
On the background, we assume
when 
The Bellman equation with expectation is
adding one control variable of L.
We firstly set a feasible policy function as
for 
.
Then, we can have
We also have
due to the same kind of process mentioned before.
The Bellman equation between the first two period is
FOC w.r.t. Kf is
for some L.
Thus
On the other hand, FOC
w.r.t. L is
Substituting Kf into
this, we get
Thus
Thus
in terms
of 
.
The detailed calculation of initial value function is as follows:
.
.
.
where
starting from 
and 
when t=1.
Here,
where
starting from 
and 
when t=1.