Solving a HJB with a probability to transit to a new state

Question

I am trying to solve the problem of a firm facing the possibility of a future tax, in continuous time.
The firm maximizes $V(k)=int_{t=0}^{infty}e^{-rt} pi_t dt$ with $pi_t=f(k_t)-i_t$ and $dot{k}=i_t-delta k_t$. There is a probability $rho$ per unit of time to transit to a new state where a tax is imposed, and where the profit becomes $f(k_t)-i_t-tau k_t$. This is a partial equilibrium problem and we assume that $r$, $delta$ and $rho$ are exogenous.
I am trying to solve this problem using Hamiltonian-Jacobi-Bellman (HJB) functions:
$$begin{align*}
rV_1 &= max_{i} {f(k_t)-i_t+rho (V_2-V_1)+dot{V}_1} \
rV_2 &= max_{i} {f(k_t)-i-tau k_{t}+dot{V}_2}
end{align*}$$
I know how to solve the second equation, following the method of Walde 2012 using dynamic programming. First, rewrite $$f(k_t)-i_t-tau k_t + V'_2(k_t)dot{k_t}=f(k_t)-i_t-tau k_t+ V'_2(k_t)(i_t-delta k_t)$$ Then, take the FOC with respect to $i_t$, which yields
$$V'_2(k_t)=1$$
Then, use the envelope condition to find
begin{align*}
rV'_2=f'(k_t)-tau+V''_2(k_t)(i_t-delta k_t)-delta V'_2(k_t)
end{align*}
This can be simplified, using the FOC, as$$begin{align*}
f'(k_t)=r+delta+tau
end{align*}$$
which yields the same solution as a simple Hamiltonian would.
However, I am unsure how to proceed next and how to solve for the solution before the tax is imposed and the uncertainty resolved.
Additionally, if you have some references regarding dynamic control in continuous time, I would be very interested, especially if they treat the case of HJB with additional constraints.
Thank you in advance for your help!
EDIT: for clarification, once we reach the new state where the tax is imposed, there is no possibility to go back to the previous state. The only uncertainty is about when the tax will be imposed, i.e, when the uncertainty will be resolved.

Mr. Fafa · Answer

Following the answer of user28714, I tried the following.
First, substituting for the FOC, I rewrite $V_2$ as
begin{align*}
rV_2 &= f(k_t)-i_t - tau k_t + i_t-delta k_t \
&= f(k_t - tau k_t - delta k_t
end{align*}
Thus, we get
$$ V_2 = frac{1}{r}left(f(k_t) - k_t(tau + delta) right)$$
Substituting in $V_1$, we get
$$ rV_1 = max_{i} left{ f(k_t)-i_t + rholeft(frac{1}{r}left(f(k_t) - k_t(tau + delta) right)-V_1right) + V'_1(i_t-delta k_t)   right}$$
The FOC is unchanged:
$ V'_1=1$, and the envelope condition becomes
begin{align*}
rV'_1 = f'(k_t)+rholeft(frac{1}{r}(f'(k_t)-tau - delta)-V'_1right)+V''_1(i_t-delta k_t) - delta V'_1
end{align*}
Noting that $dot{V'_1} = V''_1 (i_t-delta k_t)$ and substituting using the envelope condition, we find
begin{align*}
dot{V_1}=V'_1(r+delta+rho)-f'(k_t)-frac{rho}{r}(f'(k_t)-tau - delta)
end{align*}
Using $V'_1=1$ and $dot{V'_1}=0$, we get
begin{align*}
f'(k_t)(1+frac{rho}{r})&= r+delta +rho +frac{rho}{r}(tau + delta) \
f'(k_t) &= frac{r}{r+rho}left( r+delta +rho +frac{rho}{r}(tau + delta)  right) \
f'(k_t) &= r + delta + frac{rho }{r+rho}(r+ frac{rho}{r}tau)
end{align*}
Which is not the most elegant result... Could somebody confirm me this result?

user28714 · Answer

I would leave this as a comment but I cant. You are on the right track.

Once you know $V_2(k)$ then you can plug that into to the first hjb and solve.

To solve for $V_2$ you need to find the optimal $i$ as a function of $k$. Then plug $i(k)$ into the 2nd HJB. That will give you a second order ode. Solving that will give you $V_2(k)$ and you go to 1.

Solving a HJB with a probability to transit to a new state

2 Answers

Add your own answers!

Ask a Question