Asymptotic normality for nonsmooth objective functions

Question

Assume that $f ({bf x}; theta): mathbb{R}^p times Theta to mathbb{R}$, where ${bf x}$ is the vector of inputs (with some distribution) and $theta$ is the vector of parameters.

Also, assume that $E_{bf x}[f({bf x};theta)]$ is maximized at $theta^*$.

To estimate $theta^*$ I solve the following problem:
$$
widehat{theta} = argmax_{theta} frac{1}{n} sum_{i=1}^n f ({bf x}_i; theta), qquad (1)
$$
where ${bf x}_i$ are i.i.d. samples.

If $f$ is differentiable, under some regularity conditions, it is very well-known that $widehat{theta}$ is a consistent estimator of $theta^*$, and is also asymptotically Normal.

In my research I have encountered a problem where $f$ is discontinuous and non-differentiable (actually it is piece-wise constant), but its expected value $E_{bf x}[f({bf x};theta)]$ is continuous and differentiable.

I have shown that solving (1) gives a consistent estimator for $theta^*$, but cannot prove the asymptotic normality of the estimator.

To me this seems like a general problem which must have been addressed in the literature, but cannot find any reference.

Is there a paper that has addressed this problem?

A.K · Answer

Asymptotic normality results with non smooth objective functions has been addressed in the Econometric literature by Daniels (1961), Huber (1967), Pollard (1985), and Pakes and Pollard (1989).

Theorem.

Suppose that $hat{Q}_n(hat{theta}) ge sup_{thetainTheta}hat{Q}_n(theta) - o_p(n^{-1}), hat{theta}totheta_0$, and (i) $Q_o(theta)$ is maximized on $Theta$ at $theta_o$; (ii) $theta_0$ is an interior point of $Theta$, (iii) $Q_o(theta)$ is twice differentiable at $theta_0$ with nonsingular second derivative $H$; (iv) $sqrt{n}hat{D}to_d N(0,Omega)$; (v) for any $delta_nto0$, $sup_{||theta-theta_n|| le delta_n}|hat{R}_n(theta)/[1+sqrt{n}||theta-theta_0||]to_p 0$. Then $sqrt{n}(theta-theta_0)to_d N(0,H^{-1}Omega H^{-1})$.

Note: For reference you can refer to chapter on Large Sample Estimation and Hypothesis Testing by Newey and McFadden (Handbook of Econometrics).

Asymptotic normality for nonsmooth objective functions

One Answer

Add your own answers!

Ask a Question