How good is a forecast? (Many outputs)

Question

I have several macroeconomic models (eg two DSGEs and a VAR). They each produce forecasts for GDP, inflation and unemployment, and I have data to test them on, going back years.
How can I say which model has the "best" overall performance?
I realise this is vague, so here's more detail:
Assume that model X performs better than model Y on GDP, but worse on inflation.
If I'm just looking at GDP, I could just ask something like "which model has the smallest mean square error?" (predicted GDP - actual GDP, squared)
Similarly, if I only care about inflation, I could do the same thing.
But I actually care about forecast accuracy over all three variables (GDP/inf/unemployment).
Is there a way to assess accuracy over all three variables, without just assigning weights to each MSE value and adding them up?
Thanks for your help!

Herr K. · Answer

The MSE is essentially a squared Euclidean distance between two vectors, say $mathbf y$ and $hat{mathbf y}$, where $mathbf y$ is the actual economic data over $T$ periods and $hat{mathbf{y}}$ the predicted values. A natural extension of this to matrices $mathbf Y=(y_{it})$ and $widehat{mathbf Y}=(hat y_{it})$ where $i=1,dots,n$ and $t=1,dots,T$ ($n$ variables over $T$ periods) would be
begin{equation}
text{MSE}^*=frac{1}{nT}sum_{i=1}^nsum_{t=1}^T(y_{it}-hat y_{it})^2.
end{equation}
But beware of the pitfall. Of course there are other matrix norms available as well.

How good is a forecast? (Many outputs)

One Answer

Add your own answers!

Ask a Question