Cross Validated Asked by user235039 on November 6, 2021
I am building a model for forecasting some number of activations. My data set has a panel structure.
Now, I want to come up with a forecast performance measure to assess the performance of my model and to compare it with other models. In this measure I want to punish too low forecasts differently from too high forecasts (so an asymmetric performance measure). I was thinking about the MAPE, but my data also contains 0 (and small) values, so the MAPE will not work. As an extra I would like to be able to change the degree to which too high predictions are punished differently from too low predictions.
I want the asymmetric measure because too high forecasts are worse than too low forecasts, since too high predictions leads to unused stock while too low stock allows for acquiring new stock which is less severe in my case.
Does anyone have a suggestion which performance measure I could use?
If you are forecasting for inventory control, then you actually want a (high) quantile forecast. That is, you need a quantity that exceeds the actual demand in 90% or 95% of all periods.
Your forecasting tool almost certainly does not aim at or output a quantile forecast, but a forecast of expected demand. Differentially penalizing over-/underforecasts of this expectation forecast will not get you to a good quantile forecast.
I discuss some aspects of this in more detail here. That answer also includes, at the very bottom, a link to possible error measures for quantile forecasts, which you may be interested in. Just make sure you don't apply them to an expectation forecast, which makes no sense.
Answered by Stephan Kolassa on November 6, 2021
Get help from others!
Recent Answers
Recent Questions
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP