Tweedie Loss for Keras

Question

We are currently using XGBoost model with Tweedie loss for solving a regression problem which works very good, now I wanted to move our model to Keras and experience with neural networks, do anybody know how I can implement Tweedie loss for Keras? I only care about the instance when p=1.5 which gives us the best result in XGBoost.

Thanks

keras loss function regression xgboost

Thanks

odyse · Accepted Answer

I end up implementing something like this:

def tweedieloss(y_true, y_pred):
    p=1.5
    dev = 2 * (K.pow(y_true, 2-p)/((1-p) * (2-p)) -
                   y_true * K.pow(y_pred, 1-p)/(1-p) +
                   K.pow(y_pred, 2-p)/(2-p))
    return K.mean(dev)

which I don't know how right it is, for now it seems to be working.

IVR · Answer

Hey thanks for sharing your solution, I was looking for this too! Quite frankly, I'm surprised that the Tweedie distribution isn't that popular since zero-inflated data is fairly common.

Just wanted to add a few comments on that, I'm submitting an "answer" because there is a strict character limit on the comments:

Your solution is the Tweedie deviance which is the same as $-2LL$, which means that you can safely drop the multiplier 2 and minimize the negative log-likelihood instead (shouldn't affect your results)
Technically there is a minor omission in your formula: in the first ratio instead of $frac{y^{2-p}}{(1-p)(2-p)}$, it should be $frac{max(y,0)^{2-p}}{(1-p)(2-p)}$ although I think in practice you'd be working with non-negative values only, so it shouldn't matter
Lastly, I'd also like to note that this definition is only suitable for variance power values $p notin {0,1,2}$, for those special cases see the aforementioned link on Tweedie deviance

Tweedie Loss for Keras

2 Answers

Add your own answers!

Ask a Question