TransWikia.com

Normalizing zero inflated predictors for multiple regression

Cross Validated Asked by Eugen Cuic on November 14, 2021

Hope I got it right, as this is my first active post 🙂
I was trying to find a solution the whole day for my problem.

I am trying to predict a continuos variable based on 20 different predictors. The predicted variable is an average m’3 per transportation unit across all palets. the 20 different predictors are the ordered volume per department.

The data looks like this:

RAW DATA

You can see the predictors in the columns VS01 – VS20. Lots of them are zero, because not every order buys something from a department.

In order to create a linear regression I was trying to normalize data with different approaches.

  • Log is not working due to the zeros
  • Square Root is giving me kind of good distribution but does not solve the issue with the zeros
  • Yeo Johnson is the same as the Sqare Root try

What I’ve found is that the solutions for zero inflated issues are focusing only on the predicted value, not on the predictors.

So I am trying to ask the community, if you have another approach to try out?

Bellow you can find the 2nd picutre about 2 of the department’s volume from the orders. Almost all of them look like this:

  • Once transformed with Square Root
  • Once transformed with Yeo Johnson
  • Once the original data set, where it is visible that the data is not only skewed also that there is the gap between the zeros and the next bin of values

Transformations

Additional what would be nice to understand a bit more is how to transform such a non linear distribution into something normal distributed

NonLinear

What I would also be open, if someone could suggest a different approach than linear regression. Maybe:

  • A decision tree approach
  • A PCA approach

Thanks a lot

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP