Can we use feature selection and dimensionality reduction together?

Question

I have a dataset having about 10,000s of features. The features have a hierarchy inherent to them. I found an algorithm performing feature engineering, taking the hierarchy of the features into consideration. After the procedure the feature space will be changed and the original features may not exist. This algorithm will reduce the number of features to about 2000 features. As the next step I am planning to use autoencoders(to perform dimensionality reduction) and obtain a latent representation to perform the classification task. The reason I didn't use the original dataset for the autoencoders is because I want to use the information on the hierarchy of the features for my model. Is this a meaningful model? Is it pointless to compress the feature space twice?
Thank you!

Oliver Foster · Accepted Answer

Every time you compress the feature space you are losing some information. The original feature engineering stage you outlined sounds like a meaningful compression & might make sense in the context of your problem. The second compression on the other hand might only serve to lose some information.
I would only perform the second compression if the classifier you are training is struggling resource-wise to train your model after the initial compression. In this case you might want to sacrifice some information to train the model quicker. But this step should only be investigated after you try training your model with only the original compression & are not satisfied with the speed of training.
Good luck!

Can we use feature selection and dimensionality reduction together?

One Answer

Add your own answers!

Ask a Question