Should you use the same algorithm in your feature selection as your model

Question

My question is should you use the same algorithm in feature selection as your model?
If I'm using a KNN model for classification should I also use a KNN algo when running feature selection? Or lightGBM algo for feature selection and lightGBM model?
Or is there a best algo for feature selection that you can use those features with any model?
I know there are several different feature engineering techniques I just don't know which techniques should be used with different machine learning algorithms.

Brian Spiering · Answer

Yes - You can use the same the same algorithm for both feature selection and prediction. The most common examples are L1 regression and tree-based algorithms. Those algorithms both find the most important features and predict targets using the same fitting mechanism.
The difference between those algorithms and your examples is that those algorithms do both steps concurrently and your examples are sequential.
There is no single best algorithm for feature selection that can also be used with other models for prediction. Different algorithms will perform better on different datasets.

Should you use the same algorithm in your feature selection as your model

One Answer

Add your own answers!

Ask a Question