What kind of algorithms can be used as a stacker in stacked generalization?

Question

In stacked generalization, several algorithms (I use some random trees, booster trees, etc.) are first trained and used to make the predictions which are used as input for another algorithm. However, can I use any kind of algorithms, or is there a preference?
P.S.: I often see people using linear models in this case.

enterML · Accepted Answer

There is no preference for stacked generalization. You can use any algorithm whether it be a NN, RF model, XGB model, etc. The only thing that you need to take care of is the fact that the algorithm which you are applying to your model is useful to your data or not. Also, model stacked at level 1 may not be a very good model for level 2. For example, suppose at level 1 you stacked an XGB model with max_depth=12, eta=0.01, colsample_bytree=0.7, but at second level it might happen that the same depth is just an overkill and hence you should discard using that model at second level. So, in short there is no preference for stacked models, the preference is for the data on which you are going to train your model.

Correct answer by enterML on June 11, 2021

What kind of algorithms can be used as a stacker in stacked generalization?

One Answer

Add your own answers!

Ask a Question