Is there a way of performing stratified cross validation using xgboost module in python?

Question

I am training and predicting on the same data-set, but I want to perform 10-fold cross-validation and predict on the left out fold and thus predict on the whole data set. How can I do this?

The libraries which I am using are:

from sklearn import cross_validation
import xgboost as xgb

Regenschein · Answer

xgboost comes with an own cv method, see an example here

Dawny33 · Answer

What you are doing is a typical example of k-fold cross validation.

XGBoost is just used for boosting the performance and signifies "distributed gradient boosting".

First, run the cross-validation step:

kfld = sklearn.cross_validation.KFold(labels.size, n_folds=10)

Then, use the train and test indices in kfld for constructing the XGBoost matrix and re-scaling weights by looping over them(the indices).

A very neat implementation has been given as a Kaggle example here.

So, cross validation is not done with the xgboost package, it is done with the cross_validation module of sklearn, and then the gradient boosting is done on the indices of the k-fold validation variable results.

Is there a way of performing stratified cross validation using xgboost module in python?

2 Answers

Add your own answers!

Ask a Question