Is ROCR applied to training data or testing data?

Well, in R if you want to use the ROCR package, you use it on your test data. I think it is crucial to understand the steps involved prior to the plotting of the ROC curve.

1. You divide your data into train data and test data.
2. You do whatever regression on your train data.
3. Now, with the model (step 2) that you have just 'trained' on your train data, you can now use it predict the outcome of your dependent variable based on all the independent variables from your test data.
4. After you have your predicted outcome of the dependent variable, you create a confusion matrix to see how well you have predicted compared to your test data.
5. And, only now you can start to look into the ROC plot and AUC based on your prediction and your test data.

Below is a code for further elaboration, here glmnet is used, where newx is set to be the test data:

training.samples <- df$dependent %>% createDataPartition(p = 0.8, list = FALSE)
train <- df[training.samples, ]
test <- df[-training.samples, ]
x.train <- data.frame(train[, names(train) != "dependent"])
y.train <- train$dependent
x.test <- data.frame(test[, names(test) != "dependent"])
y.test <- test$dependent
model <- glmnet(x.train, y.train, family = "whatever you need")
coef <- coef(model, s = lambda.1se)
predicted <- predict(model, s = lambda.1se, newx = x.test)
t <- 0.4
predict_binary <- ifelse(predicted > t, 1, 0)
CM <- confusionMatrix(as.factor(predict_binary), as.factor(y.test))
pred <- prediction(predict_binary, y.test)
perf <- performance(pred, "tpr", "fpr")
plot(perf)
auc_ROCR <- performance(pred, measure = "whatever you need")
auc_ROCR <- [email protected][[1]]
auc_ROCR