TransWikia.com

Why can't do ridge regression with one predictor?

Cross Validated Asked on January 5, 2022

I’m trying to fit a ridge regression model with a single predictor. However, when I try to do so in three different R packages I get the three following errors:

Error in colMeans(X[, -Inter]) : 
  'x' must be an array of at least two dimensions

Error in if (is.null(np) | (np[2] <= 1)) stop("x should be a matrix with 2 or more columns") : 
  argument is of length zero

Error in colMeans(x[, -Inter]) : 
  'x' must be an array of at least two dimensions

The bottom line from these errors is that x needs to have at least 2 dimensions. Why is this necessary for ridge regression? Does this mean that I can’t use ridge regression with a single predictor? Just seems weird I couldn’t use ridge regression to get regularization for something like a t-test.

Here is my code:

library(lmridge)
library(glmnet)
library(ridge)

# data 
set.seet(100)
y <- rnorm(100)
x <- rbinom(100, 1, .5)
z <- rbinom(100, 1, .5)
data <- cbind.data.frame(y, x, z)

# ridge
linearRidge(y ~ x, data = data)

# glmnet
glmnet(data$x, data$y, nlambda = 25, alpha = 0, family = 'gaussian', lambda = .5)

# lmridge
lmridge(y ~ x, data = data, scaling = "sc", K = seq(0, 1, 0.001))

One Answer

StatQuest does ridge with one predictor just fine in his video.

https://youtube.com/watch?v=Q81RR3yKn30

The method is somewhat silly to use in a regression with just one parameter, but I am surprised the common software implementation don’t allow it. Perhaps the StatQuest example could make sense in some setting.

But that’s just an issue with the software implementation. You’re still able to write your parameter vector as $hat{beta}_R = (X^TX+lambda I)^{-1}X^Ty$ and do the calculation.

($I$ is the identity matrix; $lambda$ is your ridge regression hyperparameter.)

Another popular software implementation of ridge regression is the sklearn packing in Python. Perhaps give that a whirl.

Answered by Dave on January 5, 2022

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP