Data Science Asked by Gaurang Swarge on December 14, 2020
Im trying to implement following gradient descent function in Python for logistic regression:
$∇θ(−logL)=−X^T ?(y−e^{Xθ}?)$
This is my python implementation:
def gradient(X, y, theta):
dtheta = -(np.dot(X.T,y - np.exp(X * theta)))
return dtheta
X
is a dataframe of size: (2458, 31),
y
is a dataframe of size: (2458, 1)
theta
is dataframe of size: (2458,1)
when i pass values to my gradient descent function, it returns a dtheta
parameter with size (31,31) due to which i cannot update my theta to pass it to cost function, i cannot fig out where im going wrong.
any help will be appreciated.
Error i keep getting is: ValueError: operands could not be broadcast together with shapes (2458,1) (31,31)
and this is how im implementing the algorithm:
theta = np.random.uniform(low=-0.1,high=0.1, size=(2458,1))
# Iterate and update theta by using the gradient of the negative log-likelihood
max_iter = 100
learning_rate = 1e-3
for i in range(max_iter):
# Calculate the gradient
dtheta = gradient(X,y,theta)
# Update theta
theta = (theta - learning_rate) * dtheta
# Calculate the value of the log-likelihood
cost = negative_loglikelihood(X,y,theta)
# Print iteration
print("Iteration %d, cost function %.3f" % (i+1,cost))
Check your theta
dimensions.
Most likely, your X
dimensions indicate you have 2458
training samples per iteration with each having 31
features. Hence, your theta
should be a matrix of shape (31, 1)
.
With X
having shape (2458, 31)
and if theta
has shape (31, 1)
, X*theta
will have dimensions (2458, 1)
, same as y
and as expected. Now, y-theta
has the same dimensions as that of y
or theta
. And so does exp(y-theta)
X_T
has shape (31, 2458)
and hence, d_theta = - X_T*exp(y-theta)
will have shape (31, 1)
, same as our initial assumed theta
shape and now, you can subtract d_theta
from theta
.
Answered by Yash Jakhotiya on December 14, 2020
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP