TransWikia.com

Cannot fig out error in my gradient function implementation in python

Data Science Asked by Gaurang Swarge on December 14, 2020

Im trying to implement following gradient descent function in Python for logistic regression:

$∇θ(−logL)=−X^T ?(y−e^{Xθ}?)$

This is my python implementation:

def gradient(X, y, theta):
    dtheta = -(np.dot(X.T,y - np.exp(X * theta)))
    return dtheta

X is a dataframe of size: (2458, 31),
y is a dataframe of size: (2458, 1)
theta is dataframe of size: (2458,1)

when i pass values to my gradient descent function, it returns a dtheta parameter with size (31,31) due to which i cannot update my theta to pass it to cost function, i cannot fig out where im going wrong.
any help will be appreciated.

Error i keep getting is: ValueError: operands could not be broadcast together with shapes (2458,1) (31,31)

and this is how im implementing the algorithm:

theta = np.random.uniform(low=-0.1,high=0.1, size=(2458,1))
# Iterate and update theta by using the gradient of the negative log-likelihood
max_iter = 100
learning_rate = 1e-3
for i in range(max_iter):
    # Calculate the gradient
    dtheta = gradient(X,y,theta)

    # Update theta

    theta = (theta - learning_rate) * dtheta

    # Calculate the value of the log-likelihood
    cost = negative_loglikelihood(X,y,theta)

    # Print iteration
    print("Iteration %d, cost function %.3f" % (i+1,cost))

One Answer

Check your theta dimensions.

Most likely, your X dimensions indicate you have 2458 training samples per iteration with each having 31 features. Hence, your theta should be a matrix of shape (31, 1).

With X having shape (2458, 31) and if theta has shape (31, 1), X*theta will have dimensions (2458, 1), same as y and as expected. Now, y-theta has the same dimensions as that of y or theta. And so does exp(y-theta)

X_T has shape (31, 2458) and hence, d_theta = - X_T*exp(y-theta) will have shape (31, 1), same as our initial assumed theta shape and now, you can subtract d_theta from theta.

Answered by Yash Jakhotiya on December 14, 2020

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP