'Why does loss history go to zero for momentum
I am trying to implement momentum for a linear regression model, I will like to save my plus at each point and plot against the iterations, but the loss decreases in the first few epochs and goes to zero, how do I solve this
''' def compute_cost(self, X, y): loss = float((np.sum((X.dot(self.theta) - y) ** 2) ))/ 2 return loss
def fitmomentum(self, X, y,momentum = 0.9 ):
X = self.add_ones(X.values)
self.theta = np.zeros(X.shape[1])
self.cost_history = np.zeros(self.epoch)
self.momentum = momentum
self.velocity = np.zeros(self.epoch)
self.theta = self.momentum(g, self.velocity)
for i in range(len((X.T @ (X @ self.theta - y)))):
self.velocity[i] = self.momentum * self.velocity[i] + self.lr * ((X.T @ (X @ self.theta - y)))[i]
self.theta[i] += self.velocity[i]
self.cost_history[i] = self.compute_cost(X, y)
return self.theta, self.cost_history '''
This is what I see after printing the loss '''array([4.20078101e+13, 6.06945342e+13, 7.01728125e+17, ..., 0.00000000e+00, 0.00000000e+00, 0.00000000e+00])'''
This is what the graph looks like.. plot of epochs against loss history
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|
