MSE errors on autoencoder for dim reduction decreases in a weird patteren and I would love some help to dechyper it

I’m training a denoising autoencoder right now to reduce the dimension of a feature vector I designed of dim 58 to a latent space of dim 10, or less hopefully. I’m having a hard time understanding what does the following error decreasing pattern imply :

]1

My intuition tells me the gradient descent is "crawling" toward a valley for a few epochs, once he reached that valley he quickly improve and then crawl again. So if I’m correct, does it mean my learning rate is too low? If I’m wrong I would love some help.

• My final goal is to use Diffusion maps algorithms on a low dim dataset to graph it and find meaningful clusters and patterns.
• I’m also adding 0.05 Gaussian noise to each feature vector during the start of each forward pass.
• While each feature range is a bit different, all of them range between [5,-2.5]
• LATENT_SPACE_DIM=10
• The feature vector is extracted from 50k users rating in the Netflix dataset

Code in pytorch for the AE architture :

Class Denoising_AE(nn.Module):
   def __init__(self):

    super(Denoising_AE, self).__init__()

    # encoder
    self.enc0 = nn.Linear(in_features=58, out_features=45)
    self.enc1 = nn.Linear(in_features=45, out_features=30)
    self.enc2 = nn.Linear(in_features=30, out_features=15)
    self.enc21 = nn.Linear(in_features=15, out_features=LATENT_SPACE_DIM)

    # self.enc6 = nn.Linear(in_features=10, out_features=8)
    # self.enc = nn.Linear(in_features=8, out_features=6)
    #
    #
    # # decoder
    # self.dec = nn.Linear(in_features=6, out_features=8)
    self.dec0 = nn.Linear(in_features=LATENT_SPACE_DIM, out_features=15)
    self.dec1 = nn.Linear(in_features=15, out_features=30)
    self.dec2 = nn.Linear(in_features=30, out_features=45)
    self.dec3 = nn.Linear(in_features=45, out_features=58)