TransWikia.com

Backpropagation through time for stacked RNNs

Cross Validated Asked by E Fresher on November 21, 2021

I was able to find the partial derivative of the cost function with respects to a single variable without much difficulty. However, this requires propagating backwards through the network for each parameter. Is there a way to do this by propagating backwards through the network once? For example, for a MLP, one could find the partial derivative with respects to the activation levels of neurones by propagation backwards only once, and then finding the partial derivatives of the weights and biases by applying the chain rule. Unfortunately, for a stacked RNN, this proved way less straightforward due to the parameters being the same at each time step. I think it might have something to do with ordered derivatives but can’t seem to find much resources on the topic.

Add your own answers!

Ask a Question

Get help from others!

© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP