How is the hidden state of a GRU initialized
Data Science Asked by Batman on December 22, 2020
This is a GRU. Now, what will be the value of $h_t$, at $t$=$0$. That is, what will be the value of the hidden state at just the starting?
One Answer
If you think intuitively what h_t (that is h_t-1) shoud represent is, very freely speaking, amount of rembering from the previous steps. Since you are just starting, and want to remember everything, very intuitive thing is to set the value at 1. Than you can see in the equations that the mutiplication with h_t-1 makes no difference in the beginning.
Answered by Noah Weber on December 22, 2020
Add your own answers!
Ask a Question
Get help from others!
Recent Answers
- Jon Church on Why fry rice before boiling?
- haakon.io on Why fry rice before boiling?
- Peter Machado on Why fry rice before boiling?
- Lex on Does Google Analytics track 404 page responses as valid page views?
- Joshua Engel on Why fry rice before boiling?
Recent Questions
- How can I transform graph image into a tikzpicture LaTeX code?
- How Do I Get The Ifruit App Off Of Gta 5 / Grand Theft Auto 5
- Iv’e designed a space elevator using a series of lasers. do you know anybody i could submit the designs too that could manufacture the concept and put it to use
- Need help finding a book. Female OP protagonist, magic
- Why is the WWF pending games (“Your turn”) area replaced w/ a column of “Bonus & Reward”gift boxes?