Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
data_mining:neural_network:recurrentnn [2018/06/09 00:00] – [Bidirektional RNN] phreazer | data_mining:neural_network:sequences:recurrentnn [2020/05/29 18:23] (current) – [Vanishing gradient] phreazer | ||
---|---|---|---|
Line 77: | Line 77: | ||
Problem: | Problem: | ||
- | Not very good at capturing long-term dependencies (single/ | + | Not very good at capturing long-term dependencies (single/ |
Exploding gradients can happen, but easier to spot => NAN. | Exploding gradients can happen, but easier to spot => NAN. | ||
Line 133: | Line 133: | ||
Take info from sequence on the right | Take info from sequence on the right | ||
+ | |||
+ | Going forward to last unit and back from there; Acyclic graph, two activations per step (forward, backward). | ||
+ | |||
+ | Activation blocks can be GRU or LSTM | ||
+ | |||
+ | ===== Deep RNN ===== | ||
+ | |||
+ |