data_mining:neural_network:deep_neural_nets

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
Last revisionBoth sides next revision
data_mining:neural_network:deep_neural_nets [2017/08/20 19:31] – [Backward prop for layer l] phreazerdata_mining:neural_network:deep_neural_nets [2017/08/20 20:04] – [Matrix dimensions] phreazer
Line 45: Line 45:
 X -> ReLU -> ReLU -> Sigmoid -> $\hat{y}$ -> $L(\hat{y}, y)$ X -> ReLU -> ReLU -> Sigmoid -> $\hat{y}$ -> $L(\hat{y}, y)$
  
 +Init backprop with derivative of $L$.
 +
 +
 +===== Matrix dimensions =====
 +$l=5$
 +2-3-5-4-2-1
 +
 +$Z^1 = W^1 * x + b^1 $
 +
 +$Z^1 :(3,1)$
 +
 +$x : (2,1)$
 +
 +$W^1 :(n^1,n^0) => W^1 (3,2), W^2(5,3)$
 +
 +$W^l :(n^l, n^{l-1})$
 +
 +$b^1 : (3,1)$
 +
 +$b^L : (n^l, 1)$
 +
 +analog with $dW^l$ and $db^l$
 +
 +==== Vectorized ====
 +
 +$Z^1 : n^1,m$
 +
 +$W^1 :(n^1, n^0)$
 +
 +$X : (n^0, m)$
 +
 +$b^1: (n^1,m)$
  • data_mining/neural_network/deep_neural_nets.txt
  • Last modified: 2017/08/20 20:04
  • by phreazer