Differences

This shows you the differences between two versions of the page.

--- data_mining:neural_network:deep_neural_nets [2017/08/20 19:28] – [Backward prop for layer l] phreazer
+++ data_mining:neural_network:deep_neural_nets [2017/08/20 20:04] (current) – [Vectorized] phreazer
@@ Line 31: / Line 31: @@
-$dZ^{[l]} = dA^{[l]} * g'^{[l]}(Z^{[l]})$ // * element-wise product
+$dZ^{[l]} = dA^{[l]} * g'^{[l]}(Z^{[l]})$ # element-wise product
 $dW^{[l]} = 1/m * dZ^{[l]} * A^{[l-1]^T}$
@@ Line 39: / Line 39: @@
 $dA^{[l-1]} = W^{[l]^T} * dZ^{[l]}$
+===== Flow =====
+Forward:
+X -> ReLU -> ReLU -> Sigmoid -> $\hat{y}$ -> $L(\hat{y}, y)$
+Init backprop with derivative of $L$.
+===== Matrix dimensions =====
+$l=5$
+-3-5-4-2-1
+$Z^1 = W^1 * x + b^1 $
+$Z^1 :(3,1)$
+$x : (2,1)$
+$W^1 :(n^1,n^0) => W^1 (3,2), W^2(5,3)$
+$W^l :(n^l, n^{l-1})$
+$b^1 : (3,1)$
+$b^L : (n^l, 1)$
+analog with $dW^l$ and $db^l$
+==== Vectorized ====
+$Z^1 : (n^1,m)$
+$W^1 :(n^1, n^0)$
+$X : (n^0, m)$
+$b^1: (n^1,m)$