Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| data_mining:neural_network:short_overview [2018/04/22 00:15] – [Calculate predictions errors:] phreazer | data_mining:neural_network:short_overview [2018/05/10 17:32] (current) – [Cost function] phreazer | ||
|---|---|---|---|
| Line 96: | Line 96: | ||
| * Sum over k: number of outputs | * Sum over k: number of outputs | ||
| * Sum over all $\theta_{ji}^{(l)}$ without bias units. | * Sum over all $\theta_{ji}^{(l)}$ without bias units. | ||
| + | * Frobenius norm for regularization, | ||
| ===== Backpropagation Algorithm ===== | ===== Backpropagation Algorithm ===== | ||
| Line 108: | Line 109: | ||
| a^{(1)} = x \\ | a^{(1)} = x \\ | ||
| z^{(2)} = \theta^{(1)} a^{(1)} \\ | z^{(2)} = \theta^{(1)} a^{(1)} \\ | ||
| - | a^{(2)} = g(z^{(2)}) \text{ | + | a^{(2)} = g(z^{(2)}) \text{ |
| \dots | \dots | ||
| $$ | $$ | ||
| Line 121: | Line 122: | ||
| Vectorized: $\delta^{(4)} = a^{(4)} - y$ | Vectorized: $\delta^{(4)} = a^{(4)} - y$ | ||
| + | |||
| + | $.*$ is element-wise multiplication | ||
| $$\delta_j^{(3)} = (\theta^{(3)})^T\delta^{(4)}.*g' | $$\delta_j^{(3)} = (\theta^{(3)})^T\delta^{(4)}.*g' | ||
| Line 130: | Line 133: | ||
| Algorithmus | Algorithmus | ||
| - | $$\Delta_{ij}^{(l)} = 0 \text{für alle i,j,l} \\ | + | $$\text{Set } \Delta_{ij}^{(l)} = 0 \text{ |
| \text{For i=1 to m:} \\ | \text{For i=1 to m:} \\ | ||
| - | \text{Set} a^{(1)} = x^{(i)}$$ | + | \text{Set } a^{(1)} = x^{(i)}$$ |
| Forward propagation to compute $a^{(l)}$ für $l=2, | Forward propagation to compute $a^{(l)}$ für $l=2, | ||