This is an old revision of the document!
Gradient checking
Approximating derivatives:
(Large triangle (+/- triangle, two-sided difference))
$\frac{f(\Theta + \epsilon) - f(\Theta - \epsilon)}{2 \epsilon} \approx g(\Theta)$
$f'(\Theta) = \lim_{\epsilon->0} \frac{f(\Theta + \epsilon) - f(\Theta - \epsilon)}{2 \epsilon}$
Approx error is in $O(\epsilon^2)$
Take $W^{[1]}, b^{[1]}, \dots, W^{[L]},b^{[L]}$ and put it in a big vector $\theta$.