Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revisionNext revision | Previous revision | ||
| data_mining:neural_network:neurons [2017/08/19 17:40] – [Rectified Linear Neurons] phreazer | data_mining:neural_network:neurons [2017/08/19 17:43] (current) – [Rectified Linear Neurons] phreazer | ||
|---|---|---|---|
| Line 75: | Line 75: | ||
| $z=b+\sum_{i} x_{i} w_{i}$ | $z=b+\sum_{i} x_{i} w_{i}$ | ||
| - | $y = \begin{cases} z, & \text{if } z > 0 \\ 0, & \text{otherwhise}\end{cases} = max(0,z)$ | + | $y = \begin{cases} z, & \text{if } z > 0 \\ 0, & \text{otherwhise}\end{cases} = \max(0,z)$ |
| Above 0, it is linear, at 0 it is 0 | Above 0, it is linear, at 0 it is 0 | ||
| + | |||
| + | Faster computation, | ||
| + | |||
| + | Leaky ReLU: | ||
| + | |||
| + | $y =\max(0.01 z,z)$ | ||