data_mining:neural_network:loss_functions

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
data_mining:neural_network:loss_functions [2019/11/10 00:22] phreazerdata_mining:neural_network:loss_functions [2019/11/10 00:40] (current) – [Binary cross entropy] phreazer
Line 38: Line 38:
  
 For an algo we need to find closes $p$ for $q$ by minimizing cross-entropy. For an algo we need to find closes $p$ for $q$ by minimizing cross-entropy.
 +
 +In training we have $N$ samples. For one particular example the distribution is known to which class it belongs. The loss function should minimize the average cross-entropy.
 +
 +Outcome: Scalar [0,1] using sigmoid
 +
 +===== Cross-entropy =====
 +
 +$-\sum_{i}^C(y_i log(\hat{y_i})$
 +$C$ is number of classes
 +
 +Outcome: Vector [0,1] using softmax
 +
 +===== Binary cross entropy with multi labels =====
 +
 +$-\sum_{i}^C(y_i log(\hat{y_i}) + (1-y_i) log(1-\hat{y_i})$
 +
 +Ouctome: Vector [0,1] using sigmoid
 +
  • data_mining/neural_network/loss_functions.txt
  • Last modified: 2019/11/10 00:40
  • by phreazer