data_mining:neural_network:loss_functions

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
data_mining:neural_network:loss_functions [2019/11/10 00:31] – [Binary cross entropy] phreazerdata_mining:neural_network:loss_functions [2019/11/10 00:40] (current) – [Binary cross entropy] phreazer
Line 40: Line 40:
  
 In training we have $N$ samples. For one particular example the distribution is known to which class it belongs. The loss function should minimize the average cross-entropy. In training we have $N$ samples. For one particular example the distribution is known to which class it belongs. The loss function should minimize the average cross-entropy.
 +
 +Outcome: Scalar [0,1] using sigmoid
 +
 +===== Cross-entropy =====
 +
 +$-\sum_{i}^C(y_i log(\hat{y_i})$
 +$C$ is number of classes
 +
 +Outcome: Vector [0,1] using softmax
 +
 +===== Binary cross entropy with multi labels =====
 +
 +$-\sum_{i}^C(y_i log(\hat{y_i}) + (1-y_i) log(1-\hat{y_i})$
 +
 +Ouctome: Vector [0,1] using sigmoid
 +
  • data_mining/neural_network/loss_functions.txt
  • Last modified: 2019/11/10 00:40
  • by phreazer