data_mining:neural_network:loss_functions

# Differences

This shows you the differences between two versions of the page.

 data_mining:neural_network:loss_functions [2019/11/10 00:31]phreazer [Binary cross entropy] data_mining:neural_network:loss_functions [2019/11/10 00:40] (current)phreazer [Binary cross entropy] 2019/11/10 00:40 phreazer [Binary cross entropy] 2019/11/10 00:31 phreazer [Binary cross entropy] 2019/11/10 00:22 phreazer 2019/11/10 00:10 phreazer [Binary cross entropy] 2019/11/10 00:04 phreazer [Binary cross entropy] 2019/11/09 23:58 phreazer [Binary cross entropy] 2019/11/09 23:30 phreazer created 2019/11/10 00:40 phreazer [Binary cross entropy] 2019/11/10 00:31 phreazer [Binary cross entropy] 2019/11/10 00:22 phreazer 2019/11/10 00:10 phreazer [Binary cross entropy] 2019/11/10 00:04 phreazer [Binary cross entropy] 2019/11/09 23:58 phreazer [Binary cross entropy] 2019/11/09 23:30 phreazer created Line 40: Line 40: In training we have $N$ samples. For one particular example the distribution is known to which class it belongs. The loss function should minimize the average cross-entropy. In training we have $N$ samples. For one particular example the distribution is known to which class it belongs. The loss function should minimize the average cross-entropy. + + Outcome: Scalar [0,1] using sigmoid + + ===== Cross-entropy ===== + + $-\sum_{i}^C(y_i log(\hat{y_i})$ + $C$ is number of classes + + Outcome: Vector [0,1] using softmax + + ===== Binary cross entropy with multi labels ===== + + $-\sum_{i}^C(y_i log(\hat{y_i}) + (1-y_i) log(1-\hat{y_i})$ + + Ouctome: Vector [0,1] using sigmoid +
• data_mining/neural_network/loss_functions.txt