Differences

This shows you the differences between two versions of the page.

--- data_mining:neural_network:belief_nets [2017/04/30 09:57] – [Discriminative fine-tuning for DBNs] phreazer
+++ data_mining:neural_network:belief_nets [2017/07/30 16:05] (current) – [Structure] phreazer
@@ Line 128: / Line 128: @@
 More layers => lower error with pretraining.
+Solutions are qualitative different.
+==== Model real-valued data with RBMS ====
+Mean-field logistic units cannot represent precise inetermediate values (e.g. pixel intensity in image).
+Model pixels as Gaussian variables. Alternating Gibbs sampling, with lower learning rate.
+Parabolic containment function. (keep visible unit close to b_i).
+Energy-gradient.
+Stepped sigmoid units. Many copies of a stochastic binary unist. All copies have same weiths and bias, b, but they have different fixed offsets to the bias (b-0.5, b-1.5, ...).
+==== Structure ====
+Autoencoder, then feed forward NN