Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revision | Next revisionBoth sides next revision | ||
data_mining:entropie [2013/09/15 17:20] – [Mutual information] phreazer | data_mining:entropie [2013/09/15 18:04] – [Mutual information] phreazer | ||
---|---|---|---|
Line 39: | Line 39: | ||
p(~hate, | p(~hate, | ||
$$ | $$ | ||
+ | |||
+ | Features selection => Die, die höchste MI haben, allerdings zu rechenintensiv | ||
+ | |||
+ | Proxies: IDF; iterativ AdaBoost | ||
+ | |||
+ | Mehr features -> | ||
+ | NBC verbessert sich, fällt dann. | ||
+ | |||
+ | Redundante Features, Annahme von Bayes | ||
+ | |||
+ | ====== Beispiel ====== | ||
+ | p(+) = 10.000/ | ||
+ | p(-) = 5.000/ | ||
+ | p(hate) = 3.000/ | ||
+ | p(~hate) = 0,8\\ | ||
+ | p(hate,+) =1/15.000 \text{(kommt in keinem positiven Kommentar vor, 1 anstelle von Null => Smoothing)}\\ | ||
+ | p(~hate,+) = 10.000/ | ||
+ | p(hate,-) = 3.000/ | ||
+ | p(~hate,-) = 2.000/ | ||
+ | |||
+ | $$ | ||
+ | I(H,S) = p(hate,+) * log \frac{p(hate, | ||
+ | $$ | ||
+ | |||
+ | |||
+ | ====== Kapazität eines Kanals ====== | ||
+ | |||
+ | Maximale mutual information, | ||
+ | |||
+ | Äquivalent im ML: Wie viele Trainingsdaten notwendig -> Abhängig vom Konzept | ||
+ | |||
+ | |||
+ | |||
+ | |||
+ | |||