Differences

This shows you the differences between two versions of the page.

--- data_mining:neural_network:word_embeddings [2018/06/09 18:26] – [Application] phreazer
+++ data_mining:neural_network:word_embeddings [2018/06/09 18:40] (current) – [Debiasing word embeddings] phreazer
@@ Line 98: / Line 98: @@
 ===== Application =====
-Sentiment classification
+==== Sentiment classification ====
+=== Simple model ===
+  * Extract embedding vector for each word
+  * Sum or Avg those vectors
+  * Pass to softmax to gain output (1-5 stars)
+Problem: Doesn't include order/sequence of words
+=== RNN for sentiment classification ===
+  * Extract embedding vector for each word
+  * Feed into RNN with softmax output
+===== Debiasing word embeddings =====
+Bias in text
+Addressing bias in word embessing:
+  - Identify bias direction (e.g. gender)
+    * $e_{he} - e_{she}$, average them
+  -  Neutralize: For every word that is not definitial (legitimate gender component), project
+  - Equalize pairs: Only difference should be gender (e.g. grandfather vs. grandmother); equidistant