script to evaluate trained models

Summary: This script processes the trained unsupervised models and evaluates them against different tests.

Reviewed By: EdouardGrave

Differential Revision: D5115345

fbshipit-source-id: 78c0b5f5773ca5b789d37a20ef8ef4550127ac6d

eval.py

@@ -40,6 +40,8 @@ for i, line in enumerate(fin):
         tab = compat_splitting(line)
         vec = np.array(tab[1:], dtype=float)
         word = tab[0]
+        if np.linalg.norm(vec) == 0:
+            continue
         if not word in vectors:
             vectors[word] = vec
     except ValueError: