Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move load and save word2vec_format out of word2vec class to KeyedVectors #1107

Merged
merged 62 commits into from
Jan 27, 2017
Merged

Move load and save word2vec_format out of word2vec class to KeyedVectors #1107

merged 62 commits into from
Jan 27, 2017

Conversation

tmylk
Copy link
Contributor

@tmylk tmylk commented Jan 24, 2017 •
edited
Loading

Move save_word2vec_format and load_word_vec_format out of word2vec class and into KeyedVectors.

Benefits of refactoring:

  • support of word2vec C format in FastText, WordRank and other word embedding wrappers

Deprecation warnings in word2vec class on save_word2vec_format and load_word_vec_format

word2vec.load_word2vec_format still returns a word2vec class. This method will soon be removed.

Continuing #1078.

droudy and others added 30 commits September 12, 2016 19:06
@jayantj
updated refactor
55a4fc9
@jayantj
commit missed file
e916f7e
@jayantj
docstring added
e5416ed
@jayantj
more refactoring
e64766b
@jayantj
add missing docstring
c34cf37
@jayantj
fix docstring format
c9b31f9
@droudy @jayantj
clearer docstring
a0329af
@jayantj
minor typo in word2vec wmdistance
0c0e2fa
@jayantj
pyemd error in keyedvecs
cdefeb0
@jayantj
relative import of keyedvecs from word2vec fails
1aec5a2
@jayantj
bug in init_sims in word2vec
e7368a3
@jayantj
property descriptors for syn0, syn0norm, index2word, vocab - fixes bu…
fe283c2 
…g in saving
@jayantj
tests for loading older word2vec models
9b36bc4
@jayantj
backwards compatibility for loading older models
dfe1893
@jayantj
test for syn0norm not saved to file
4a03f20
@jayantj
syn0norm not saved to file for KeyedVectors
09b6ebe
@jayantj
tests and fix for accuracy
7df4138
@jayantj
minor bug in finalized vocab check
4c54d9b
@jayantj
warnings for direct syn0/syn0norm access
a28f9f1
@jayantj
fixes use of most_similar in accuracy
bf1182e
@jayantj
changes logging level to ERROR in word2vec tests
5a6b97b
@jayantj
renames kv to wv in word2vec
cfb2e1c
@jayantj
minor bugs with checking existence of syn0
b002765
@jayantj
replaces syn0 and syn0norm with wv.syn0 and wv.syn0norm in tests and …
27c0a14 
…cython files
@jayantj
adds changelog
81f8cbb
@jayantj
initial fastText wrapper class
aa7e632
@jayantj
fasttext load binary data + oov vectors
c780b9b
@jayantj
tests for fasttext wrapper
ccf5a47
@jayantj
reduced memory requirements for fasttext model
708113b
@jayantj
annoy indexer tests for fasttext
b7de266
jayantj and others added 24 commits December 22, 2016 15:58
@jayantj
refactors FastText to use subclass of KeyedVectors, updates tests
3b9483b
@jayantj
Merge branch 'develop' into fasttext
f5cdfb6 
Conflicts:
	CHANGELOG.md
	gensim/models/word2vec.py
@jayantj
changes setUp for fast text unittests to setUpClass to reduce time taken
700dd26
@jayantj
adds normalized ngram vectors for fasttext model, tests
d30ea56
@jayantj
deletes training files after loading model, tests
bb6e538
@jayantj
doesnt match with oov words, tests
c7a5d07
@jayantj
more asserts while loading from fasttext model file, renames some var…
734057b 
…iables
@jayantj
updates FastText __contains__ to return True for all words for which …
56d89e9 
…vectors exist
@jayantj
updates docstrings, adds comments for fasttext wrapper and tests
dc51096
@jayantj
adds fasttext test models
bb48663
@jayantj
changes setUpClass to setUp to allow python2.6 compatibility
b58dd53
@jayantj
updates word2vec test model files
461a6b4
@jayantj
python2.6 compatibility for fasttext tests
9137090
@jayantj
Revert "updates keyedvector load tests to use actual values"
e5ae899 
This reverts commit 6e20834.

Conflicts:
	gensim/test/test_word2vec.py
@jayantj
Merge branch 'develop' into fasttext
b98b40f 
Conflicts:
	gensim/models/word2vec.py
@jayantj
replaces all instances of vocab and syn0 being accessed directly thro…
5eb8f75 
…ugh word2vec instance
@jayantj
adds fasttext tutorial notebook
27bec7b
@jayantj
minor doc updates
ef0e1e2
@jayantj
removes direct vocab access in FastText
ab07ef9
@jayantj
suppresses numpy overflow warning while computing fasttext hash
2f37b04
@jayantj
load_word2vec_format returns KeyedVector, minor refactoring
5653632
@tmylk
Move load and save word2vec_format to KeyedVectors
2425478
@tmylk
Fix save_word2vec_format in kv
21c2099
@tmylk
Fix merge artifacts in test_wor2vec
95da5b3
@tmylk
Copy link
Contributor Author

tmylk commented Jan 24, 2017

Pinging @jayantj and @gojomo for review.

@cedias cedias mentioned this pull request Jan 27, 2017
Closed
@tmylk tmylk merged commit 41ff1f4 into develop Jan 27, 2017
@tmylk tmylk deleted the keyed_vectors_load_save_w2v_c_format branch January 27, 2017 16:31
@tmylk tmylk mentioned this pull request Feb 25, 2017
Closed
@tmylk tmylk mentioned this pull request Mar 12, 2017
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@tmylk @droudy @jayantj