Word2vec update before train error message. Fix #1162

[hemavakade]

Using Python 2.7.12 on MAC OSX VERSION 10.11.6
Used Sublime Text Build 3126 to make changes

Replicated the error by running the following

import gensim
from nltk.corpus import brown, movie_reviews, treebank
b = gensim.models.Word2Vec()
b_sents = brown.sents()
b.build_vocab(b_sents, keep_raw_vocab=False, trim_rule=None, progress_per=10000, update=True)
b.train(b_sents)

Error displayed:
ValueError: all the input array dimensions except for the concatenation axis must match exactly

Made changes in file gensim\models\word2vec.py in the function update_weights.
Added the following code which checks if the model weights have been initialized.
(line 1072 - 1076)

# Raise an error in an online update is run before initial training on a corpus
if not len(self.wv.syn0):
    raise RuntimeError("You can do an online update of vocabulary on a pre-trained model. " \
       "Or first build the vocabulary of your model with a corpus and train it " \
       "before doing an online update.")

Post-change testing:
I tested using the following code:

1. New Error message will be displayed instead of a ValueError

import gensim
from nltk.corpus import brown, movie_reviews, treebank
b = gensim.models.Word2Vec()
b_sents = brown.sents()
b.build_vocab(b_sents, keep_raw_vocab=False, trim_rule=None, progress_per=10000, update=True)
b.train(b_sents)

and

model = gensim.models.Word2Vec()
sentences = gensim.models.word2vec.Text8Corpus('/path/to/text8')
model.build_vocab(sentences, update=True)
model.train(sentences)

2. No Error message will be displayed.

import gensim
from nltk.corpus import brown, movie_reviews, treebank
b = gensim.models.Word2Vec()
b_sents = brown.sents()
b.build_vocab(b_sents, keep_raw_vocab=False, trim_rule=None, progress_per=10000, update=False)
b.train(b_sents)
b.build_vocab(b_sents, keep_raw_vocab=False, trim_rule=None, progress_per=10000, update=True)
b.train(b_sents)

model = gensim.models.Word2Vec()
sentences = gensim.models.word2vec.Text8Corpus('/path/to/text8')
model.build_vocab(sentences, update=False)
model.train(sentences)
model.build_vocab(sentences, update=True)
model.train(sentences)

[gojomo]

This is a more-helpful error than before. It'd be best to catch the user error even earlier, closer to where it happens: for example, as soon as update=True is passed to a model whose vocabulary has never been initially-built (instead of deeper in the methods called by that process).

(Separate but related: the prior existing code seems a bit off in how a pre-vocabulary model has a wv with a temporary zero-length syn0 - rather than just no wv at all – and then majorly mutates that empty KeyedVectors from outside the class. It's not so good as a matter of clear-object-lifecycle or separation-of-concerns. But I don't know about prioritizing a cleanup unless/until some larger refactoring is being done.)

[tmylk]

Please start the message by stating the cause of the warning: "The model has not yet been trained."

[gojomo]

Note it isn't necessary any training has happened - just that no initial/prior vocabulary-discovery has happened (and the update=True case assumes that it has). So perhaps instead: "Cannot update vocabulary of model which has no prior vocabulary."

[hemavakade]

I have updated the error message as,

"You cannot do an online vocabulary-update of a model which has no prior vocabulary. First build the vocabulary of your model with a corpus before doing an online update."

@gojomo, it does make sense to flag the error earlier than waiting till update_weights is called. I made a few changes on another file locally ( Note that this change is not currently reflected on github. It is local to me.) and tested the following code change.

    def build_vocab(self, sentences, keep_raw_vocab=False, trim_rule=None, progress_per=10000, update=False):
        """
        Build vocabulary from a sequence of sentences (can be a once-only generator stream).
        Each sentence must be a list of unicode strings.

        """
        if update:
            if not len(self.wv.vocab):
                raise RuntimeError("You cannot do an online vocabulary-update of a model which has no prior vocabulary." \
                "First build the vocabulary of your model with a corpus " \
                "before doing an online update.")        
        self.scan_vocab(sentences, progress_per=progress_per, trim_rule=trim_rule)  # initial survey
        self.scale_vocab(keep_raw_vocab=keep_raw_vocab, trim_rule=trim_rule, update=update)  # trim by min_count & precalculate downsampling
        self.finalize_vocab(update=update)  # build tables & arrays

It works well with the cases that I have mentioned earlier. I also tested this,

import gensim
from nltk.corpus import brown, movie_reviews, treebank
b_sents = brown.sents()
b = gensim.models.Word2Vec(b_sents)
b.build_vocab(b_sents, keep_raw_vocab=False, trim_rule=None, progress_per=10000, update=True)
b.train(b_sents)

Executing the will not display any error message.

[tmylk]

Thanks for the improvement!