Check what's the reason to use double-precision in topic models

[menshikh-iv]

Our TMs return vectors with double-precision float64, it looks like very suspicious, because float32 is enough for all. Need to check, what's a reason of this behavior and what's a concrete method.

The first step - look at this line in the test, after it - collect all TMs, that depends on this tests and check, where and why float64 happened.

Result - detailed description (where and why), and fixing this behavior after discussion (if needed)

[gojomo]

FWIW, the Word2Vec/Doc2Vec classes use float32 in vectors/calculations. (I even did experiments converting models to float16 as an optimization; unfortunately it seems bulk array calculations still coerce back to/from float32, meaning no time savings and space savings only at rest.)

[xelez]

Started investigation.

Test classes that use TestBaseTopicModel

• TestAuthorTopicModel
• TestHdpModel
• TestLdaMallet
• TestLdaModel
• TestLsiModel

Went on to LdaModel.

Basically, numpy uses float64 by default, and in LdaModel is never specified explicitly.

• LdaState.__init__() uses np.zeros() for sstats (which then used for computing topics)
• LdaModel.__init__() does self.state.sstats = self.random_state.gamma(100., 1. / 100., (self.num_topics, self.num_terms))
• nd.asarray is used in many places that deduces type from argument by default
• ndarrays can be passed in __init__(), and they will be used as-is (float64, float32 or whatever)

So, as a result, everything just defaults to float64 so float64 is returned in get_topics().

[menshikh-iv]

Good investigation, thanks @xelez.
I think the best solution is to add dtype argument in the constructor and explicitly set dtype everywhere when needed (instead of default float64). Potentially, we'll have same problems for another method (including for training). If this is true - we can easily reduce memory twice & speedup model 🔥

[xelez]

@menshikh-iv Are there any plans on refactoring? If yes, I think this should be done with refactoring.
If no, well.. The only way I can see is to examine every line of code identifying places to apply dtype.

Well, I think I can do it for LdaModel, and maybe other models later. But if anyone will be working on it too, it could easily become a nightmare to merge.

[menshikh-iv]

@xelez
Refactoring - yes, but not soon.
We discussed this problem and decided that adding dtype argument to each model in __init__ will be good idea (new field for experiments with comparison models with hi/lo precisions)

It will be very nice if you make PR and start work with it:+1:

[piskvorky]

I believe the default should be float32 everywhere. This makes no difference to word2vec/doc2vec (which are float32 already), but potentially changing LSI/LDA etc.