Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Distributed and multicore LDA print messages saying LDA is being run serially #375

Closed
ghost opened this issue Jul 1, 2015 · 2 comments
Closed

Comments

@ghost
Copy link

ghost commented Jul 1, 2015

There is some code in ldamodel.py that passes distributed=False when the user specified distributed=True. This causes the confusing error message that LDA is running serially on that node to be printed.

@ghost
Copy link
Author

ghost commented Jul 1, 2015

I have to say, these messages are quite confusing, especially since only one core is being used.

2015-07-01 00:33:08,859 : INFO : registering worker #1 at PYRO:gensim.lda_worker.63bd2@127.0.0.1:35365
2015-07-01 00:33:08,961 : INFO : initializing worker #1
2015-07-01 00:33:08,965 : INFO : using symmetric alpha at 0.0001
2015-07-01 00:33:08,966 : INFO : using serial LDA version on this node
2015-07-01 00:36:31,625 : INFO : registering worker #2 at PYRO:gensim.lda_worker.3ea807@127.0.0.1:40761
2015-07-01 00:36:31,725 : INFO : initializing worker #2
2015-07-01 00:36:31,728 : INFO : using symmetric alpha at 0.0001
2015-07-01 00:36:31,730 : INFO : using serial LDA version on this node
2015-07-01 00:39:54,802 : INFO : registering worker #3 at PYRO:gensim.lda_worker.52cba6@127.0.0.1:34256
2015-07-01 00:39:54,900 : INFO : initializing worker #3
2015-07-01 00:39:54,904 : INFO : using symmetric alpha at 0.0001
2015-07-01 00:39:54,905 : INFO : using serial LDA version on this node

@piskvorky
Copy link
Owner

Only the master node is distributed; all other nodes (workers) run serial computations.

The flow is simple: master prepares and sends out tasks to workers, workers do work and send results back to master, master aggregates them into the LDA model.

@tmylk tmylk closed this as completed Jan 10, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants