TypeError when importing BERTopic object

[abdelatifsd]

Hello:)

After pip installing the library, importing from the module (running "from bertopic import BERTopic") leads to the following error: TypeError: load() missing 1 required positional argument: 'Loader'

Thanks!

[rtshome]

Same error for me too:

`---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
in ()
----> 1 from bertopic import BERTopic

13 frames
/usr/local/lib/python3.7/dist-packages/distributed/config.py in ()
18
19 with open(fn) as f:
---> 20 defaults = yaml.load(f)
21
22 dask.config.update_defaults(defaults)

TypeError: load() missing 1 required positional argument: 'Loader' `

[stonescenter]

the same error for me when I tried to install

[lishanda]

Looks like a dependency problem. Try pip install distributed==2021.9.0
It seems that latest commits in distributed==2021.9.1 break something.
One thing to notice - importing takes much longer time than before these changes. Other than that everything works fine for me.

[MaartenGr]

Strange, I just tested out some things and it turns out that the entire distributed package is giving issues. Just installing BERTopic in a Google Colab environment results in errors. Although this seems to be related to the distributed package, I do not believe it is a dependency within BERTopic that is causing the issue. For example, when I install BERTopic in an environment in which distributed is not installed, distributed will not be installed.

After some testing, it seems that distributed==1.25.3 is the issue here since the following fixes are working for me.

Fixes

First, with distributed==2021.9.0:

!pip install bertopic
!pip install distributed==2021.9.0

Second, it actually works with distributed==2021.9.1:

!pip install bertopic
!pip install distributed==2021.9.1

Third, it works by fully uninstalling distributed:

!pip install bertopic
!pip uninstall distributed

Source of the Issue

It seems that I can reproduce the error by uninstalling distributed==1.25.3 and then re-install it again distributed==1.25.3:

!pip install bertopic
!pip uninstall distributed
!pip install distributed==1.25.3

This means that I would advise either upgrading distributed or simply uninstalling it as it does not seem to impact the functionality of BERTopic.

New release

In the new release, I am likely just inserting distributed>1.25.3 in the requirements to make sure that version is not used. I would also need to make sure what the minimum version is needed for it to work. For example, 1.25.4 may also not work.

[MaartenGr]

@rtshome @stonescenter @abdelatifsd @lishanda
Although I am not sure why the distributed package (1.25.3) suddenly has issues with BERTopic, a fix can be found in the pull request here which you can install like this:

!pip install git+https://github.com/MaartenGr/BERTopic.git@quickfix#egg=bertopic

If this solves the problem for all of you and no large issues are found in the next coming days, I will release this to pypi as soon as possible.

[stonescenter]

It works for me, thank so match.

[rtshome]

@MaartenGr it works thanks.
At the end of the install I get the errors below. All is working fine but I don’t know if they are expected or not.
Thank you

    Attempting uninstall: distributed
    Found existing installation: distributed 1.25.3
    Uninstalling distributed-1.25.3:
      Successfully uninstalled distributed-1.25.3
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
tensorflow 2.6.0 requires numpy~=1.19.2, but you have numpy 1.21.2 which is incompatible.
gym 0.17.3 requires cloudpickle<1.7.0,>=1.2.0, but you have cloudpickle 2.0.0 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
albumentations 0.1.12 requires imgaug<0.2.7,>=0.2.5, but you have imgaug 0.2.9 which is incompatible.
Successfully installed bertopic-0.9.3 cloudpickle-2.0.0 dask-2021.9.0 distributed-2021.9.0 fsspec-2021.10.0 hdbscan-0.8.27 huggingface-hub-0.0.19 locket-0.2.1 numpy-1.21.2 partd-1.2.0 plotly-4.14.2 pynndescent-0.5.4 pyyaml-6.0 sacremoses-0.0.46 sentence-transformers-2.1.0 sentencepiece-0.1.96 tokenizers-0.10.3 transformers-4.11.3 umap-learn-0.5.1
WARNING: The following packages were previously imported in this runtime:
  [numpy]
You must restart the runtime in order to use newly installed versions.

[MaartenGr]

@rtshome Glad to hear that it works! The errors are expected and are not something that you should worry about.

[MaartenGr]

@rtshome @stonescenter @abdelatifsd @lishanda
A new version of BERTopic (v0.9.3) was released that should fix this issue and some others that should be helpful. You can install that version through pip install --upgrade bertopic. If you have any questions regarding this issue, release, or some other issue, please let me know!