Using Opacus in Google Colab: torchcsprng issue.

[mmalekzadeh]

At the moment, using Opacus on Google Colab faces this error from privacy_engine.py:

---> 10 from torchcsprng._C import *
ImportError: libcudart.so.10.2: cannot open shared object file: No such file or directory

Due to this line:

opacus/opacus/privacy_engine.py

Line 10 in b1c1bec

import torchcsprng as csprng

Silly, but my current hack is to just comment out this line before running the code. For sure Colab is not a production environment :)

I was wondering whether there is a better solution for this or not?

[Darktex]

Yes there is :) The root issue is that Colab is still on Cuda 10.1, while PyTorch has moved on to Cuda 10.2 as its default. You would actually see this issue even with installing PyTorch - you don't see it because PyTorch comes pre-installed in Colab so they have the right Cuda version already.

The fix is to just install the package for the right Cuda version you have :)

Here's the copypasta:

pip install torchcsprng==0.1.2+cu101 torch==1.6.0+cu101 torchvision==0.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html

After this, you can just pip install opacus and it will work :)

Notice our convention in naming package versions: <lib_version>+cu<cuda_version>. If at any point you lose the copypasta or forget the convention, you can simply go to the PyTorch install page, select Cuda 10.1 for Linux and you'll see it: https://pytorch.org/get-started/locally/

[mmalekzadeh]

Just a quick fix that torch==1.6.0+cu101 should be changed to torch==1.7.0+cu101 because of the recent update.

[Darktex]

Good catch! Colab also updated torch to 1.7 on their end, so this can further be reduced to just this import statement:

pip install torchcsprng==0.1.3+cu101 -f https://download.pytorch.org/whl/torch_stable.html
pip install opacus