upgrade pip3

[bertsky]

attempts to fix the docker-cuda deploy failure.

[kba]

I did try that already, that fixes this problem. But I stil fail locally to get opencv built.

[bertsky]

See this comment

[stweil]

Upgrading to a container which is based on Ubuntu > 18.04 would also have solved the problem because then python3-pip installs /usr/bin/pip, too.

[bertsky]

Ok, it seems we got everything working here, except Scrutinizer, now failing for an outdated libgeos. Looks like the base image there is quite old.

[bertsky]

Looks like the base image there is quite old.

Wow! Just found out Scrutinizer still uses Trusty (Ubuntu 14)! I'll try upgrading to Bionic...

[bertsky]

@kba this resolves everything, at last!

[bertsky]

Not so sure about everything though: At least the Python 3.6 run looks suspiciously short.

[bertsky]

Note:

46a53ab

… is necessary because when deps-test comes first, opencv-python-headless and numpy are already pulled, and so 385c595 would be too late

276ab39

… is a first shot at a GHCR deployment. I'm not sure whether it works. (One condition is that in the project Settings > Actions > General > Workflow Permissions, you have allowed read+write for GITHUB_TOKEN of this repo.)

[kba]

Note:

46a53ab

… is necessary because when deps-test comes first, opencv-python-headless and numpy are already pulled, and so 385c595 would be too late

276ab39

… is a first shot at a GHCR deployment. I'm not sure whether it works. (One condition is that in the project Settings > Actions > General > Workflow Permissions, you have allowed read+write for GITHUB_TOKEN of this repo.)

Excellent. Yes GITHUB_TOKEN is allowed.

I'm testing the docker deployment - we also need to update the CUDA Ubuntu base image to 20.04, right?

[bertsky]

I'm testing the docker deployment - we also need to update the CUDA Ubuntu base image to 20.04, right?

No we should already have that since f1527a2.

The CI VM itself uses ubuntu-latest.

[kba]

I'm testing the docker deployment - we also need to update the CUDA Ubuntu base image to 20.04, right?

No we should already have that since f1527a2.

The CI VM itself uses ubuntu-latest.

https://github.com/OCR-D/core/pull/986/files#diff-76ed074a9305c04054cdebb9e9aad2d818052b07091de1f20cad0bbac34ffb52R222 For make docker-cuda we currently use 11.3.1-cudnn8-runtime-ubuntu18.04 as the base image, should be using 11.3.1-cudnn8-runtime-ubuntu20.04 is what I meant.

[bertsky]

should be using 11.3.1-cudnn8-runtime-ubuntu20.04 is what I meant.

Oh, right!

[kba]

Unfortunately, I cannot seem to get docker-cuda build running.

The old 18.04/3.6 image (11.3.1-cudnn8-runtime-ubuntu18.04) fails because opencv-python-headless won't build.

With 11.3.1-cudnn8-runtime-ubuntu20.04:

ldconfig
apt-get -y install --no-install-recommends cuda-runtime-10-0 cuda-runtime-10-1 cuda-runtime-10-2 cuda-runtime-11-0 cuda-runtime-11-1 cuda-runtime-11-3 libcudnn7    
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package cuda-runtime-10-0
E: Unable to locate package cuda-runtime-10-1
E: Unable to locate package cuda-runtime-10-2
E: Unable to locate package libcudnn7
make: *** [Makefile:234: cuda-ubuntu] Error 100

From running nvidia/cuda:11.3.1-cudnn8-runtime-ubuntu20.04 directly, I can see that we need another apt-get update before installing. But:

root@b4480b7d3280:/# apt-cache search cuda-runtime
cuda-runtime-11-0 - CUDA Runtime 11.0 meta-package
cuda-runtime-11-1 - CUDA Runtime 11.1 meta-package
cuda-runtime-11-2 - CUDA Runtime 11.2 meta-package
cuda-runtime-11-3 - CUDA Runtime 11.3 meta-package
cuda-runtime-11-4 - CUDA Runtime 11.4 meta-package
cuda-runtime-11-5 - CUDA Runtime 11.5 meta-package
cuda-runtime-11-6 - CUDA Runtime 11.6 meta-package
cuda-runtime-11-7 - CUDA Runtime 11.7 meta-package
cuda-runtime-11-8 - CUDA Runtime 11.8 meta-package
cuda-runtime-12-0 - CUDA Runtime 12.0 meta-package
cuda-runtime-12-1 - CUDA Runtime 12.1 meta-package

and

root@b4480b7d3280:/# apt-cache search libcudnn
libcudnn8 - cuDNN runtime libraries
libcudnn8-dev - cuDNN development libraries and headers

I am not sure about the ramifications of updating libcudnn and not supporting CUDA 10 runtime here - @bertsky?

[bertsky]

I am not sure about the ramifications of updating libcudnn and not supporting CUDA 10 runtime here - @bertsky?

Indeed, this will have grave implications for our TF 1.15 processors, probably also Detectron2. I recommend keeping core-cuda on the old (now defunct) image, tracking this in a new issue and merging here – we need the new release!

[bertsky]

we need the new release!

ok, this sort of defies the original intent of this PR. But the fixes here are mostly for non-cuda now....

[kba]

OK, let's release as-is and fix cuda later.