Torchserve-kserve 0.8.0-gpu image is missing several dependencies

[cjidboon94]

🐛 Describe the bug

I'm in the process of upgrading my torchserve runtime on KServe to 0.8.0, for this I updated my Inferenceservice's image to:
https://hub.docker.com/layers/pytorch/torchserve-kfs/0.8.0-gpu

However, on startup it seems that Tornado is not installed and the following error occurs: ModuleNotFoundError: No module named 'tornado'

  File "/home/model-server/kserve_wrapper/__main__.py", line 7, in <module>
    from TorchserveModel import TorchserveModel
  File "/home/model-server/kserve_wrapper/TorchserveModel.py", line 9, in <module>
    import tornado.web

Error logs

  File "/home/model-server/kserve_wrapper/__main__.py", line 7, in <module>
    from TorchserveModel import TorchserveModel
  File "/home/model-server/kserve_wrapper/TorchserveModel.py", line 9, in <module>
    import tornado.web

Installation instructions

I pulled the image from https://hub.docker.com/layers/pytorch/torchserve-kfs/0.8.0-gpu

Model Packaing

Does not seem relevant as this happens before model loading occurs.

config.properties

Not relevant

Versions

See the docker image pytorch/torchserve-kfs:0.8.0-gpu

Repro instructions

docker run --rm --gpus=all pytorch/torchserve-kfs:0.8.0-gpu

Possible Solution

Installing tornado

[cjidboon94]

BTW, KServe has moved away from Tornado in favor of FastApi/Uvicorn: kserve/kserve#2444
Might be better to make the necessary changes to match that instead of just installing tornado

[cjidboon94]

For now, implemented workaround from #2123

[cjidboon94]

Installed Tornado, however there is another dependency missing. This time from the kserve package:

Traceback (most recent call last):
  File "/home/model-server/kserve_wrapper/__main__.py", line 7, in <module>
    from TorchserveModel import TorchserveModel
  File "/home/model-server/kserve_wrapper/TorchserveModel.py", line 11, in <module>
    from kserve.model import ModelMissingError
ImportError: cannot import name 'ModelMissingError' from 'kserve.model' (/home/venv/lib/python3.9/site-packages/kserve/model.py)

Seems like this class has been moved to more appropriate kserve.errors module.

[cjidboon94]

Probably would be fixed by: #2175

[cjidboon94]

Building torchserve-kfs:0.8.0-gpu based on #2175 works for me.

[msaroufim]

Alright I merged that PR - cc @agunapal we need to release the binaries

[agunapal]

@msaroufim So, should we revert this #2355

[msaroufim]

I don't believe #2355 was merged but also worth doing a manual sanity check like @cjidboon94 did or ideally merging your kserve CI PR

[agunapal]

@cjidboon94 Thanks for reporting the issue and finding the solution. Republished the images. Please check.

[agunapal]

I dont see the import error with the new images when I run the docker run command

[cjidboon94]

@agunapal Thanks! It works for me too. Good to be able to use the officially released image again

[cjidboon94]

@agunapal @msaroufim Btw, you might want to rebuild the 0.7.1 torchserve-kserve images too, as this issue also seems to be occurring there: #2123