Tensorflow 2.13.0 cannot be imported after install #8271
Comments
dre-hh
added
kind/bug
dre-hh
changed the title
|
tensorflow's metadata does not express that dependency: No mention here of I guess this is another variation on tensorflow/tensorflow#58674 please encourage the tensorflow folk to publish consistent metadata in all of their wheels, with platform-specific variations described by markers |
|
@dre-hh Have using environment markers to get around the issue? If we take a look at tensorflow/tools/pip_package/setup.py, we see that they require the tensorflow-macos package where the environment marker has been satisfied. It's generated, but would look something like this. Following the Poetry documentation here, your dependency specification should look this. I'd tend to agree with @dimbleby, the devs over at @tensorflow should improve the compatibility of their project with package managers, but that's another battle entirely. |
|
For people still looking for a full combo workaround, this should work for tensorflow 2.11 in all situations (in reality I haven't tested everything).
This should work with windows, new mac, old mac, docker in new mac, amd and non-amd linux machines ...
Currently, I advise to copy the full code below in your # Issue between poetry and tensorflow metadata since >=2.11
# This is a temporary workaround
# related to https://github.com/python-poetry/poetry/issues/8271
# Inspired from https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L148-L162
tensorflow = {version = "^2.13.0" }
tensorflow-macos = { version = "^2.13.0", platform = "darwin", markers = "platform_machine=='arm64'" }
tensorflow-intel = { version = "^2.13.0", platform = "win32" }
tensorflow-cpu = [
{ version = "^2.13.0", platform = "linux", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
{ version = "^2.13.0", platform = "darwin", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
]
tensorflow-cpu-aws = { version = "^2.13.0", platform = "linux", markers = "platform_machine=='arm64' or platform_machine=='aarch64'" }
# https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L107-L108
# https://github.com/python-poetry/poetry/issues/8271#issuecomment-1697740447
tensorflow-io-gcs-filesystem = [
{ version = ">= 0.23.1", markers = "platform_machine!='arm64' or platform_system!='Darwin'" },
{ version = "< 0.32.0", markers = "platform_system == 'Windows'" }
]EDIT : I fixed above the tensorflow-io-gcs-filesystem issue from @RRiva |
|
Thanks a lot for the suggestion. Poetry generated the lock file, but when I test my package on Windows the installation fails with It's not entirely Poetry's fault, as this package doesn't provide wheels for Windows on PyPi, but still... |
|
Hi @RRiva , I went back to the tensorflow file and added what seems to be their way to specify "tensorflow-io-gcs-filesystem" Still a temporary fix. Wondering about publishing something like a package with just a pyproject.toml to avoid everybody having to fix it. I am afraid it will made both side lazy. :/ @dimbleby you seem to better grasp what's up. Do you have any documentation or an issue that will give an initial roadway on how to fix this ? |
|
Thanks again 🙂 Unfortunately, the lock file is the same as before, and therefore the installation fails in the same way. Might be related to #1616. |
|
#1616 is about different path dependencies with the same version (and resolved), thus, not related. It's related to #2012 and #4046. If there is no wheel for Windows (and no sdist), you'll want to exclude that version for Windows: |
|
@radoering adding it to my previous answering and citing this to have a standalone. |
|
Unfortunately the workaround by Wirg and radoering still does not seem to work on Windows. The pyproject.toml: [tool.poetry]
name = "tt2"
version = "0.1.0"
description = ""
authors = ["Hannes Krumbiegel <Vuizur@users.noreply.github.com>"]
readme = "README.md"
[tool.poetry.dependencies]
python = "^3.10,<3.12"
# Issue between poetry and tensorflow metadata since >=2.11
# This is a temporary workaround
# related to https://github.com/python-poetry/poetry/issues/8271
# Inspired from https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L148-L162
tensorflow = {version = "^2.13.0" }
tensorflow-macos = { version = "^2.13.0", platform = "darwin", markers = "platform_machine=='arm64'" }
tensorflow-intel = { version = "^2.13.0", platform = "windows" }
tensorflow-cpu = [
{ version = "^2.13.0", platform = "linux", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
{ version = "^2.13.0", platform = "darwin", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
]
tensorflow-cpu-aws = { version = "^2.13.0", platform = "linux", markers = "platform_machine=='arm64' or platform_machine=='aarch64'" }
# https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L107-L108
# https://github.com/python-poetry/poetry/issues/8271#issuecomment-1697740447
tensorflow-io-gcs-filesystem = [
{ version = ">= 0.23.1", markers = "platform_machine!='arm64' or platform_system!='Darwin'" },
{ version = "< 0.32.0", markers = "platform_system == 'Windows'" }
]
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"causes an error with: (For both Python 3.10 and 3.11 I get an error like this.) I got the idea to upgrade Poetry from 3.5 to 3.6 or so, and the install seems to work: But when I try to execute a module with |
|
I found the solution: [tool.poetry.dependencies]
python = "^3.10,<3.12"
# Issue between poetry and tensorflow metadata since >=2.11
# This is a temporary workaround
# related to https://github.com/python-poetry/poetry/issues/8271
# Inspired from https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L148-L162
tensorflow = {version = "^2.13.0" }
tensorflow-macos = { version = "^2.13.0", platform = "darwin", markers = "platform_machine=='arm64'" }
tensorflow-intel = { version = "^2.13.0", platform = "win32" }
tensorflow-cpu = [
{ version = "^2.13.0", platform = "linux", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
{ version = "^2.13.0", platform = "darwin", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },]
tensorflow-cpu-aws = { version = "^2.13.0", platform = "linux", markers = "platform_machine=='arm64' or platform_machine=='aarch64'" }
# https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L107-L108
# https://github.com/python-poetry/poetry/issues/8271#issuecomment-1697740447
tensorflow-io-gcs-filesystem = [
{ version = ">= 0.23.1", markers = "platform_machine!='arm64' or platform_system!='Darwin'" },
{ version = "< 0.32.0", markers = "platform_system == 'Windows'" }
] |
|
Data point: Vuizur 's solution worked for me on M2 max. |
Just tried using this solution already taking into account the 3rd edit, and it's not working on windows, currently using Poetry version: 1.4.2, CUDA: 11.8 and windows 10. Adding this to the toml either keeps trying to install i/o package v0.34.0 which it shouldn't, and when forced to install 0.31.0, it tries also installing tensorflow-intel, and regardless of that, at the end tensorflow still cannot be imported. Has it worked for anyone else in windows? or any hints towards what I could do next? |
At first, please try the latest version of Poetry. 1.4.2 had already been out of date when this issue was created. (I don't know if it works with a more recent version, but it does not help to find bugs that may have already been fixed.) |
Thank you for pointing this out, I didn't notice I was not working on the latest Poetry release. Already updated to Poetry v1.7.1, installed the project, which indeed now is able to install all depedencies smoothly, unlike when installing with 1.4.2, but still tensorflow cannot be imported. I will include my toml and the output from toml:
|
|
looks like you simply haven't activated the virtual environment |
no, I am activating the environment. I also tried this inside a jupyter notebook inside a poetry shell and the same happens, it's only tensorflow that seems to not work. Edit: Actually I just noticed that apparently inside my jupyter notebook, tensorflow and all other libraries can now be imported, but in the ps prompt when I run |
|
I'm having this same issue with I do not have any of these packages installed globally. So, I know my environment is fine. I even did ah, I see the issues linked at the top ... I should have followed those first |
|
I've just tried Vuizur's solution. I can generate the lock file, and install my package on Linux and Windows, using Poetry 1.7.1. What I don't understand is why TensorFlow can be imported on my Windows 11 pc, but not on the CI machine with Docker container |
|
tensorflow versions up to but not including 2.15.0.post1 fail to declare their windows-only dependency on tensorflow 2.15.0.post1 declares which would be right, except that they have failed to publish a y'all should be reporting these things to tensorflow, not poetry |
|
The issue has been resolved on Windows: please refer tensorflow/tensorflow#62899 (comment) |
|
Hi @dimbleby, can you please let me know what error are you getting and on which platform? |
|
Telling users that they should manually add the dependencies that tensorflow forgot to declare is at best a workaround, not a resolution. |
|
Hi @dimbleby , TensorFlow supports pip as Python package manager: https://www.tensorflow.org/install/pip#windows-native. We use pip for our internal testing. Does the issue reported here happen only for Poetry? |
is obviously going to be true for everyone, I do not understand what testing you can be doing that does not find this building cross-platform solutions and therefore wanting consistent metadata in all distributions is more poetry-specific, though you would have to ask all the other tools - rather than ask here - if you want to know whether all the other tools also mind. Apart from the failure to publish 2.15.0.post1 versions of things it seems as though tensorflow is almost there... |
|
For some reason, even though I can see When I install it in my virtual environment, the dependency doesn't seem to be there. When I attempt to download if directly via PS: I made sure to upgrade Regardless, even with v2.12.0, Any thoughts anyone? |
|
Update: In fact, I can get |
this optimism seems to have been unwarranted, tensorflow have now rearranged things significantly - and they are once again providing completely different metadata in different distributions of their most recent release. eg compare the output of to be clear: this variation is (unfortunately!) allowed by the packaging specifications, if being poetry-compatible is more trouble than it is worth to tensorflow users and contributors then so be it. But this is not and almost certainly never will be compatible with installation through poetry. |
|
Things should improve with the next TensorFlow release. Fingers crossed 🙂 |
|
On windows, Also should do |
Secrus
added
the
status/external-issue
macOS-13.3-arm64-arm-64bit-vvvoption) and have included the output below.Issue
After installing tensorflow 2.13.0 with above pyproject.toml on macOs Apple Sillicon , importing fails with
ModuleNotFoundError: No module named 'tensorflowSame method works under linux.
HINT: in tensorflow 2.13 the authors introduce platform specific install for pip. Under macOs arm it is supposed to install
tensorflow-macos . under linux there is another dependency tree
The text was updated successfully, but these errors were encountered: