-
Notifications
You must be signed in to change notification settings - Fork 752
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
RuntimeError: Already borrowed when using context managers #2525
Comments
I think it could be related to this code: /// https://github.com/PyO3/pyo3/issues/1205#issuecomment-1164096251 for advice on `__enter__`
pub fn __enter__(slf: Py<Self>) -> Py<Self> {
slf
} But I really don't know. |
@davidhewitt any chance this could be fixed by reverting to the #1205 (comment) solution? I guess @bellini666 trying out a build using I'd be happier if we knew in what situation this happens and why. |
I would think that more likely the
|
@davidhewitt just send me the instructions and I can try :) |
You can try the following to (hopefully) get a better error message:
pyo3 = { git = "https://github.com/mejrs/pyo3", branch ="debug_pycell", features = ["abi3-py37", "extension-module", "debug_pycell"] }
I would try it myself, but your repro doesn't work on windows unfortunately (signal support is rather bad on it) |
Watchfiles does work on windows, but no one other than @bellini666 has reported this error, so you probably won't see the error. |
@mejrs @samuelcolvin I'll try the instructions here. Just so I know, how exactly am I supposed to build and run watchfiles from source? I never worked with a python package with rust extensions before (and I'm not used to rust itself), so I'm not really sure the steps I should follow. |
@bellini666 you should be able to do the following:
If you get stuck at any point feel free to ask here - the point of PyO3 is to make building Rust & Python software straightforward |
Hi @davidhewitt , sorry for not being able to do this soon. I got
I did install it in the venv using just
|
@bellini666 Try |
@messense is correct, but just to give some context, |
In this case, it's because @messense I wonder if we should make |
Or a symlink? |
Imo there's not a 1:1 relationship between the source and virtualenv lib dir - the same source could be used for many virtualenvs. I don't think the overhead of making a copy is likely to matter? |
...maybe a symlink to the |
This is what I personally like to go for when working on mixed Rust-Python packages, so that I can immediately |
I'd like to take a look at how setuptools handles this before making the change. |
Taking hiredis-py as an example, looks like setuptools does the same as maturin currently. $ pip install .
Processing /Users/messense/Projects/hiredis-py
Preparing metadata (setup.py) ... done
Building wheels for collected packages: hiredis
Building wheel for hiredis (setup.py) ... done
Created wheel for hiredis: filename=hiredis-2.0.0-cp39-cp39-macosx_12_0_arm64.whl size=24683 sha256=86eb40187ac9a12c7207603879b4c66de4aac6d5f69b4f94e4ca134a6689294c
Stored in directory: /Users/messense/Library/Caches/pip/wheels/8a/65/26/c52a303875a91e19d401a8aa5d20317422b76f647db8608423
Successfully built hiredis
Installing collected packages: hiredis
Successfully installed hiredis-2.0.0
$ ls hiredis/
__init__.py __pycache__ hiredis.pyi py.typed version.py
$ ls hiredis/__pycache__/
version.cpython-39.pyc
$ python
Python 3.9.11 (main, Mar 21 2022, 19:37:23)
[Clang 13.1.6 (clang-1316.0.21.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import hiredis
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/messense/Projects/hiredis-py/hiredis/__init__.py", line 1, in <module>
from .hiredis import Reader, HiredisError, ProtocolError, ReplyError
ModuleNotFoundError: No module named 'hiredis.hiredis' |
Ok, in which case probably best to keep consistency with setuptools for now, thanks for checking 👍 |
Hey guys, The issue suddenly is not happening anymore.. I tried at the weekend in my personal computer (which also runs Debian Sid) and was unable to reproduce the issue. I tried now in my office computer, which I could reproduce the issue 100% of the times and I also can't. What changed since I opened the issue is that Debian updated the python3 package from 3.10.4 to 3.10.5. Maybe it fixed it? Strange that searching for Either way, this can probably be closed. |
Feel free to reopen if it happens again. |
as you can se this is reliebly hapening even on fresh docker instalation.... please try to solve it |
@07pepa The next step here is still #2525 (comment) done by someone who is able to reproduce this, so that we can get a backtrace of the Rust side to figure where the exception originates and determine whether something in PyO3 or in watchfiles needs to be changed. |
ok i will try over next few weekends... and if i get it done in docker you will get dockerfile as well (to help fixing it) |
Hey all- I've reproduced this on a fresh diff --git a/Cargo.toml b/Cargo.toml
index 2e2395a..1dd4065 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -25,7 +25,8 @@ include = [
[dependencies]
crossbeam-channel = "0.5.4"
notify = "5.0.0"
-pyo3 = {version = "0.17.3", features = ["extension-module", "abi3-py37"]}
+# pyo3 = {version = "0.17.3", features = ["extension-module", "abi3-py37"]}
+pyo3 = { git = "https://github.com/mejrs/pyo3", branch ="debug_pycell", features = ["abi3-py37", "extension-module", "debug_pycell"] }
[lib]
name = "_rust_notify" and
diff --git a/Cargo.lock b/Cargo.lock
index 9ac837b..f40deb3 100644
--- a/Cargo.lock
+++ b/Cargo.lock
@@ -240,9 +240,8 @@ dependencies = [
|
I tried this and it does reproduce here as well, but I don't think this is a PyO3 issue. The program blocks in raw_changes = await anyio.to_thread.run_sync(watcher.watch, debounce, step, timeout, stop_event_) from watchfile's py.allow_threads(|| sleep(step_time)); in I think the fix here is to adjust |
thanks so much both of you for looking into this. Would be wonderful if someone could submit a PR to watchfiles, otherwise I'll try to get to it when I have time. |
…signal (#221) Done as suggested in PyO3/pyo3#2525 (comment) Fixes #200
Bug Description
This issue was originally reported in watchfiles, but the maintainer there said that it is probably an issue with this lib, maybe related to #1205
The issue is that I get
RuntimeError: Already borrowed
when exiting a code that uses a context managerSteps to Reproduce
I created this small script which will raise the issue everytime I finish the process:
Just create a
venv
, installwatchfiles
in it and run the script. Close it withctrl+C
or by sending orkill -2 <PID>
Backtrace
Your operating system and version
Debian Sid
Your Python version (
python --version
)Python 3.10.5
Your Rust version (
rustc --version
)rustc 1.59.0
Your PyO3 version
The one used by watchfiles
How did you install python? Did you use a virtualenv?
apt
Additional Info
No response
The text was updated successfully, but these errors were encountered: