Regression in handling concurrency when creating environments

[vlad-ivanov-name]

Checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

pixi run --frozen --manifest-path pixi.toml some-command |
pixi run --frozen --manifest-path pixi.toml another-command -
⠁ updating packages in 'default'                                                                                                                                                                                                                                                                                                                          
⠁ updating packages in 'default'
⠁ updating packages in 'default'
  ✔ downloading          [00:00:01] Done!
  ✔ linking              [00:00:01] Done!                                                                                                                                                                                                                                                                                                                   × failed to post process the environment (unclobbering)
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
BrokenPipeError: [Errno 32] Broken pipe

[project]
name = "requirements-3.10.13"
version = "0.1.0"
channels = ["conda-forge"]
platforms = ["linux-64"]

[tasks]

[dependencies]
python = "==3.10.13"
pip-tools = ">=7.3.0,<7.4"

Issue description

In this example, pixi runs a command in an environment, and pipes the output to another command. This seems to break (seemingly after switch to uv but could be unrelated).

The example is run from a clean state.

Doesn't work with 0.16.1

Expected behavior

Works with 0.15.2

[ruben-arts]

He @vlad-ivanov-name,
I'm can't seem to easily reproduce it, or think of a reason this would have changed. Could you give the actual commands that would trigger this error?

[vlad-ivanov-name]

I tried a few times and got a different error now but I think the root cause is probably related:

pixi run --frozen --manifest-path pixi.toml python3 file.py |
pixi run --frozen --manifest-path pixi.toml pip-compile -r - --output-file requirements.txt

⠁ updating packages in 'default'
  × failed to link 'bin/python3.10'
  ├─▶ could not open destination file for writing
  ╰─▶ Text file busy (os error 26)

file.py

if __name__ == "__main__":
    print("pytest==7.1.3")

Perhaps there's no lock on the .pixi environment dir so initializing it from both commands fails?

[ruben-arts]

On my Fedora machine I can not reproduce. Is this on windows perhaps?

That said maybe @baszalmstra you have any ideas?

[vlad-ivanov-name]

I’m also testing on Fedora 🤔

[wolfv]

I wonder if it's something related to this? rust-lang/rust#46016

Rye also had this issue: astral-sh/rye#861

[vlad-ivanov-name]

Just to clarify - the issue only occurs for me when the .pixi folder is deleted. The lockfile is present. The error above is on 0.17.1

[wolfv]

One other idea would be to set PYTHONUNBUFFERED=1, does that change anything?

[vlad-ivanov-name]

One other idea would be to set PYTHONUNBUFFERED=1, does that change anything?

Unfortunately no, still getting errors.

I noticed that the errors are sporadic, it's probably a race condition. I used this snippet to try to run the command until the error appears:

for i in $(seq 1 100) ; do rm -rf .pixi && ( pixi run --frozen --manifest-path pixi.toml python3 file.py | pixi run --frozen --manifest-path pixi.toml pip-compile -r - --output-file requirements.txt ) || break ; done

[vlad-ivanov-name]

I think perhaps broken pipe error is just a consequence of the consumer end of the pipe failing to properly initialize and read stdin

Screen.Recording.2024-03-26.at.09.40.49.mov

[vlad-ivanov-name]

Actually the minimal reproducible example for me is even simpler, no python involved at all:

pixi run --frozen --manifest-path pixi.toml echo | pixi run --frozen --manifest-path pixi.toml cat

[vlad-ivanov-name]

Same bug on MacOS:

[vlad-ivanov-name]

Just tried the latest version, the bug is still there:

⠁ updating packages in 'py38'
⠁ updating packages in 'py38'
  × failed to post process the environment (unclobbering)
  × failed to link 'include/readline/tilde.h'
  ╰─▶ EOF while parsing a value at line 1 column 0

  ├─▶ could not update destination file permissions
  ╰─▶ No such file or directory (os error 2)

[wolfv]

So what is happening here is that two pixi' instances are running at the same time and overwriting each others files, is that correct? We should probably add a file lock so that they don't modify the same environment concurrently, correct?

[vlad-ivanov-name]

Yes I think a file lock is necessary. There is https://lib.rs/crates/fs4 crate -- looks like it's cross platform and maintained

[wolfv]

We already have a file lock in pixi / rattler (https://github.com/mamba-org/rattler/blob/40803042b829346ed7522f797d70fd4da20fc12e/crates/rattler_repodata_gateway/src/utils/flock.rs#L204) - so that should be relatively straight-forward :)

[ruben-arts]

This issue seems to be solve somewhere along the way. Closing for now

[vlad-ivanov-name]

hmm I don't think this is solved. With the same manifest and commands as above, and with

pixi --version
pixi 0.30.0

I'm still getting the same error

  ⠁ installing packages  [00:00:00] [━━──────────────────]     2/24 libffi                                                                                                     × failed to link libffi-3.4.2-h3422bc3_5.tar.bz2
  ├─▶ failed to link 'share/man/man3/ffi_prep_cif_var.3'

both pixi processes will try to init the same directory

[vlad-ivanov-name]

should this be reopened or should i open a new issue?