Decouple project from the existence of Pipfile. #3386

frostming · 2018-12-17T07:16:39Z

Improve message when working under the root directory
Related: pip9/baseparser.py line 149 in __init__ assert self.name AssertionError #2210
Fixes Running pipenv in / causes a stacktrace AssertionError on assert self.name #3434

Quick example:
```
$ cd /
$ pipenv install
...
ERROR:: Pipenv is not intended to work under root directory, please choose another path.
```
Show the project path in the output of pipenv --venv. This may be useful when the Pipfile is not under the current path.
Allow $ pipenv --venv and $ pipenv --rm to work even if Pipfile is not present.

techalchemy

Ill get ahead of this and just say this has been attempted. The idea of a project in pipenv is inherently tied to a Pipfile and this probably treads in the behavior change area. It might not— and that’s not a dealbreaker. But I should look at it before it gets merged

I like the idea so if it can be done with no compatibility issues I’ll be happy

techalchemy · 2018-12-19T06:39:41Z

Note that anything that touches location related code needs careful consideration and test coverage

frostming · 2018-12-19T06:42:22Z

@techalchemy I started this PR to provide a user-friendly message and found it needed several tweaks to make it work intuitively. It ended up cleaning a TODO item somehow, so I just changed the title.

I made some comments on changed lines to see what is changed clearly.

frostming · 2018-12-19T07:12:46Z

pipenv/core.py

+            "please choose another path.".format(crayons.red("ERROR")),
+            err=True
+        )
+        sys.exit(1)


We don't touch an empty Pipfile here.

frostming · 2018-12-19T07:13:36Z

pipenv/core.py

-        if not bare:
-            location = shorten_path(location)
-        if not location:
+        if not project.pipfile_exists:


pipfile_location no longer returns None, checking the existence makes more sense.

frostming · 2018-12-19T07:14:41Z

pipenv/project.py

@@ -222,7 +222,7 @@ def name(self):

    @property
    def pipfile_exists(self):
-        return bool(self.pipfile_location)
+        return os.path.isfile(self.pipfile_location)


Check the existence of the file instead

frostming · 2018-12-19T07:16:33Z

pipenv/project.py

-
-        else:
-            return None
+        return os.path.abspath(os.path.join(self.pipfile_location, os.pardir))


Return the project home even if there is no Pipfile existing.

NOTE: $ pipenv --where still prints an error if Pipfile doesn't exist.

frostming · 2018-12-19T07:17:12Z

pipenv/project.py

@@ -255,8 +251,7 @@ def is_venv_in_project(self):

    @property
    def virtualenv_exists(self):
-        # TODO: Decouple project from existence of Pipfile.
-        if self.pipfile_exists and os.path.exists(self.virtualenv_location):
+        if os.path.exists(self.virtualenv_location):


So pipenv --venv and pipenv --rm is possible when Pipfile is not present.

techalchemy

Most of this is ok but some things are changing for no reason and some things will break functionality even if it isn’t intuitive (there is a reason it is written that way to begin with). I’m not willing to regress things just to be extra sure a user supplied path is actually a file for instance

techalchemy · 2018-12-19T06:56:54Z

pipenv/core.py

-        if deploy is True:
-            raise exceptions.PipfileNotFound
-        else:
-            project.touch_pipfile()


We do still want to create non existent pipfiles when we ensure projects exist, no?

techalchemy · 2018-12-19T07:04:26Z

pipenv/core.py

-        if not bare:
-            location = shorten_path(location)
-        if not location:
+        if not project.pipfile_exists:


So this isn’t different fundamentally. pipfile_exists is just bool(pipfile_location). It is arguably cleaner so it seems ok

techalchemy · 2018-12-19T07:06:05Z

pipenv/core.py

+        location = project.pipfile_location
+        # Shorten the virtual display of the path to the virtualenv.
+        if not bare:
+            location = shorten_path(location)


if not bare...elif not bare seems like it might be a mistake

techalchemy · 2018-12-19T07:07:57Z

pipenv/project.py

@@ -53,7 +53,7 @@ def _normalized(p):
        path_str = matches and matches[0] or str(loc)
    else:
        path_str = str(loc)
-    return normalize_drive(path_str)
+    return normalize_drive(os.path.abspath(path_str))


Don’t change this one, it is going to regress environment discovery on windows unless you change every reference to it

a relative path also breaks https://github.com/pypa/pipenv/blob/master/pipenv/project.py#L217-L221

techalchemy · 2018-12-19T07:15:43Z

pipenv/project.py

@@ -222,7 +222,7 @@ def name(self):

    @property
    def pipfile_exists(self):
-        return bool(self.pipfile_location)
+        return os.path.isfile(self.pipfile_location)


Don’t use isfile. I know it seems intuitive but this will break pipenv for lots of people. Changing things like this en mass for no reason isn’t going to fly. This library is downloaded millions of times a month. Even if 1% of the users set their Pipfile to a network path or use the envvar to point at a temp file that doesn’t exist to work around some issue, you just fundamentally broke pipenv for those users because of this change.

Improvements are good and important and I’m willing to break compatibility for some things. But changes like this one only screw users over and make people like Hynek write about how many regressions they experience

This change is made to work with #3386 (diff)

techalchemy · 2018-12-19T07:20:50Z

pipenv/project.py

@@ -470,7 +465,7 @@ def pipfile_location(self):
            try:
                loc = pipfile.Pipfile.find(max_depth=PIPENV_MAX_DEPTH)
            except RuntimeError:
-                loc = None
+                loc = "Pipfile"


I’m pretty sure this returns an actual path like project.path_to("Pipfile")

In case the Pipfile is not found it returns None. Then https://github.com/pypa/pipenv/blob/master/pipenv/project.py#L217-L221 will be broken.

news/3386.trivial.rst

techalchemy · 2019-04-01T06:15:15Z

If you're comfortable with this and feel it's well tested I'm good with merging it

2020.5.28 (2020-05-28) ====================== Features & Improvements ----------------------- - `pipenv install` and `pipenv sync` will no longer attempt to install satisfied dependencies during installation. pypa#3057, pypa#3506 - Added support for resolution of direct-url dependencies in `setup.py` files to respect `PEP-508` style URL dependencies. pypa#3148 - Added full support for resolution of all dependency types including direct URLs, zip archives, tarballs, etc. - Improved error handling and formatting. - Introduced improved cross platform stream wrappers for better `stdout` and `stderr` consistency. pypa#3298 - For consistency with other commands and the `--dev` option description, `pipenv lock --requirements --dev` now emits both default and development dependencies. The new `--dev-only` option requests the previous behaviour (e.g. to generate a `dev-requirements.txt` file). pypa#3316 - Pipenv will now successfully recursively lock VCS sub-dependencies. pypa#3328 - Added support for `--verbose` output to `pipenv run`. pypa#3348 - Pipenv will now discover and resolve the intrinsic dependencies of **all** VCS dependencies, whether they are editable or not, to prevent resolution conflicts. pypa#3368 - Added a new environment variable, `PIPENV_RESOLVE_VCS`, to toggle dependency resolution off for non-editable VCS, file, and URL based dependencies. pypa#3577 - Added the ability for Windows users to enable emojis by setting `PIPENV_HIDE_EMOJIS=0`. pypa#3595 - Allow overriding `PIPENV_INSTALL_TIMEOUT` environment variable (in seconds). pypa#3652 - Allow overriding `PIP_EXISTS_ACTION` evironment variable (value is passed to pip install). Possible values here: <https://pip.pypa.io/en/stable/reference/pip/#exists-action-option> Useful when you need to `PIP\_EXISTS\_ACTION=i` (ignore existing packages) - great for CI environments, where you need really fast setup. pypa#3738 - Pipenv will no longer forcibly override `PIP_NO_DEPS` on all vcs and file dependencies as resolution happens on these in a pre-lock step. pypa#3763 - Improved verbose logging output during `pipenv lock` will now stream output to the console while maintaining a spinner. pypa#3810 - Added support for automatic python installs via `asdf` and associated `PIPENV_DONT_USE_ASDF` environment variable. pypa#4018 - Pyenv/asdf can now be used whether or not they are available on PATH. Setting `PYENV_ROOT`/`ASDF_DIR` in a `.env` file allows Pipenv to install an interpreter without any shell customizations, so long as pyenv/asdf is installed. pypa#4245 - Added `--key` command line parameter for including personal PyUp.io API tokens when running `pipenv check`. pypa#4257 Behavior Changes ---------------- - Make conservative checks of known exceptions when subprocess returns output, so user won\'t see the whole traceback - just the error. pypa#2553 - Do not touch Pipfile early and rely on it so that one can do `pipenv sync` without a Pipfile. pypa#3386 - Re-enable `--help` option for `pipenv run` command. pypa#3844 - Make sure `pipenv lock -r --pypi-mirror {MIRROR_URL}` will respect the pypi-mirror in requirements output. pypa#4199 Bug Fixes --------- - Raise `PipenvUsageError` when \[\[source\]\] does not contain url field. pypa#2373 - Fixed a bug which caused editable package resolution to sometimes fail with an unhelpful setuptools-related error message. pypa#2722 - Fixed an issue which caused errors due to reliance on the system utilities `which` and `where` which may not always exist on some systems. - Fixed a bug which caused periodic failures in python discovery when executables named `python` were not present on the target `$PATH`. pypa#2783 - Dependency resolution now writes hashes for local and remote files to the lockfile. pypa#3053 - Fixed a bug which prevented `pipenv graph` from correctly showing all dependencies when running from within `pipenv shell`. pypa#3071 - Fixed resolution of direct-url dependencies in `setup.py` files to respect `PEP-508` style URL dependencies. pypa#3148 - Fixed a bug which caused failures in warning reporting when running pipenv inside a virtualenv under some circumstances. - Fixed a bug with package discovery when running `pipenv clean`. pypa#3298 - Quote command arguments with carets (`^`) on Windows to work around unintended shell escapes. pypa#3307 - Handle alternate names for UTF-8 encoding. pypa#3313 - Abort pipenv before adding the non-exist package to Pipfile. pypa#3318 - Don\'t normalize the package name user passes in. pypa#3324 - Fix a bug where custom virtualenv can not be activated with pipenv shell pypa#3339 - Fix a bug that `--site-packages` flag is not recognized. pypa#3351 - Fix a bug where `pipenv --clear` is not working pypa#3353 - Fix unhashable type error during `$ pipenv install --selective-upgrade` pypa#3384 - Dependencies with direct `PEP508` compliant VCS URLs specified in their `install_requires` will now be successfully locked during the resolution process. pypa#3396 - Fixed a keyerror which could occur when locking VCS dependencies in some cases. pypa#3404 - Fixed a bug that `ValidationError` is thrown when some fields are missing in source section. pypa#3427 - Updated the index names in lock file when source name in Pipfile is changed. pypa#3449 - Fixed an issue which caused `pipenv install --help` to show duplicate entries for `--pre`. pypa#3479 - Fix bug causing `[SSL: CERTIFICATE_VERIFY_FAILED]` when Pipfile `[[source]]` has `verify_ssl=false` and url with custom port. pypa#3502 - Fix `sync --sequential` ignoring `pip install` errors and logs. pypa#3537 - Fix the issue that lock file can\'t be created when `PIPENV_PIPFILE` is not under working directory. pypa#3584 - Pipenv will no longer inadvertently set `editable=True` on all vcs dependencies. pypa#3647 - The `--keep-outdated` argument to `pipenv install` and `pipenv lock` will now drop specifier constraints when encountering editable dependencies. - In addition, `--keep-outdated` will retain specifiers that would otherwise be dropped from any entries that have not been updated. pypa#3656 - Fixed a bug which sometimes caused pipenv to fail to respect the `--site-packages` flag when passed with `pipenv install`. pypa#3718 - Normalize the package names to lowercase when comparing used and in-Pipfile packages. pypa#3745 - `pipenv update --outdated` will now correctly handle comparisons between pre/post-releases and normal releases. pypa#3766 - Fixed a `KeyError` which could occur when pinning outdated VCS dependencies via `pipenv lock --keep-outdated`. pypa#3768 - Resolved an issue which caused resolution to fail when encountering poorly formatted `python_version` markers in `setup.py` and `setup.cfg` files. pypa#3786 - Fix a bug that installation errors are displayed as a list. pypa#3794 - Update `pythonfinder` to fix a problem that `python.exe` will be mistakenly chosen for virtualenv creation under WSL. pypa#3807 - Fixed several bugs which could prevent editable VCS dependencies from being installed into target environments, even when reporting successful installation. pypa#3809 - `pipenv check --system` should find the correct Python interpreter when `python` does not exist on the system. pypa#3819 - Resolve the symlinks when the path is absolute. pypa#3842 - Pass `--pre` and `--clear` options to `pipenv update --outdated`. pypa#3879 - Fixed a bug which prevented resolution of direct URL dependencies which have PEP508 style direct url VCS sub-dependencies with subdirectories. pypa#3976 - Honor `PIPENV_SPINNER` environment variable pypa#4045 - Fixed an issue with `pipenv check` failing due to an invalid API key from `pyup.io`. pypa#4188 - Fixed a bug which caused versions from VCS dependencies to be included in `Pipfile.lock` inadvertently. pypa#4217 - Fixed a bug which caused pipenv to search non-existent virtual environments for `pip` when installing using `--system`. pypa#4220 - `Requires-Python` values specifying constraint versions of python starting from `1.x` will now be parsed successfully. pypa#4226 - Fix a bug of `pipenv update --outdated` that can\'t print output correctly. pypa#4229 - Fixed a bug which caused pipenv to prefer source distributions over wheels from `PyPI` during the dependency resolution phase. Fixed an issue which prevented proper build isolation using `pep517` based builders during dependency resolution. pypa#4231 - Don\'t fallback to system Python when no matching Python version is found. pypa#4232 Vendored Libraries ------------------ - Updated `pip_shims` to support `--outdated` with new pip versions. pypa#3766 - Update vendored dependencies and invocations - Update vendored and patched dependencies - Update patches on `piptools`, `pip`, `pip-shims`, `tomlkit` - Fix invocations of dependencies - Fix custom `InstallCommand` instantiation - Update `PackageFinder` usage - Fix `Bool` stringify attempts from `tomlkit` - Updated vendored dependencies: - **attrs**: `18.2.0 => `19.1.0` - **certifi**: `2018.10.15 => `2019.3.9` - **cached\_property**: `1.4.3 => `1.5.1` - **cerberus**: `1.2.0 => `1.3.1` - **click**: `7.0.0 => `7.1.1` - **click-completion**: `0.5.0 => `0.5.1` - **colorama**: `0.3.9 => `0.4.3` - **contextlib2**: `(new) => `0.6.0.post1` - **distlib**: `0.2.8 => `0.2.9` - **funcsigs**: `(new) => `1.0.2` - **importlib\_metadata** `1.3.0 => `1.5.1` - **importlib-resources**: `(new) => `1.4.0` - **idna**: `2.7 => `2.9` - **jinja2**: `2.10.0 => `2.11.1` - **markupsafe**: `1.0 => `1.1.1` - **more-itertools**: `(new) => `5.0.0` - **orderedmultidict**: `(new) => `1.0` - **packaging**: `18.0 => `19.0` - **parse**: `1.9.0 => `1.15.0` - **pathlib2**: `2.3.2 => `2.3.3` - **pep517**: `(new) => `0.5.0` - **pexpect**: `4.6.0 => `4.8.0` - **pip-shims**: `0.2.0 => `0.5.1` - **pipdeptree**: `0.13.0 => `0.13.2` - **pyparsing**: `2.2.2 => `2.4.6` - **python-dotenv**: `0.9.1 => `0.10.2` - **pythonfinder**: `1.1.10 => `1.2.2` - **pytoml**: `(new) => `0.1.20` - **requests**: `2.20.1 => `2.23.0` - **requirementslib**: `1.3.3 => `1.5.4` - **scandir**: `1.9.0 => `1.10.0` - **shellingham**: `1.2.7 => `1.3.2` - **six**: `1.11.0 => `1.14.0` - **tomlkit**: `0.5.2 => `0.5.11` - **urllib3**: `1.24 => `1.25.8` - **vistir**: `0.3.0 => `0.5.0` - **yaspin**: `0.14.0 => `0.14.3` - **zipp**: `0.6.0` - Removed vendored dependency **cursor**. pypa#4169 - Add and update vendored dependencies to accommodate `safety` vendoring: - **safety** `(none)` => `1.8.7` - **dparse** `(none)` => `0.5.0` - **pyyaml** `(none)` => `5.3.1` - **urllib3** `1.25.8` => `1.25.9` - **certifi** `2019.11.28` => `2020.4.5.1` - **pyparsing** `2.4.6` => `2.4.7` - **resolvelib** `0.2.2` => `0.3.0` - **importlib-metadata** `1.5.1` => `1.6.0` - **pip-shims** `0.5.1` => `0.5.2` - **requirementslib** `1.5.5` => `1.5.6` pypa#4188 - Updated vendored `pip` => `20.0.2` and `pip-tools` => `5.0.0`. pypa#4215 - Updated vendored dependencies to latest versions for security and bug fixes: - **requirementslib** `1.5.8` => `1.5.9` - **vistir** `0.5.0` => `0.5.1` - **jinja2** `2.11.1` => `2.11.2` - **click** `7.1.1` => `7.1.2` - **dateutil** `(none)` => `2.8.1` - **backports.functools\_lru\_cache** `1.5.0` => `1.6.1` - **enum34** `1.1.6` => `1.1.10` - **toml** `0.10.0` => `0.10.1` - **importlib\_resources** `1.4.0` => `1.5.0` pypa#4226 - Changed attrs import path in vendored dependencies to always import from `pipenv.vendor`. pypa#4267 Improved Documentation ---------------------- - Added documenation about variable expansion in `Pipfile` entries. pypa#2317 - Consolidate all contributing docs in the rst file pypa#3120 - Update the out-dated manual page. pypa#3246 - Move CLI docs to its own page. pypa#3346 - Replace (non-existant) video on docs index.rst with equivalent gif. pypa#3499 - Clarify wording in Basic Usage example on using double quotes to escape shell redirection pypa#3522 - Ensure docs show navigation on small-screen devices pypa#3527 - Added a link to the TOML Spec under General Recommendations & Version Control to clarify how Pipfiles should be written. pypa#3629 - Updated the documentation with the new `pytest` entrypoint. pypa#3759 - Fix link to GIF in README.md demonstrating Pipenv\'s usage, and add descriptive alt text. pypa#3911 - Added a line describing potential issues in fancy extension. pypa#3912 - Documental description of how Pipfile works and association with Pipenv. pypa#3913 - Clarify the proper value of `python_version` and `python_full_version`. pypa#3914 - Write description for `--deploy` extension and few extensions differences. pypa#3915 - More documentation for `.env` files pypa#4100 - Updated documentation to point to working links. pypa#4137 - Replace docs.pipenv.org with pipenv.pypa.io pypa#4167 - Added functionality to check spelling in documentation and cleaned up existing typographical issues. pypa#4209

frostming force-pushed the improve-message branch from c0c4683 to 67e7ab5 Compare December 18, 2018 05:17

Decouple project from the existence of Pipfile.

c9041b8

frostming changed the title ~~Improve message for working under root directory.~~ Decouple project from the existence of Pipfile. Dec 19, 2018

frostming force-pushed the improve-message branch from c149009 to c9041b8 Compare December 19, 2018 02:36

make path absolute

c17f36f

techalchemy reviewed Dec 19, 2018

View reviewed changes

frostming commented Dec 19, 2018

View reviewed changes

techalchemy requested changes Dec 19, 2018

View reviewed changes

remove redundant branch

0efa7e9

roman-parkhunovskyi mentioned this pull request Feb 26, 2019

pipenv fails to install packages on a Jenkins slave #3564

Closed

frostming mentioned this pull request Mar 7, 2019

Don't rely on pipfile_location to determine lockfile_location #3589

Closed

techalchemy reviewed Apr 1, 2019

View reviewed changes

news/3386.trivial.rst Outdated Show resolved Hide resolved

frostming added 2 commits April 1, 2019 12:52

merge branch 'master'

08b3571

Refine the news entry

09e1d55

techalchemy added Type: Enhancement 💡 This is a feature or enhancement request. Priority: Low This item is low priority and may not be looked at in the next few release cycles. Status: Accepted ✔️ This item has been reviewed and accepted. labels Apr 1, 2019

techalchemy added this to the March Release milestone Apr 1, 2019

frostming merged commit 8aa9283 into master Apr 1, 2019

frostming deleted the improve-message branch April 1, 2019 08:46

gps035 mentioned this pull request May 28, 2020

Pipenv no longer working under the root directory is not listed as a breaking change #4273

Closed

frostming mentioned this pull request May 28, 2020

Don't check against root directory #4275

Merged

2 tasks

ulrikjohansson mentioned this pull request Nov 14, 2020

pipenv sync fails without Pipfile present #2834

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Decouple project from the existence of Pipfile. #3386

Decouple project from the existence of Pipfile. #3386

frostming commented Dec 17, 2018 •

edited

Loading

techalchemy left a comment

techalchemy commented Dec 19, 2018

frostming commented Dec 19, 2018 •

edited

Loading

frostming Dec 19, 2018

frostming Dec 19, 2018

frostming Dec 19, 2018

frostming Dec 19, 2018

frostming Dec 19, 2018

techalchemy left a comment

techalchemy Dec 19, 2018

techalchemy Dec 19, 2018

techalchemy Dec 19, 2018

techalchemy Dec 19, 2018

frostming Dec 19, 2018 •

edited

Loading

techalchemy Dec 19, 2018

frostming Dec 19, 2018 •

edited

Loading

techalchemy Dec 19, 2018

frostming Dec 19, 2018 •

edited

Loading

techalchemy commented Apr 1, 2019

Decouple project from the existence of Pipfile. #3386

Decouple project from the existence of Pipfile. #3386

Conversation

frostming commented Dec 17, 2018 • edited Loading

techalchemy left a comment

Choose a reason for hiding this comment

techalchemy commented Dec 19, 2018

frostming commented Dec 19, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

techalchemy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

frostming Dec 19, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

frostming Dec 19, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

frostming Dec 19, 2018 • edited Loading

Choose a reason for hiding this comment

techalchemy commented Apr 1, 2019

frostming commented Dec 17, 2018 •

edited

Loading

frostming commented Dec 19, 2018 •

edited

Loading

frostming Dec 19, 2018 •

edited

Loading

frostming Dec 19, 2018 •

edited

Loading

frostming Dec 19, 2018 •

edited

Loading