flake8 config error

[ivirshup]

There is a space in front of the per-file-ignore entry in the flake8 config (added in scverse/scanpy#1689). This is the same in scanpy, as the config here was just copied from there. This causes a parse failure which leads to the the per-file-ignores becoming global instead. To demonstrate:

Parse failure

From the root of the repo

>>> from flake8.api.legacy import get_style_guide
>>> get_style_guide().options.ignore
['#',
 'module',
 'imported',
 'but',
 'unused',
 '->',
 'required',
 'for',
 'Scanpys',
 'API',
 'F401',
 'W503',
 'W504',
 'E501',
 'E203',
 'E231',
 'E402',
 'E126',
 'E262',
 'E266',
 'E731',
 'E741',
 'per-file-ignores',
 '=',
 'tests/test*.py:',
 'F811',
 'anndata/compat/_overloaded_dict.py:',
 'F821',
 'anndata/tests/test_readwrite.py:',
 'E721']

Fixing the parse error reveals a number of flake8 errors which were accidentally being ignored. I came across this while adding back F401, since it should only be ignored for files which define a namespace.

It is kind of ironic that a tool for enforcement of code style is permissive to a fault about config parsing.

(ping @Zethson)

[flying-sheep]

@asottile can you help making such bugs impossible in the future? Implementing PyCQA/flake8#234 might be an easy way to do it.

[Zethson]

Nice catch @ivirshup. Sorry for this. I'll fix the config file.

[ivirshup]

@flying-sheep has the first issue here been resolved PyCQA/flake8#234 (comment)? If not, I don't think this is going to happen anytime soon. Even after this is implemented for flake8, other tools which read the flake8 config would need to recognize the pyproject.toml entries (e.g. vscode, autopep8, etc.).

@Zethson no worries, all three of us managed to miss this. I'd really expect this sort of thing to cause a parse error.

[asottile]

really, this was all just a ploy to @ mention me about the pyproject.toml issue? welcome to my blocklist

[flying-sheep]

@ivirshup yes it has been. pyproject.toml needs to have the build-system table to change pip’s behavior.

@asottile No, but I’m sorry I touched an apparently very emotional issue for you. I mentioned it because it’s the easy way. Manual validation would be another option that’s harder to implement. But if you really blocked me you’re not going to read this anyway …

[asottile]

yes it has been. pyproject.toml needs to have the build-system table to change pip’s behavior.

it has not, the presence of the file alone still changes pip's behaviour in 21.0.1

[flying-sheep]

Oh interesting! I thought this was about when the presence of pyproject.toml made pip assume it’s not a legacy build (and therefore made it not use setup.py). They fixed that one long ago: Unless the build-system table is there, pip just assumes a setup.py build.

/edit: Ah found it. pypa/pip#8437 (comment)

So basically the’re saying that the only code path that changes is very much deprecated. Therefore, if I understand correctly, the behavior that happens when pyproject.toml exists is the one that should be used, and projects should switch to a standard way of building instead of sticking with “just setup.py”. Did I get this right?

Direct installs

What does have an impact here is the even older ("legacy legacy", if you like 🙂) behaviour of installing directly with setup.py install. We've been trying to kill that behaviour for years now - it bypasses all modern standards and behaviour, and uses unsafe and generally deprecated mechanisms. Removal of that behaviour generally gets discussed under the term "install via wheel".

The thing is, if you are doing a direct install, build isolation is irrelevant, because we don't do a build! So anything that uses the direct-install code path doesn't use build isolation. This is (I believe) the code path that is being taken by projects that are complaining that adding pyproject.toml imposes build isolation on them.

Important part:

But what I am saying is that every part of this behaviour is in code that we've been trying to deprecate and remove for at least 3 years, and probably longer. We're retaining the old behaviour because we don't have a good way of warning users that change is coming - so we have to make the transition extremely long. But "fixing" breakages where users have encountered the change and want some of the benefits but don't want to complete the work to switch, is deliberately extending the transition period, and does nothing to help us achieve the ultimate goal of removing all of that old code.

So I guess when they finally delete that code and only the new behavior exists, there will be no differential behavior anymore.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity.
Please add a comment if you want to keep the issue open. Thank you for your contributions!

[flying-sheep]

Fixed in #863 and then superseded by #891.

Anyone interested in isolated builds in pip can go here: pypa/pip#9175