feat: Support linting in out-of-source directories

[gremat]

Type of Changes

	Type
✓	✨ New feature

Description

Linting files/modules that import external modules works just fine by adding the paths containing those modules to PYTHONPATH. However, there are some well-known situations where a mostly automated inclusion of certain directories is beneficial. E.g., in a python project setup where source files are located in src/, you want to lint the tests located in tests/ without import errors for the main module.
Similar to other python projects, like pytest, this PR introduces the configuration option main.pythonpath where you can specify absolute and, especially, relative paths that will always be included to sys.path when linting.

Notes on related issues:

• Main intention of the PR was to get rid of the usual way to solve this problem which is described in Document the setting of the sys/PYTHONPATH in our doc and on a popular stackoverflow answer #9507, i.e., use an init-hook.
• There was a related attempt in Patch sys.path on a per-file basis #7357 which, by now, is stale because it was introduced before the support for implicit namespace packages 71b6325 was merged.
• The PR is a prerequisite for Pylint is not using a src/ folder #5644, where an init-hook is most likely not an option.
• There are multiple other issues that are concerned with sys.path manipulations, e.g., Linting more than one dir or package, can hide errors or cause wrong errors #2966, Support an Option to Disable sys.path patching #5226, to name a few. Of course, caution is advised in general for those manipulations as they alter the global state. However, I don't see immediate problems with the implementations. Please speak up, if you do.

Refs #9507
Refs #7357
Refs #5644

P.S.: Credits to the authors of 71b6325 which made this implementation straight forward.

EDIT:
In a first version, I've added --pythonpath in Pyreverse, too (though corrupted because unfortunately I missed that Pyreverse does not share config with the linter). But I gave it another thought and I don't see the point of integrating pythonpath there, too. So I will leave it out until someone argues in favor for it.

[gremat]

FYI: Changelog checks fails because of: actions/setup-python#886 .

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.85%. Comparing base (3f1f7b8) to head (3f62568).
Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #9721   +/-   ##
=======================================
  Coverage   95.84%   95.85%           
=======================================
  Files         174      174           
  Lines       18865    18868    +3     
=======================================
+ Hits        18082    18085    +3     
  Misses        783      783           

Files	Coverage Δ
pylint/lint/__init__.py	91.66% <100.00%> (ø)
pylint/lint/base_options.py	100.00% <ø> (ø)
pylint/lint/parallel.py	92.85% <100.00%> (ø)
pylint/lint/pylinter.py	96.66% <100.00%> (+<0.01%)	⬆️
pylint/lint/utils.py	96.07% <100.00%> (+0.16%)	⬆️

[github-actions]

🤖 According to the primer, this change has no effect on the checked open source code. 🤖🎉

This comment was generated for commit 3f62568

[DanielNoord]

I don't really understand why we would provide another method to manipulate PYTHONPATH when users have multiple ways to do so.

I understand that we want to make this easier for users, but as far as I understand it setting up PYTHONPATH should be done before you call into your Python executable. The init-hook is the earliest thing that gets executed so that is probably the best place to get this behaviour? What was your reason for not documenting the current way to do this but add another option?

By the way, some of the PRs you link are not really related to this. For example, #5644 is just an issue with the project structure and not really related to any of the issues you mention in this PR.

[gremat]

By the way, some of the PRs you link are not really related to this. For example, #5644 is just an issue with the project structure and not really related to any of the issues you mention in this PR.

Actually, #5644 provides a very prominent - because it's pylint itself! - example for the benefit of my suggested changes. As far as I can see, and please correct me, if I'm wrong, as soon as pylint's code is moved to the src folder (as suggested in #5644), pylint's own pylintrc will have to do something about the import situation if you do not want to see import errors when editing files in tests. Currently, this would be the init-hook workaround as you mentioned.

I understand that we want to make this easier for users, but as far as I understand it setting up PYTHONPATH should be done before you call into your Python executable. The init-hook is the earliest thing that gets executed so that is probably the best place to get this behaviour? What was your reason for not documenting the current way to do this but add another option?

Yes, init-hook might be one of the first places to hook up into. However, I consider this workaround to be rather hacky and ugly. Compare, e.g., solutions of other parties like pytest, mypy, isort or setuptools which all provide a nice and clean way with a specific argument/configuration option to tackle this standard project layout. (Nowadays, that is; see link from #5644 for older alternatives.)
That said, my reason to add another option is mainly to provide a nice, clean and similarly-looking option in pylint, too. It will get easier for (new) users, in fact, easy enough that you almost don't need to consult the documentation/help output and could easily guesstimate the required option in your pyproject.toml.

I don't really understand why we would provide another method to manipulate PYTHONPATH when users have multiple ways to do so.

For pylint, I am only aware of --source-roots to manipulate PYTHONPATH but it serves a different purpose because it is related to implicit namespace packages and does not help with the init-hook vs. suggested option.
You are of course right that you could set PYTHONPATH externally, e.g., PYTHONPATH=./src pylint. But I see two problems with that. Firstly, you will need to convince your editor/IDE of that which is again error-prone, not out-of-the-box and you have to do that for every editor you use (moreover, might be even required on a per-project basis). Secondly, you might very well get into trouble finding and specifying the right location which depends on your current working directory, the file location, the project root and from where exactly it will be called.
Thus I don't consider the external option to be a viable approach.

[DanielNoord]

Actually, #5644 provides a very prominent - because it's pylint itself! - example for the benefit of my suggested changes.

I'm not sure if this is correct. Wouldn't this be solved by doing a pip install -e . before calling pylint? In that setup, the tests directory won't be able to import from pylint unless it is installed. Changing the PYTHONPATH to act as if it can seems incorrect.

Compare, e.g., solutions of other parties

I think the mypy example is exactly what frightens me from adding this. The amount of configuration and gotcha's on different platforms seems like a headache. For example, why use globs and not regular expressions like we do for other configuration options.
What I personally like about the init-hook is that we don't need to provide any "support". We can point users to common patterns/commands that might satisfy their needs, but there is no reason to look into our code to see whether we automatically expand home directories (to take the example from mypy). It's just Python code.

For pylint, I am only aware of --source-roots to manipulate PYTHONPATH but it serves a different purpose because it is related to implicit namespace packages and does not help with the init-hook vs. suggested option. You are of course right that you could set PYTHONPATH externally, e.g., PYTHONPATH=./src pylint. But I see two problems with that. Firstly, you will need to convince your editor/IDE of that which is again error-prone, not out-of-the-box and you have to do that for every editor you use (moreover, might be even required on a per-project basis). Secondly, you might very well get into trouble finding and specifying the right location which depends on your current working directory, the file location, the project root and from where exactly it will be called. Thus I don't consider the external option to be a viable approach.

I understand your point, and can see how it is far from optimal. On the other hand, the same argument as I provided above applies here. We can tap into and refer to existing documentation for setting PYTHONPATH for other tools and IDEs. Making our own custom solution increases the scope for bugs and edge cases we didn't consider.

I want to stress that I really do understand where you are coming from and see this as a nice feature for most users. However, from the perspective of a maintainers it seems to invite more edge cases and bugs to our tool without providing any features that the currently available features don't already provide. Perhaps I missed this so let me ask explicitly: is there anything that this solution allows you to do that you can't using the latest version of pylint? If there is then that might convince me to be more open to this change!