fix: Migrate tests out of src for path protection

[matthewfeickert]

Addresses the other part of Issue #15 mentioned in #15 (comment).

For the benefits of a src/ directory layout to work the package inside src/ needs to be isolated from the tests, so there should be a top level directory structure like:

$ tree
.
├── src
│   └── example_package
└── tests

The reason being that if tests is alongside example_package under src

$ tree
.
└── src
    ├── example_package
    └── tests

then when the tests run the version of example_package that will be imported is the code that is currently under src/example_package, whatever its current state and not the installed version of the package, which is the thing you want to be testing.

Pull request type

Please check the type of change your PR introduces:

• Bugfix

What is the current behavior?

The tests are always testing the state of src/the_well_maintained_test, regardless of if this is intentional or not, and not able to test the state of any installed code. Here's an example that show this (all commands are run inside of a python:3.9 Docker container):

root@d9a72199c19b:/# python -m venv venv && . venv/bin/activate
(venv) root@d9a72199c19b:/# git clone https://github.com/ryancheley/the-well-maintained-test.git && cd the-well-maintained-test
Cloning into 'the-well-maintained-test'...
remote: Enumerating objects: 392, done.
remote: Counting objects: 100% (392/392), done.
remote: Compressing objects: 100% (207/207), done.
remote: Total 392 (delta 224), reused 336 (delta 179), pack-reused 0
Receiving objects: 100% (392/392), 77.53 KiB | 1.55 MiB/s, done.
Resolving deltas: 100% (224/224), done.
(venv) root@d9a72199c19b:/the-well-maintained-test# python -m pip --quiet install --upgrade pip setuptools wheel
(venv) root@d9a72199c19b:/the-well-maintained-test# python -m pip --quiet install .[test]  # Intentionally install the package not as editable
(venv) root@d9a72199c19b:/the-well-maintained-test# pip show the-well-maintained-test  # shows installed in site-packages as expected and not local
Name: the-well-maintained-test
Version: 0.6.0
Summary: Programatically tries to answer the 12 questions from         Adam Johnson's blog post https://adamj.eu/tech/2021/11/04/the-well-maintained-test/
Home-page: https://github.com/ryancheley/the-well-maintained-test
Author: Ryan Cheley
Author-email: 
License: Apache License, Version 2.0
Location: /venv/lib/python3.9/site-packages
Requires: click, requests, rich
Required-by: 
(venv) root@d9a72199c19b:/the-well-maintained-test# pytest  # pytest passes as expected
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.9.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /the-well-maintained-test
collected 41 items                                                                                                                                                                                        

src/tests/test_the_well_maintained_test.py .........................................                                                                                                                [100%]

=========================================================================================== 41 passed in 0.26s ============================================================================================
(venv) root@d9a72199c19b:/the-well-maintained-test# vi src/the_well_maintained_test/cli.py  # Now let's put in a breakpoint() into the local code, which is _not_ the installed code
(venv) root@d9a72199c19b:/the-well-maintained-test# git diff
diff --git a/src/the_well_maintained_test/cli.py b/src/the_well_maintained_test/cli.py
index 53c366c..0ba1485 100644
--- a/src/the_well_maintained_test/cli.py
+++ b/src/the_well_maintained_test/cli.py
@@ -27,6 +27,8 @@ from .utils import (
     well_used,
 )
 
+breakpoint()
+
 console = Console(record=True)
 question_style = "bold blue"
 answer_style = "italic"
(venv) root@d9a72199c19b:/the-well-maintained-test# pytest  # pytest now hits the breakpoint because the tests are importing the _local_ code and not the installed code we wanted!
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.9.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /the-well-maintained-test
collecting ... 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /the-well-maintained-test/src/the_well_maintained_test/cli.py(32)<module>()
-> console = Console(record=True)
(Pdb) 
# KeyboardIntterupt out of the breakpoint
(venv) root@d9a72199c19b:/the-well-maintained-test# git mv src/tests .  # move the tests out of src so don't see the_well_maintained_test in same local path
(venv) root@d9a72199c19b:/the-well-maintained-test# pytest  # pytest doesn't see the breakpoint as desired as the installed code is being tested. :+1:
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.9.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /the-well-maintained-test
collected 41 items                                                                                                                                                                                        

tests/test_the_well_maintained_test.py .........................................                                                                                                                    [100%]

=========================================================================================== 41 passed in 0.23s ============================================================================================

Note that if we intentionally wanted to be testing the local code (e.g. doing a dev workflow of using an editable install) then we hit the breakpoint() again as expected and desired

(venv) root@d9a72199c19b:/the-well-maintained-test# python -m pip --quiet install --upgrade -e .
(venv) root@d9a72199c19b:/the-well-maintained-test# pytest
=========================================================================================== test session starts ===========================================================================================
platform linux -- Python 3.9.9, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /the-well-maintained-test
collecting ... 
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /the-well-maintained-test/src/the_well_maintained_test/cli.py(32)<module>()
-> console = Console(record=True)
(Pdb) 

👍

What is the new behavior?

Ensuring that the version of the code tested is the installed version and not the state of whatever is in src/the_well_maintained_test. You can of course still test the state of src/the_well_maintained_test with an editable install (which is a very reasonable thing to want to do!) but you also want to make sure that you can test the installed version properly.

[matthewfeickert]

This PR is ready for review. I wrote this pretty fast so I'm happy to try to clarify anything!

[ryancheley]

@matthewfeickert it looks like there's still a tests folder in src with the file test_the_well_maintained_test.py. Is that intentional? Does it need to be there, or can / should it be removed? Thanks!

[matthewfeickert]

it looks like there's still a tests folder in src with the file test_the_well_maintained_test.py. Is that intentional?

Can you check that you've pulled from main to get this PR's changes? I've synced my fork with the HEAD of main (upstream for me) and I see things migrated okay:

$ git branch
* main
$ git pull upstream main 
From git://github.com/ryancheley/the-well-maintained-test
 * branch            main       -> FETCH_HEAD
Current branch main is up to date.
$ git ls-files -- src/
src/the_well_maintained_test/__init__.py
src/the_well_maintained_test/cli.py
src/the_well_maintained_test/data/__init__.py
src/the_well_maintained_test/data/questions.json
src/the_well_maintained_test/helpers.py
src/the_well_maintained_test/utils.py
$ git ls-files -- tests/
tests/__init__.py
tests/test_classes.py
tests/test_the_well_maintained_test.py