[bug] conan test doesn't work with python_requires packages

[samuel-emrys]

Environment details

• Operating System+version: Arch Linux
• Compiler+version: clang 16
• Conan version: 2.0.17
• Python version: 3.11.1

Steps to reproduce

Using the following files:

• conanfile.py

from conan import ConanFile

class FuncsPackage(ConanFile):
    name = "funcs"
    version = "0.1.0"
    package_type = "python-require"

def func_one(args):
    if args:
        return "Args passed"
    return "No arguments"

• test_package/conanfile.py

from conan import ConanFile
from conan.tools.layout import basic_layout
from conan.tools.build import cross_building

class PyvenvTestConan(ConanFile):
    settings = "os", "build_type", "arch", "compiler"
    apply_env = False
    test_type = "explicit"
    build_policy = "missing"

    def config_options(self):
        del self.settings.build_type
        del self.settings.arch
        del self.settings.compiler

    def layout(self):
        basic_layout(self)

    def test(self):
        if not cross_building(self):
            func_one = self.python_requires["funcs"].module.func_one
            result = func_one("These are the arguments")
            self.run("echo 'Result: {}'".format(result), env="conanrun")

1. Execute conan create . --user user - this will work correctly
2. Execute conan test test_package funcs/0.1.0@user, the below error message will be thrown.

Logs

$ conan create . --user user

======== Exporting recipe to the cache ========
funcs/0.1.0@user: Exporting package recipe: /tmp/pyrequires/conanfile.py
funcs/0.1.0@user: Copied 1 '.py' file: conanfile.py
funcs/0.1.0@user: Exported to cache folder: /home/user/.conan2/p/funcs9df5001c34428/e
funcs/0.1.0@user: Exported: funcs/0.1.0@user#8fb741c2645c0dd35d1db5c5b7cc49c3 (2024-01-20 01:38:45 UTC)

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}


======== Computing dependency graph ========

======== Launching test_package ========

======== Computing dependency graph ========
Graph root
    funcs/0.1.0@user (test package): /tmp/pyrequires/test_package/conanfile.py
Python requires
    funcs/0.1.0@user#8fb741c2645c0dd35d1db5c5b7cc49c3 - Cache

======== Computing necessary packages ========

======== Installing packages ========

======== Testing the package ========
Removing previously existing 'test_package' build folder: /tmp/pyrequires/test_package/build
funcs/0.1.0@user (test package): Test package build: build
funcs/0.1.0@user (test package): Test package build folder: /tmp/pyrequires/test_package/build
funcs/0.1.0@user (test package): Generating aggregated env files
funcs/0.1.0@user (test package): Generated aggregated env files: ['conanbuild.sh', 'conanrun.sh']

======== Testing the package: Building ========

======== Testing the package: Executing test ========
funcs/0.1.0@user (test package): Running test()
funcs/0.1.0@user (test package): RUN: echo 'Result: Args passed'
Result: Args passed

$ conan test test_package funcs/0.1.0@user
======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}


======== Launching test_package ========

======== Computing dependency graph ========
Graph root
    funcs/0.1.0@user (test package): /tmp/pyrequires/test_package/conanfile.py

======== Computing necessary packages ========

======== Installing packages ========

======== Testing the package ========
ERROR: The conanfile at '/tmp/pyrequires/test_package/conanfile.py' doesn't declare any requirement, use `self.tested_reference_str` to require the package being created.

[samuel-emrys]

Additionally, if I modify the test_package/conanfile.py to include tested_reference_str (in this only way I could work out how to - this works with a conan create command):

diff --git a/test_package/conanfile.py b/test_package/conanfile.py
index c4cc2cc..4f6092b 100644
--- a/test_package/conanfile.py
+++ b/test_package/conanfile.py
@@ -7,6 +7,7 @@ class PyvenvTestConan(ConanFile):
     settings = "os", "build_type", "arch", "compiler"
     apply_env = False
     test_type = "explicit"
+    python_requires = ConanFile.tested_reference_str
 
     def config_options(self):
         del self.settings.build_type

This gives the following error:

$ conan test test_package funcs/0.1.0@user

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}


======== Launching test_package ========

======== Computing dependency graph ========
ERROR: Traceback (most recent call last):
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/cli.py", line 272, in main
    cli.run(args)
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/cli.py", line 172, in run
    command.run(self._conan_api, args[0][1:])
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/command.py", line 125, in run
    info = self._method(conan_api, parser, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/commands/test.py", line 40, in test
    deps_graph = run_test(conan_api, path, ref, profile_host, profile_build, remotes, lockfile,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/commands/test.py", line 67, in run_test
    print_graph_basic(deps_graph)
  File "/home/user/.local/pipx/venvs/conan/lib/python3.11/site-packages/conan/cli/printers/graph.py", line 20, in print_graph_basic
    for r in node.conanfile.python_requires._pyrequires.values():  # TODO: improve interface
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute '_pyrequires'

ERROR: 'NoneType' object has no attribute '_pyrequires'

[samuel-emrys]

using

diff --git a/test_package/conanfile.py b/test_package/conanfile.py
index c4cc2cc..640bf0b 100644
--- a/test_package/conanfile.py
+++ b/test_package/conanfile.py
@@ -7,6 +7,7 @@ class PyvenvTestConan(ConanFile):
     settings = "os", "build_type", "arch", "compiler"
     apply_env = False
     test_type = "explicit"
+    python_requires = self.tested_reference_str
 
     def config_options(self):
         del self.settings.build_type

Does not work with conan create, and gives:

$ conan test test_package funcs/0.1.0@user

======== Input profiles ========
Profile host:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}

Profile build:
[settings]
arch=x86_64
build_type=Release
compiler=clang
compiler.cppstd=20
compiler.libcxx=libstdc++11
compiler.version=16
os=Linux
[conf]
tools.build:compiler_executables={'c': '/usr/bin/clang', 'cpp': '/usr/bin/clang++'}

ERROR: Error loading conanfile at '/tmp/pyrequires/test_package/conanfile.py': Unable to load conanfile in /tmp/pyrequires/test_package/conanfile.py
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/tmp/pyrequires/test_package/conanfile.py", line 6, in <module>
    class PyvenvTestConan(ConanFile):
  File "/tmp/pyrequires/test_package/conanfile.py", line 10, in PyvenvTestConan
    python_requires = self.tested_reference_str
                      ^^^^
NameError: name 'self' is not defined

[memsharded]

Hi @samuel-emrys

Thanks for the report.

The python_requires is injected, cannot be defined in test_package/conanfile.py recipe, because they work at the class level, so self. wont work here. This is why it only works with conan create but not with the independent conan test command. I am having a look to see if it is possible to avoid this issue.

[samuel-emrys]

Thanks @memsharded. Would it be feasible to just avoid raising when there are no requires or tool_requires, but still a python_requires statement?

ERROR: The conanfile at '/tmp/pyrequires/test_package/conanfile.py' doesn't declare any requirement, use `self.tested_reference_str` to require the package being created.

[memsharded]

Thanks @memsharded. Would it be feasible to just avoid raising when there are no requires or tool_requires, but still a python_requires statement?

But hardcoding the python_requires is also inconvenient, it has to change for every version, etc. I am exploring other possibilities now. The thing is that there is missing context for conan test command, I am trying to add it explicitly.

[memsharded]

Proposing #15485, seems explicit better than implicit, but can't guarantee, we need to make sure we don't break

[samuel-emrys]

Yes, agree. As someone who doesn't look at the mechanics of how this works super frequently, being explicit is better to immediately understand it. After not looking at my python requires recipes for a few months, it took me some time to realise that I didn't need to specify the python_requires explicitly for it to be available to the test package - it was very confusing initially.

[memsharded]

Fixed in #15485 for next 2.1