[question] deploy() in conanfile.py

[Vasile-Creuco]

What is your question?

Hi!

The question is how can I get dependencies from cache in my current project with deploy() method from conanfile.py?

When I try to run conan install command with --deployer-package=* argument, I get this error:
line 272, in deploy
File "", line 108, in join
TypeError: expected str, bytes or os.PathLike object, not NoneType

ERROR: expected str, bytes or os.PathLike object, not NoneType

I tried to use the syntax from documentation:

def deploy(self):
        copy(self, "*", src=self.package_folder, dst=self.deploy_folder)

self.package_folder - what represents? Is it the root for conan dependencies?
What I understood, self.package_folder will take binary artifacts from artifactory server, not from my dependencies.
But I would like to get dependencies from my cache.

Thank you!

[memsharded]

Hi @Vasile-Creuco

Thanks for your question

What I understood, self.package_folder will take binary artifacts from artifactory server, not from my dependencies.

No, the self.package_folder is always a local folder to the cache, to a package that has already been downloaded and installed in the cache.

The syntax is intended to be used as conan install --requires=mypkg/1.0 --deployer-package="*", is this the use case? Maybe you can share a conanfile and the exact command to reproduce this? (at the very least, it is a UX bug, that kind of trace shouldn't happen, so we might want to improve the error message)

[Vasile-Creuco]

This is the deploy() method from my conanfile:

def deploy(self):
    _build_type = self.build_type
    if 'package_destinations' in self.conan_data:
        pkgDestinations = self.conan_data['package_destinations']
        if _build_type in pkgDestinations:
            deployData = pkgDestinations[_build_type]
        else:
            deployData = pkgDestinations['default']
            
    # Iterate through the imports
    for dImport in deployData:
      # Import / Copy files from conan cache to workspace
      folderA = os.path.join(self.package_folder, dImport['source'])
      print(f"src folder: {folderA}")
      folderB = os.path.join('.', dImport['target'])
      print(f"dst folder: {folderB}\n")
      # def copy(conanfile, pattern, src, dst, keep_path=True, excludes=None, ignore_case=True):
      copy(self, pattern=dImport['pattern'], src=folderA, dst=folderB, keep_path=True, excludes=("conan*.txt", "conan*.bat", "conan*.sh"))

I'm also using conadata.yml to get requirements:

requires: []
# package dependency requirements for tools
tool_requires: []
# package dependency requirements only for package testing
test_requires: []
# package dependency requirements only to build
build_requires: [name/1.x.x@user/channel]

package_destinations:
  default:
  - pattern: '*.*'
    source: .
    target: ../../rootfolder
  development:
  - pattern: '*.*'
    source: .
    target: ../../rootfolder

[memsharded]

Thanks for the feedback.
I am afraid that I still don't have enough info to know what is happening. A full conanfile and the exact conan ... command would be very appreciated.

Some minor comments:

• if 'package_destinations' in self.conan_data: is unnecessary. If package_destinations is not in the data, the code will crash anyway, because deployData will not be defined

• Having different destinations, specially having a development one with such a relative target = ../.. path reads a bit confusing. This reads more like an external deployer than a in-recipe deploy() method, which is not intended for further development, but for final deploy (well, that is the case of deployers in general). Surely I don't know enough about the use case and the setup, but that variability, coming also from a self.build_type that is so associated to Release\Debug build-type, reads complicated to understand for me.

• Check about build_requires: [name/1.x.x@user/channel] in https://docs.conan.io/2/reference/conanfile/attributes.html#build-requires

  build_requires are used in Conan 2 to provide compatibility with the Conan 1.X syntax, but their use is discouraged in Conan 2 and will be deprecated in future 2.X releases. Please use tool_requires instead of build_requires in your Conan 2 recipes.

  Not sure how you are using it in your recipe, but probably you shouldn't use it.

[Vasile-Creuco]

Many thanks, that you gave me some good improvements which I can add it in my code.

This is the conan command:

conan install --name=nameProject --version=1.x.x --user=username --channel=release --build=nameProject -pr:h=profileDevelopment -pr:b=profileDevelopment -r remoteRepo --deployer-package=*  'path/where/is/my/conanfile'

I created my own profile which I added in my %USERNAME%/.conan2/profiles:

profileDevelopment

[settings]
build_type=Development
compiler=compiler
compiler.version=versionCompiler
os=Windows
arch=x86_64

Now, I thing you understand where it comes from _build_type.

Regarding the target part, why I'm using ../../models, it is because my folder structure it looks like that and if I run conan install command in the rootfolder where is conanfile (work/conanrecipes), it shoud go two steps back to add dependencies in 'models' folder, because I need them to be there:

     
    ├── ...
    ├── work         
    │   ├── conanrecipes 
    │           ├── conanfile.py
    │           ├── conandata.yml
    │   └── models           
    └── ...
    

I hope I made myself clear now and maybe you have an overall an idea what I'm doing.

[memsharded]

build_type=Development

Yeah, overriding build_type with other types rather than Release/Debug/etc, you need to make sure the build system integrations are doing what you want to do, as by default they will only manage the built-ins Release, Debug etc. So CMakeToolchain, CMakeDeps and all other build-systems integration might rely on the built-ins values for many of the automated configuration.

Thanks for the details, I have managed to reproduce.
Indeed the issue is when a conanfile installed by <path> contains a deploy() method.
The feature is intended to deploy packages, not the current one, so only for conan install --requires=xxx not for conan isntall <path>. This means most likely the current <path> conanfile must not be deployed, because there is no package yet to deploy.

[Vasile-Creuco]

Regarding the build_type, I'm sure it works fine. I also added in settings.yml file from my %username/.conan2 in build_type section 'Development', and what I want and need.

So, you mean that deploy() method will work for a specific require when I just run command like conan install --requires=xxx, not for command which I showed up?

Thank you!

[memsharded]

I am submitting a fix in #15737 for next release.

I am going to leave the possibility of calling the local consumer recipe deploy() method, and fix the logic of --deployer-package allowing exclusions, same as other places in Conan.

So, you mean that deploy() method will work for a specific require when I just run command like conan install --requires=xxx, not for command which I showed up?

It will work for both, there will most likely be users that want their local deploy() method to be executed too.

[memsharded]

#15737 merged, it will be in next 2.2