Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSH Runner: Fix several issues and refactor code #17357

Open
wants to merge 8 commits into
base: develop2
Choose a base branch
from

Conversation

perseoGI
Copy link
Contributor

@perseoGI perseoGI commented Nov 21, 2024

Changelog: Feature: Finish SSH Runner by removing several bugs
Docs: TODO

Changes:

  • Fix creation of virtual environment
  • Retrieve current conan client version and install remotely if it does not exist
  • Error checks
  • Create a custom SSH Logger based on ConanOutput
  • Properly manage build and host profiles, copying to the runner machine if needed and keeping the jinja2 templates intact to allow remote runner to render them locally
  • General code refactor: some improvements like typing and simplifying code flow
  • Fix cache inconsistency when a package have been created remotely and already restored the cache locally. If the package is removed locally, the runner was never bringing the package back to local. This happens because it only compress the build recipes. Solved by always exporting all the packages and forcing a local cache refresh
  • Correctly manage special characters like & and * during ssh remote invoques
  • Refer to the issue that supports this Pull Request.
  • If the issue has missing info, explain the purpose/use case/pain/need that covers this Pull Request.
  • I've read the Contributing guide.
  • I've followed the PEP8 style guides for Python code.
  • I've opened another PR in the Conan docs repo to the develop branch, documenting this one.

@perseoGI perseoGI marked this pull request as ready for review November 22, 2024 13:15
@perseoGI perseoGI modified the milestones: 2.10.0, 2.11 Nov 29, 2024
@@ -214,7 +214,7 @@ def restore(self, path):
unzipped_metadata_folder = pref_bundle.get("metadata_folder")
if unzipped_metadata_folder:
# FIXME: Restore metadata is not incremental, but destructive
out.info(f"Restore: {pref} metadata in {unzipped_metadata_folder}")
out.info(f"Restore: {pref} metadata from {unzipped_metadata_folder}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not restoring from, it is restoring in, the folder is the destination folder

@@ -199,7 +199,7 @@ def restore(self, path):
pkg_layout = cache.create_pkg_layout(pref) # DB Folder entry
# FIXME: This is not taking into account the existence of previous package
unzipped_pkg_folder = pref_bundle["package_folder"]
out.info(f"Restore: {pref} in {unzipped_pkg_folder}")
out.info(f"Restore: {pref} from {unzipped_pkg_folder}")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not restoring from, it is restoring in, the folder is the destination folder

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, unrelated to the PR?

def __init__(self, conan_api, command, host_profile, build_profile, args, raw_args):
from paramiko.config import SSHConfig
from paramiko.client import SSHClient
def __init__(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Avoid formatting and typing changes, it makes difficult to read the changelog and check if some new argument was added to the constructor

@memsharded memsharded modified the milestones: 2.11.0, 2.12.0 Dec 18, 2024
conan/api/subapi/cache.py Outdated Show resolved Hide resolved
conan/api/subapi/cache.py Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants