Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make the "ssh wrapper" pattern a first class citizen #170

Closed
ericfranz opened this issue Dec 20, 2019 · 3 comments · Fixed by #204
Closed

Make the "ssh wrapper" pattern a first class citizen #170

ericfranz opened this issue Dec 20, 2019 · 3 comments · Fixed by #204
Assignees
Labels
all adapters enhancement interns Issues that are good for the interns to work on
Milestone

Comments

@ericfranz
Copy link
Contributor

ericfranz commented Dec 20, 2019

Like with the linux host adapter, in the job config if you specify submit_host: "some_other_host" then when executing the command to submit the batch job, each adapter would execute that command, its arguments, etc. on another host via ssh. This is the main scope of this feature.

A second feature would be to add another config use_submit_host_for_all_commands: true (a better name?), every adapter that doesn't use the C API should execute all of the scheduler commands on the specified submit host (qstat, qdel, qhold, etc.). This is problematic, however, for adapters that use the C API via FFI for anything (SGE, Torque).

Currently, the only way to use this "ssh wrapper" pattern is for a site to provide their own ssh wrapper scripts and use bin_overrides. This is onerous enough that it is not used by default, and we don't even document well how to do it.

@ericfranz
Copy link
Contributor Author

This would not work for sge or the torque adapters since they both make use of the C libraries. But this would work for Slurm and LSF.

@ericfranz ericfranz added this to the OOD1.7 milestone Dec 20, 2019
@MorganRodgers
Copy link
Contributor

This should be doable in PBSPro as well, it would also work for SGE if the site can accept less nuanced job states by not using DRMAA.

@ericfranz
Copy link
Contributor Author

We could support two options: ssh for just job submission, and ssh for all commands

@ericfranz ericfranz modified the milestones: OOD1.7, OOD1.8 Mar 18, 2020
@ericfranz ericfranz self-assigned this Apr 29, 2020
@ericfranz ericfranz added interns Issues that are good for the interns to work on and removed needs design labels Jun 8, 2020
@ericfranz ericfranz removed their assignment Jun 8, 2020
@matthu017 matthu017 linked a pull request Jun 19, 2020 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
all adapters enhancement interns Issues that are good for the interns to work on
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants