-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Submit commands on another host via ssh #204
Conversation
Made Adapters::Helper.ssh_wrap to return this command wrapped in ssh
I mentioned one example inline, but for each adapter we want to keep both host and submit host as they are different values.
For example, I could specify the submit_host as being ruby.osc.edu and the host as being owens.osc.edu. Then I would be ssh-ing to ruby.osc.edu and executing qsub with the batch server specified as owens.osc.edu to submit a batch job running on owens. |
Added commands to ssh
Made rspec tests for ssh wrapper
Open3.capture3 uses args in array format
-defaults to yes use to_s to guard against nil objects
With the realization in #201 we may have to rethink how the Instead of just passing the command, we may have to template a small script that can inline the correct environment variables (like the linux host adapter does). I don't believe we want to get involved with Let's look at how this would work for PBSPro. ood_core/lib/ood_core/job/adapters/pbspro.rb Lines 262 to 263 in 9eaf3cf
But these two environment variables need to be passed to the destination. Anything that is set in the environment to the ood_core/lib/ood_core/job/adapters/pbspro.rb Lines 164 to 165 in 9eaf3cf
Maybe instead of templating a script (because that would require all the adapters to toggle the Something to this affect: The So submitting to owens with PBSPro adapter ends up resulting in a command like this: (Note that the quotes are important becuase the shell will see the |
Also, I thought adapters like Torque had some restriction because they use shared libraries (FFI) that may not be on the destination hosts? |
Torque does not use FFI for qsub |
Is that the behavior we want, where some things are executing on the submit_host and others on the webhost? |
#170 Addresses that. This PR is the first half, enabling only qsub by specifying |
lib/ood_core/job/adapters/helper.rb
Outdated
command = "" | ||
env.each {|key, value| command += "export #{key}=#{value}; "} # exports env vars | ||
command += cmd_args.unshift(cmd).join(' ') | ||
args.push(command) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This does the same thing:
args.push(env.map {|k,v| "export #{key}=#{value};" } + [cmd] + cmd_args).join(' '))
though env.map {|k,v| "export #{key}=#{value};" }
should be a separate method
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should a new method still be made for this with the current commit? There are quite a few params in this method already, but everything in this command deals with ssh wrapping.
lib/ood_core/job/adapters/helper.rb
Outdated
check_host = strict_host_checking ? "yes" : "no" | ||
args = ['-o', 'BatchMode=yes', '-o', 'UserKnownHostsFile=/dev/null', '-o', "StrictHostKeyChecking=#{check_host}", "#{submit_host}"] | ||
env.each{|key, value| args.push("export #{key}=#{value};")} | ||
args.concat(cmd_args.unshift(cmd)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
cmd_args.unshift(cmd)
modifies the argument cmd_args
which should not happen. I know you just changed a bunch of + to concats, but in this case + may improve readability.
return 'ssh', args + [cmd] + cmd_args
Conflicts: lib/ood_core/job/adapters/slurm.rb
Made Adapters::Helper.ssh_wrap to return commands wrapped in ssh