-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Attributes from qstat command now found under native hash #198
Conversation
The XML schema for qstat can be found at the link below: https://github.com/gridengine/gridengine/blob/master/source/dist/util/resources/schemas/qstat/qstat.xsd |
Need to change unit testing to read values from |
The conversion of these hashes into Info objects is done ood_core/lib/ood_core/job/adapters/sge/batch.rb Lines 68 to 72 in 63187f8
The idea is the primary purpose of the info and info_all methods on the adapters is to return an array of objects that have the same interface, regardless of the adapter. So id, job_owner, job_name, accounting_id, status, procs, queue_name, submission_time, dispatch_time, wallclock_limit etc. are all defined https://github.com/OSC/ood_core/blob/63187f8874e62b752a73e39263f539ce0241495a/lib/ood_core/job/info.rb. The original idea for native was when building the Slurm and Torque adapters - the Info interface was small, and the qstat/squeue calls returned a lot more information than was in our interface, and Jeremy wanted to capture the original full data somewhere in that object for access. For example, the actual state in Torque might be Exiting or Transitioning but these exact states do not exist in every adapter. So the common interface to job status is OodCore::Job::Status which Info returns. But native would provide a way to get at the raw data produced by Torque, which would include the original T or E for the status. The native data would also use the original key names. So for example, Info#queue_name in Slurm native attribute might have the key name "partition". Here is an example of creating an Info object using a subset of the data from torque, and then setting native to the raw data at the end: ood_core/lib/ood_core/job/adapters/pbspro.rb Lines 443 to 459 in 63187f8
For the SGE adapter, when parsing the XML we build a hash with the keys matching the info object, so initializing a new Info object is made easier - but of course the native attribute is empty. There are several approaches.
There is likely a place in the sax parser to add every tag name and value to native - then it is just a change in one location. |
In fact if you can just add a snippet to capture all tags and put the text contents in the native hash, in the tag end method. But that might be problematic for cases where we have hierarchy of tags; or we might still want to cherry pick tags whose values we want to display to the user. The XSD file above does not contain all the possible values different versions of SGE provide - just looking at the Discourse topic you mentioned, there are a bunch of tags that are not represented. If we have the access to the output file path, that can be used for the links to the job directory in the progressive disclosure on the Active Jobs app (Files and Shell links). |
I'm not seeing anything regarding output file path in the tags. Is there another way to get the path? |
NewAttributesForSGE.txt To address the differences between other versions of grid engine one could specify a .yaml or .json file under JB_job_number: Job Number //Or whatever key you want to show up in active jobs |
I think we're good with this now. Though, I'd ask that you rebase to master and use the uge test file to make a testcase for |
Added cpu/mem_usage attributes following gridengine/qstat.xsd
b1554fa
to
595e5d0
Compare
Remove TODO
595e5d0
to
59e0eba
Compare
Added cpu/mem_usage attributes following gridengine/qstat.xsd