A site grid gatekeeper technology based solely on HTCondor components.
This package is simply a thin set of wrappers around HTCondor, allowing you to run a HTCondor-CE without disrupting a site HTCondor install.
For example, condor_ce_q is the HTCondor-CE equivalent to condor_q for the
HTCondor-CE processes. This package took much of its inspiration - and base
code - from OSGs condor-cron package.
Sites are encouraged to install the sub-package htcondor-ce-condor or
htcondor-ce-pbs, depending on which batch manager they run.
HTCondor-CE RPMs are available from the following locations:
- HTCondor Yum repositories: https://research.cs.wisc.edu/htcondor/yum/
- OSG Yum repositories: https://opensciencegrid.org/docs/common/yum/
At any given time, two versions of HTCondor-CE are maintained, a stable and a development version.
In this repository, the master branch contains the latest version of HTCondor-CE (i.e. development) while the stable
branch contains the previous version.
- Development: HTCondor-CE 4
- Stable: HTCondor-CE 3
Please share any issues or questions regarding the HTCondor-CE via the following mailing lists:
- htcondor-users@cs.wisc.edu: For issues and questions open to the community.
- htcondor-admin@cs.wisc.edu: For issues and questions containing private information.
- htcondor-security@cs.wisc.edu: For issues regarding security problems/vunerabilities. For more information regarding reporting security problems go to HTCSS Security.
Below are diagrams to show the flow of a capacity allocation request in the form of a job submitted to an HTCondor-CE to being executed in a different batch system. Diagram A showcases a setup where the HTCondor-CE is located on the same host as the destination batch system. While Diagram B showcases a setup with the HTCondor-CE submitting a job to a remote batch system over SSH.
Note: In both setups the HTCondor-CE Schedd sends Master and Schedd Ads to the Central Collector
flowchart LR
%% ID FORMAT:
%% - External nodes are numbered (0 - 100)
%% - Internal nodes are alphabetic (A - Z)
%% - Nodes within nested subgraphs are
%% are labelled by double letters (AA - ZZ)
%% - All subgraph names are capitalized (Blahp)
%% while nodes with the same name are lowercase (blahp)
subgraph HM[CE & Batch System Host Machine]
subgraph HTCondor-CE
%%direction LR %% Flowchart direction statement overrides statements in connected subgraphs; comment these out.
A[[SchedD]] -- Original Job --> B(Job Router)
B -- Routed Job --> A
A -- Routed Job --> C{Grid Manager}
%% Note: Used 1 to place Disk higher in ordering
A -- </br>-Original Job Ad</br>-Routed Job Ad --> 1[(Disk)]
subgraph Blahp
%%direction LR
%% Configure nested subgraphs above internal nodes
subgraph slurm_*.sh
direction LR %% Set direction of isolated subgraphs
AA[submit] ---|OR| BB[cancel]
BB ---|OR| CC[status]
end
E[[blahp]] --> slurm_*.sh
F[common_sub</br>_attr.sh] -->|attrs| slurm_*.sh
slurm_*.sh -->|args| F
end
end
subgraph Batch System
%%direction LR
G((qsub))
end
end
%% -- External Nodes --
0>Job Ad]
%% -- External Nodes --
%% -- Subgraph Links --
C <--> E
slurm_*.sh ---> G
0 --> A
%% Schedd connects to Global Central Collector
A == Ads ==> Z(((Central Collector)))
%% Stylize Outer host machine box
style HM fill:#FFF,stroke:#000
%% -- Subgraph Links --
In this setup where the HTCondor-CE submits to a remote batch system, incoming jobs are required to specify grid_resource = <batch> <system> <hostname>.
Note: The Blaph on the remote host works exactly the same as in Diagram A except that the Grid Manager communicates over SSH.
flowchart LR
%% ID FORMAT:
%% - External nodes are numbered (0 - 100)
%% - Internal nodes are alphabetic (A - Z)
%% - Nodes within nested subgraphs are
%% are labelled by double letters (AA - ZZ)
%% - All subgraph names are capitalized (Blahp)
%% while nodes with the same name are lowercase (blahp)
subgraph HM1[CE Host Machine]
subgraph Bosco Cluster
%%direction LR %% Flowchart direciton statement overrides statements in connected subgraphs; comment these out
A[[SchedD]] -- Original Job --> Z(Job Router)
Z -- Routed Job --> A
A -- Routed Job --> B{Grid</br>Manager}
B --- D[ssh]
B --- E[ssh]
E --> |File Transfer|B
%% Note: Used 1 to place Disk higher in ordering
A -- </br>-Original Job Ad</br>-Routed Job Ad --> 1[(Disk)]
end
end
subgraph HM2[Remote Batch Sytem Host Machine]
subgraph Remote Submit
%%direction LR
F[sshd] --> G[blahp]
%% FTGahp capitalized to be legible
H[sshd] --> I[FTGahp]
I -- SSH Tunnel --> H
end
end
%% -- External Nodes --
0>Job Ad]
%% -- External Nodes --
%% -- Subgraph Links --
D ===|Blahp:</br>-stdin</br>-stdout</br>-stderr| F
E ===|File:</br>-stdin</br>-stdout</br>-stderr| H
0 --> A
A == Ads ==> ZZ(((Central Collector)))
%% Stylize host machine boxes
style HM1 fill:#FFF,stroke:#000
style HM2 fill:#FFF,stroke:#000
%% -- Subgraph Links --
