Skip to content

Running hive or pig jobs

mlimotte edited this page May 9, 2012 · 1 revision

Running Hive and Pig scripts can be done, but since we haven't had the need to do this, it does not have a declarative interface. The suggestions below are not difficult, but they do assume a fair understanding of Clojure (and Java interoperability).

Steps created with defstep are Maps which are eventually used to construct StepConfig objects. The entity defined by defstep is passed to (fire!) to make this happen. But (fire!) can also take StepConfig instances.

So, in brief, to make a StepConfig for Hive or Pig; you can use the StepFactory to construct a HadoopJarStepConfig object. You can then call the StepConfig constructor with a name (String) and the HadoopJarStepConfig instance.

Clone this wiki locally