-
Notifications
You must be signed in to change notification settings - Fork 21
Running hive or pig jobs
mlimotte edited this page May 9, 2012
·
1 revision
Running Hive and Pig scripts can be done, but since we haven't had the need to do this, it does not have a declarative interface. The suggestions below are not difficult, but they do assume a fair understanding of Clojure (and Java interoperability).
Steps created with defstep are Maps which are eventually used to construct StepConfig objects. The entity defined by defstep is passed to (fire!) to make this happen. But (fire!) can also take StepConfig instances.
So, in brief, to make a StepConfig for Hive or Pig; you can use the StepFactory to construct a HadoopJarStepConfig object. You can then call the StepConfig constructor with a name (String) and the HadoopJarStepConfig instance.