Architecture

Jumbune architecture can be classified into the following major blocks:

Request Handler This layer supports UI based execution. Jumbune execution is triggered by the user in a sequential workflow, expressed in a simple YAML configuration. The YAML configuration consists of instructions for individual Jumbune components. Jumbune presents intuitive self-explanatory reports for the submitted workflow.

Execution This layer is responsible for the bifurcation of the flow of the Web or Shell based execution of the Hadoop job. As per the request received the Core Executor either calls Shell Executor if it receives request for running the job through shell(via Console) or the HttpExecutor if it receives request for running the job through web, then this executors are responsible for calling the Base Processor for handling the flow of individual components.

Processors This layer consists of the necessary processors for handling different components of Jumbune according to the specified component in the YAML. Base Processor handles the request received from the executors and directs the flow of control to the underlying component specific processors.

Components This layer comprises of the following components:

• Flow Debugging: This component verifies the flow of input records in user’s map reduce implementation and provides In-depth analysis of Map Reduce flow thereby helping to identifying the bottle necks in the map reduce phase.
• Cluster Monitoring: This component provides on demand Hadoop JMX and Node resource statistics. This component provides network latency across Hadoop nodes, per file per node wise replica placement, rack aware nodes view as well as data load partition across the nodes.
• JVM Profiling: This component provides Map Reduce Phase wise stats which includes per job performance of JVM, data flow rate and resource usage. This component also provides per job heap sites and CPU cycles for Mapper and Reducer.
• HDFS Data Validation: This component validates inconsistencies in the HDFS data in the form of null, data type & regex checks.

Remoting Remoting is the underlying transport layer of the Jumbune architecture which interacts with Hadoop via Non-blocking NIO Socket Transport through Jumbune Agent. This layer handles all the instructions provided by the above components to process the desired outcome.

Jumbune Agent This ultra-light weight component is responsible for handling all the interactions between the master and the worker nodes through jsch execution/shell mode hence giving the decoupled installation from Hadoop. This layer receives the instructions from the Remoting layer to perform various operations on Hadoop.

High Level Architecture Diagram of Jumbune

![Architecture Diagram](http://i.imgur.com/eyEbx0a.png"Architecture Diagram")