Skip to content

Commit

Permalink
separate cdh doc with spark_cluster_mode.md
Browse files Browse the repository at this point in the history
  • Loading branch information
astroshim committed Sep 26, 2016
1 parent e66993f commit b525f68
Show file tree
Hide file tree
Showing 5 changed files with 97 additions and 64 deletions.
2 changes: 1 addition & 1 deletion docs/_includes/themes/zeppelin/_navigation.html
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@
<li><a href="{{BASE_PATH}}/install/spark_cluster_mode.html#spark-standalone-mode">Zeppelin on Spark Cluster Mode (Standalone)</a></li>
<li><a href="{{BASE_PATH}}/install/spark_cluster_mode.html#spark-on-yarn-mode">Zeppelin on Spark Cluster Mode (YARN)</a></li>
<li><a href="{{BASE_PATH}}/install/spark_cluster_mode.html#spark-on-mesos-mode">Zeppelin on Spark Cluster Mode (Mesos)</a></li>
<li><a href="{{BASE_PATH}}/install/spark_cluster_mode.html#zeppelin-on-cdh">Zeppelin on CDH</a></li>
<li><a href="{{BASE_PATH}}/install/cdh.html#apache-zeppelin-on-cdh">Zeppelin on CDH</a></li>
<li role="separator" class="divider"></li>
<li class="title"><span><b>Contibute</b><span></li>
<li><a href="{{BASE_PATH}}/development/writingzeppelininterpreter.html">Writing Zeppelin Interpreter</a></li>
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ Join to our [Mailing list](https://zeppelin.apache.org/community.html) and repor
* [Zeppelin on Spark Cluster Mode (Standalone via Docker)](./install/spark_cluster_mode.html#spark-standalone-mode)
* [Zeppelin on Spark Cluster Mode (YARN via Docker)](./install/spark_cluster_mode.html#spark-on-yarn-mode)
* [Zeppelin on Spark Cluster Mode (Mesos via Docker)](./install/spark_cluster_mode.html#spark-on-mesos-mode)
* [Zeppelin on CDH (via Docker)](./install/spark_cluster_mode.html#zeppelin-on-cdh)
* [Zeppelin on CDH (via Docker)](./install/cdh.html#apache-zeppelin-on-cdh)
* Contribute
* [Writing Zeppelin Interpreter](./development/writingzeppelininterpreter.html)
* [Writing Zeppelin Application (Experimental)](./development/writingzeppelinapplication.html)
Expand Down
95 changes: 95 additions & 0 deletions docs/install/cdh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
---
layout: page
title: "Apache Zeppelin on CDH"
description: "This document will guide you how you can build and configure the environment on CDH with Apache Zeppelin using docker scripts."
group: install
---
<!--
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
{% include JB/setup %}

# Apache Zeppelin on CDH

Cloudera officially provide docker container so we can easily build CDH docker environment following the [link](http://www.cloudera.com/documentation/enterprise/latest/topics/quickstart_docker_container.html).

### 1. Importing the Cloudera QuickStart Docker Image

```
docker pull cloudera/quickstart:latest
```


### 2. Run docker

```
docker run -it \
-p 80:80 \
-p 4040:4040 \
-p 8020:8020 \
-p 8022:8022 \
-p 8030:8030 \
-p 8032:8032 \
-p 8033:8033 \
-p 8040:8040 \
-p 8042:8042 \
-p 8088:8088 \
-p 8480:8480 \
-p 8485:8485 \
-p 8888:8888 \
-p 9083:9083 \
-p 10020:10020 \
-p 10033:10033 \
-p 18088:18088 \
-p 19888:19888 \
-p 25000:25000 \
-p 25010:25010 \
-p 25020:25020 \
-p 50010:50010 \
-p 50020:50020 \
-p 50070:50070 \
-p 50075:50075 \
-h quickstart.cloudera --privileged=true \
agitated_payne_backup /usr/bin/docker-quickstart;
```

### 3. Verify running CDH.

You can see each application web UI for HDFS on `http://<hostname>:50070/`, YARN on `http://<hostname>:8088/cluster`.


### 4. Configure Spark interpreter in Zeppelin
Set following configurations to `conf/zeppelin-env.sh`.

```
export MASTER=yarn-client
export HADOOP_CONF_DIR=[your_hadoop_conf_path]
export SPARK_HOME=[your_spark_home_path]
```

`HADOOP_CONF_DIR`(Hadoop configuration path) is defined in `/scripts/docker/spark-cluster-managers/cdh/hdfs_conf`.

Don't forget to set Spark `master` as `yarn-client` in Zeppelin **Interpreters** setting page like below.

<img src="../assets/themes/zeppelin/img/docs-img/zeppelin_yarn_conf.png" />

### 5. Run Zeppelin with Spark interpreter
After running a single paragraph with Spark interpreter in Zeppelin,
<img src="../assets/themes/zeppelin/img/docs-img/zeppelin_with_cdh.png" />




You can browse `http://<hostname>:8088/cluster/apps` for Zeppelin application is running well or not.
<img src="../assets/themes/zeppelin/img/docs-img/cdh_yarn_applications.png" />

62 changes: 0 additions & 62 deletions docs/install/spark_cluster_mode.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,65 +202,3 @@ After running a single paragraph with Spark interpreter in Zeppelin, browse `htt

<img src="../assets/themes/zeppelin/img/docs-img/mesos_frameworks.png" />


## Zeppelin on CDH
Cloudera officially provide docker container so we can easily build CDH docker environment following the [link](https://www.cloudera.com/documentation/enterprise/5-6-x/topics/quickstart_docker_container.html).


### 1. Run docker

```
docker run -it \
-p 80:80 \
-p 4040:4040 \
-p 8020:8020 \
-p 8022:8022 \
-p 8030:8030 \
-p 8032:8032 \
-p 8033:8033 \
-p 8040:8040 \
-p 8042:8042 \
-p 8088:8088 \
-p 8480:8480 \
-p 8485:8485 \
-p 8888:8888 \
-p 9083:9083 \
-p 10020:10020 \
-p 10033:10033 \
-p 18088:18088 \
-p 19888:19888 \
-p 25000:25000 \
-p 25010:25010 \
-p 25020:25020 \
-p 50010:50010 \
-p 50020:50020 \
-p 50070:50070 \
-p 50075:50075 \
-h quickstart.cloudera --privileged=true \
agitated_payne_backup /usr/bin/docker-quickstart;
```

### 2. Verify running CDH.

You can see each application web UI for HDFS on `http://<hostname>:50070/`, YARN on `http://<hostname>:8088/cluster`.


### 3. Configure Spark interpreter in Zeppelin
Set following configurations to `conf/zeppelin-env.sh`.

```
export MASTER=yarn-client
export HADOOP_CONF_DIR=[your_hadoop_conf_path]
export SPARK_HOME=[your_spark_home_path]
```

`HADOOP_CONF_DIR`(Hadoop configuration path) is defined in `/scripts/docker/spark-cluster-managers/cdh/hdfs_conf`.

Don't forget to set Spark `master` as `yarn-client` in Zeppelin **Interpreters** setting page like below.

<img src="../assets/themes/zeppelin/img/docs-img/zeppelin_yarn_conf.png" />

### 4. Run Zeppelin with Spark interpreter
After running a single paragraph with Spark interpreter in Zeppelin, browse `http://<hostname>:8088/cluster/apps` and check Zeppelin application is running well or not.

<img src="../assets/themes/zeppelin/img/docs-img/cdh_yarn_applications.png" />

0 comments on commit b525f68

Please sign in to comment.