clj-zeppelin

A Clojure library for managing Zeppelin notebooks using the Zeppelin Notebook REST API.

Documentation

• API

Installation:

• To install, add the following dependency to your project or build file:

      [clj-zeppelin "0.1.0-SNAPSHOT"]

API Description

The operations defined in this library are as follows :

• create-note!
• list-notes
• delete-note!
• create-paragraph!
• run-paragraph-async
• run-paragraph-sync
• get-paragraph-status
• get-paragraph-info
• run-all-paragraphs

---

create-note!

This POST method creates a new note and returns the id of the newly created note.

• Function call :
  

  (create-note! notebook-server-url payload)

• Input fields :
  

  notebook-server-url - Zeppelin server url 
  
  payload - The payload is a map with (compulsory) key :name (which is the name of the notebook), and optional key :paragraphs, which is a list of paragraphs. Each paragraph is a map with keys :title and :text.

• Example :
  

  (create-note! "http://[zeppelin-server]:[zeppelin-port]" {:name "Ocean Notebook"})

• Output :
  

  2DTW93XQ9 is the id of the created note.

---

list-notes

This GET method returns a JSON array containing the name and id of all the available notes in the zeppelin server.

• Function call :
  

  (list-notes notebook-server-url)

• Input fields :
  

  notebook-server-url - Zeppelin server url 

• Example :
  

  (list-notes "http://[zeppelin-server]:[zeppelin-port]")
  
  

• Output :

  {:status "OK", :message "", :body  [{:name "Zeppelin Tutorial/Basic Features (Spark)", :id "2A94M5J1Z"}]}

---

delete-note!

This DELETE method deletes a note on the zeppelin server by the given note id and returns a JSON array containing status.

• Function call :
  

  (delete-note! notebook-server-url note-id)

• Input fields :
  

  notebook-server-url - Zeppelin server url 
  
  note-id - id of the notebook to be deleted

• Example :
  

  (delete-note! "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9")

• Output :
  

  {:status "OK", :message ""}

---

create-paragraph!

This POST method adds a new paragraph to an existing note and returns id of the newly created paragraph.

• Function call :
  

  (create-paragraph! notebook-server-url note-id paragraph_data)

• Input fields :
  

  notebook-server-url - Zeppelin server url
  
  note-id - id of existing notebook in zeppelin 
  
  paragraph_data - content to be added to the paragraph

• Example :
  

  (create-paragraph! "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9" (-> {:title "intro" :text (10 + 10)}))

• Output :
  

  20181029-070332_2137251371 is the id of the created paragraph.

---

run-paragraph-async

This POST method runs the paragraph asynchronously by given note and paragraph id and returns an OK message.

• Function call :
  

  (run-paragraph-async notebook-server-url note-id paragraph-id)

• Input fields :
  

  notebook-server-url - Zeppelin server url
  
  note-id - id of the existing notebook in zeppelin
  
  paragraph-id -  id of the paragraph in the notebook

• Example :
  

  (run-paragraph-async "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9" "20181029-070332_2137251371")

• Output :
  

  OK

---

run-paragraph-sync

This POST method runs the paragraph synchronously by given note and paragraph id and returns a JSON array containing SUCCESS or ERROR status, depending on the outcome of paragraph execution.

• Function call :
  

  (run-paragraph-sync notebook-server-url note-id paragraph-id)

• Input fields :
  

  notebook-server-url - Zeppelin server url
  
  note-id - id of the existing notebook in zeppelin
  
  paragraph-id -  id of the paragraph in the notebook

• Example :
  

  (run-paragraph-sync "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9" "20181029-070332_2137251371")

• Output :
  

  {:code "SUCCESS", :msg  [{:type "TEXT", :data "\nres1: Int = 20\n"}]}

---

get-paragraph-status

This GET method gets the status of a single paragraph by the given note and paragraph id. The returned JSON array contains the paragraph id, paragraph status, paragraph finish date, paragraph start date.

• Function call :
  

  (get-paragraph-status notebook-server-url note-id paragraph-id)

• Input fields :
  

  notebook-server-url - Zeppelin server url
  
  note-id - id of the existing notebook in zeppelin
  
  paragraph-id -  id of the paragraph in the notebook

• Example :
  

  (get-paragraph-status "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9" "20181029-070332_2137251371")

• Output :
  

  {:progress "100", :started  "Mon Oct 29 07:20:06 UTC 2018", :finished  "Mon Oct 29 07:20:07 UTC 2018",
  :id "20181029-070332_2137251371", :status "FINISHED"}

---

get-paragraph-info

This GET method retrieves an existing paragraph's information using the given id. The returned JSON array contains information about the paragraph.

• Function call :
  

  (get-paragraph-info notebook-server-url note-id paragraph-id)

• Input fields :
  

  notebook-server-url - Zeppelin server url
  
  note-id - id of the existing notebook in zeppelin
  
  paragraph-id -  id of the paragraph in the notebook

• Example :
  

  (get-paragraph-info "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9" "20181029-070332_2137251371")

• Output :
  

  {:apps [], :dateFinished  "Oct 29, 2018 7:20:07 AM", :jobName  "paragraph_1540796612560_-766558253", :config {},
  :dateUpdated  "Oct 29, 2018 7:03:32 AM", :settings {:params {}, :forms {}}, :title "intro", :status "FINISHED",
  :id "20181029-070332_2137251371", :progressUpdateIntervalMs 500, :dateCreated  "Oct 29, 2018 7:03:32 AM",
  :dateStarted  "Oct 29, 2018 7:20:06 AM", :user "anonymous", :text "20", :results  {:code "SUCCESS",  :msg
  [{:type "TEXT",    :data "\nres1: Int = 20\n"}]}}

---

run-all-paragraphs

This POST method runs all paragraphs in the specified note and returns a status message.

• Function call :
  

  (run-all-paragraphs notebook-server-url note-id)

• Input fields :
  

  notebook-server-url - Zeppelin server url
  
  note-id - id of the existing notebook in zeppelin

• Example :
  

  (run-all-paragraphs "http://[zeppelin-server]:[zeppelin-port]" "2DTW93XQ9")

• Output :
  

  {:status "OK"}

---

Development

Testing:

• Install Docker
• Verify that Zeppelin Docker instance is available locally by running docker run -p 8080:8080 apache/zeppelin:0.8.0
• Install Leiningen
• Run lein test should create the Zeppelin docker container and run the tests

License

Copyright © 2018 Dex

Distributed under the Apache 2.0 License