From 06238fbc9b3a65085db2ea1ad4aa53630b1660f3 Mon Sep 17 00:00:00 2001 From: Damien CORNEAU Date: Mon, 22 Aug 2016 13:47:46 +0900 Subject: [PATCH 01/22] Modify the Community page --- _includes/sub-views/community/contribute.md | 27 ++++++++++++++++++++ _includes/sub-views/community/mailinglist.md | 13 ++++++++++ _plugins/markdown_tag.rb | 23 +++++++++++++++++ community.md | 20 ++++++--------- 4 files changed, 71 insertions(+), 12 deletions(-) create mode 100644 _includes/sub-views/community/contribute.md create mode 100644 _includes/sub-views/community/mailinglist.md create mode 100644 _plugins/markdown_tag.rb diff --git a/_includes/sub-views/community/contribute.md b/_includes/sub-views/community/contribute.md new file mode 100644 index 00000000000..df746bb69ee --- /dev/null +++ b/_includes/sub-views/community/contribute.md @@ -0,0 +1,27 @@ +### How to contribute + +There is multiple ways you can contribute to the project. +And help is always welcome! + +#### Issue Tracker + +Apache Zeppelin uses JIRA as an [Issue Tracker](https://issues.apache.org/jira/browse/ZEPPELIN). +Don't hesitate to report new bugs and improvements ideas. + +#### Contribute to the source code + +We like to get code contributions through Pull Requests on our [Github Mirror](https://github.com/apache/zeppelin). + +But before starting, please read our [Contribution guidelines](/contribute.html), it will give +you important information about our review process, and pointers on how to make a good code contribution. + +You can visit our [issue Tracker](https://issues.apache.org/jira/browse/ZEPPELIN) to find issues to resolve, +and if your are a newcomer and don't know where to get started, we have set some [beginner issues](https://issues.apache.org/jira/browse/ZEPPELIN-1245?jql=project%20%3D%20ZEPPELIN%20AND%20status%20%3D%20Open%20AND%20labels%20%3D%20beginner). + +#### Other contributions + +Not much of a coder? There is other ways to help out: + +* Documentation and website improvements are always welcome +* Helping each other by answering questions on the Mailing List +* Participating in reviewing contributions. diff --git a/_includes/sub-views/community/mailinglist.md b/_includes/sub-views/community/mailinglist.md new file mode 100644 index 00000000000..9edd893ff65 --- /dev/null +++ b/_includes/sub-views/community/mailinglist.md @@ -0,0 +1,13 @@ +### Mailing list + +Get help using Apache Zeppelin or contribute to the project on our mailing lists: + +* __Users :__ [subscribe](mailto:users-subscribe@zeppelin.apache.org?subject=send this email to subscribe), [unsubscribe](mailto:users-unsubscribe@zeppelin.apache.org?subject=send this email to unsubscribe), [archives](http://mail-archives.apache.org/mod_mbox/zeppelin-users/) +
+for usage questions, help, and announcements. +* __Dev :__ [subscribe](mailto:dev-subscribe@zeppelin.apache.org?subject=send this email to subscribe), [unsubscribe](mailto:dev-unsubscribe@zeppelin.apache.org?subject=send this email to unsubscribe), [archives](http://mail-archives.apache.org/mod_mbox/zeppelin-dev/) +
+for people wanting to contribute to the project. +* __Commits :__ [subscribe](mailto:commits-subscribe@zeppelin.apache.org?subject=send this email to subscribe), [unsubscribe](mailto:commits-unsubscribe@zeppelin.apache.org?subject=send this email to unsubscribe), [archives](http://mail-archives.apache.org/mod_mbox/zeppelin-commits/) +
+for commit messages and patches. diff --git a/_plugins/markdown_tag.rb b/_plugins/markdown_tag.rb new file mode 100644 index 00000000000..727bf402772 --- /dev/null +++ b/_plugins/markdown_tag.rb @@ -0,0 +1,23 @@ +=begin + Jekyll tag to include Markdown text from _includes directory preprocessing with Liquid. + Usage: + {% markdown %} + Dependency: + - kramdown +=end +module Jekyll + class MarkdownTag < Liquid::Tag + def initialize(tag_name, text, tokens) + super + @text = text.strip + end + require "kramdown" + def render(context) + tmpl = File.read File.join Dir.pwd, "_includes", @text + site = context.registers[:site] + tmpl = (Liquid::Template.parse tmpl).render site.site_payload + html = Kramdown::Document.new(tmpl).to_html + end + end +end +Liquid::Template.register_tag('markdown', Jekyll::MarkdownTag) diff --git a/community.md b/community.md index 0d30a082cda..79cc8a6257c 100644 --- a/community.md +++ b/community.md @@ -19,15 +19,11 @@ limitations under the License. --> {% include JB/setup %} - -### Mailing list - -Get help using Apache Zeppelin or contribute to the project on our mailing lists: - -* [users@zeppelin.apache.org](http://mail-archives.apache.org/mod_mbox/zeppelin-users/) is for usage questions, help, and announcements. [subscribe](mailto:users-subscribe@zeppelin.apache.org?subject=send this email to subscribe), [unsubscribe](mailto:users-unsubscribe@zeppelin.apache.org?subject=send this email to unsubscribe), [archives](http://mail-archives.apache.org/mod_mbox/zeppelin-users/) -* [dev@zeppelin.apache.org](http://mail-archives.apache.org/mod_mbox/zeppelin-dev/) is for people who want to contribute code to Zeppelin. [subscribe](mailto:dev-subscribe@zeppelin.apache.org?subject=send this email to subscribe), [unsubscribe](mailto:dev-unsubscribe@zeppelin.apache.org?subject=send this email to unsubscribe), [archives](http://mail-archives.apache.org/mod_mbox/zeppelin-dev/) -* [commits@zeppelin.apache.org](http://mail-archives.apache.org/mod_mbox/zeppelin-commits/) is for commit messages and patches to Zeppelin. [subscribe](mailto:commits-subscribe@zeppelin.apache.org?subject=send this email to subscribe), [unsubscribe](mailto:commits-unsubscribe@zeppelin.apache.org?subject=send this email to unsubscribe), [archives](http://mail-archives.apache.org/mod_mbox/zeppelin-commits/) - -### Issue tracker - - [https://issues.apache.org/jira/browse/ZEPPELIN](https://issues.apache.org/jira/browse/ZEPPELIN) +
+
+ {% markdown sub-views/community/contribute.md %} +
+
+ {% markdown sub-views/community/mailinglist.md %} +
+
From cb8760861b28b78e2ffc58e65bafd1ee154ce30b Mon Sep 17 00:00:00 2001 From: Damien CORNEAU Date: Mon, 22 Aug 2016 13:51:45 +0900 Subject: [PATCH 02/22] Add forgotten license headers --- _includes/sub-views/community/contribute.md | 14 ++++++++++++++ _includes/sub-views/community/mailinglist.md | 14 ++++++++++++++ 2 files changed, 28 insertions(+) diff --git a/_includes/sub-views/community/contribute.md b/_includes/sub-views/community/contribute.md index df746bb69ee..d595742666a 100644 --- a/_includes/sub-views/community/contribute.md +++ b/_includes/sub-views/community/contribute.md @@ -1,3 +1,17 @@ + + ### How to contribute There is multiple ways you can contribute to the project. diff --git a/_includes/sub-views/community/mailinglist.md b/_includes/sub-views/community/mailinglist.md index 9edd893ff65..7b3707941b4 100644 --- a/_includes/sub-views/community/mailinglist.md +++ b/_includes/sub-views/community/mailinglist.md @@ -1,3 +1,17 @@ + + ### Mailing list Get help using Apache Zeppelin or contribute to the project on our mailing lists: From acc68b50d71e16752eb598235d5be32ad6fbf5b8 Mon Sep 17 00:00:00 2001 From: Damien CORNEAU Date: Mon, 22 Aug 2016 14:38:19 +0900 Subject: [PATCH 03/22] Add contribution Guidelines to the website --- _includes/sub-views/community/contribute.md | 2 +- _includes/sub-views/community/mailinglist.md | 2 +- contribute.md | 236 +++++++++++++++++++ 3 files changed, 238 insertions(+), 2 deletions(-) create mode 100644 contribute.md diff --git a/_includes/sub-views/community/contribute.md b/_includes/sub-views/community/contribute.md index d595742666a..3e080b99d73 100644 --- a/_includes/sub-views/community/contribute.md +++ b/_includes/sub-views/community/contribute.md @@ -12,7 +12,7 @@ See the License for the specific language governing permissions and limitations under the License. --> -### How to contribute +## How to contribute There is multiple ways you can contribute to the project. And help is always welcome! diff --git a/_includes/sub-views/community/mailinglist.md b/_includes/sub-views/community/mailinglist.md index 7b3707941b4..e604982c8ca 100644 --- a/_includes/sub-views/community/mailinglist.md +++ b/_includes/sub-views/community/mailinglist.md @@ -12,7 +12,7 @@ See the License for the specific language governing permissions and limitations under the License. --> -### Mailing list +## Mailing list Get help using Apache Zeppelin or contribute to the project on our mailing lists: diff --git a/contribute.md b/contribute.md new file mode 100644 index 00000000000..fa7ab9422d7 --- /dev/null +++ b/contribute.md @@ -0,0 +1,236 @@ +--- +layout: page +title: "Contribute" +description: "" +--- + +{% include JB/setup %} + +# Contribution Guidelines + +**Zeppelin** is [Apache2 License](https://github.com/apache/zeppelin/blob/master/CONTRIBUTING.md) Software. +Contributing to Zeppelin (Source code, Documents, Image, Website) means you agree to the Apache2 License. + +1. Make sure your issue is not already in the [Jira issue tracker](https://issues.apache.org/jira/browse/ZEPPELIN) +2. If not, create a ticket describing the change you're proposing in the [Jira issue tracker](https://issues.apache.org/jira/browse/ZEPPELIN) +3. Contribute your patch via Pull Request on our [Github Mirror](https://github.com/apache/zeppelin). + +Before you start, please read the [Code of Conduct](http://www.apache.org/foundation/policies/conduct.html) carefully, familiarize yourself with it and refer to it whenever you need it. + +For those of you who are not familiar with Apache project, understanding [How it works](http://www.apache.org/foundation/how-it-works.html) would be quite helpful. + +## Creating a Pull Request +When creating a Pull Request, you will automatically get the template below. +Filling it thoroughly can improve the speed of the review process. + +``` +### What is this PR for? +A few sentences describing the overall goals of the pull request's commits. +First time? Check out the contribution guidelines - https://zeppelin.apache.org/contribute.html + +### What type of PR is it? +[Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring] + +### Todos +* [ ] - Task + +### What is the Jira issue? +* Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN/ +* Put link here, and add [ZEPPELIN-*Jira number*] in PR title, eg. [ZEPPELIN-533] + +### How should this be tested? +Outline the steps to test the PR here. + +### Screenshots (if appropriate) + +### Questions: +* Does the licenses files need update? +* Is there breaking changes for older versions? +* Does this needs documentation? +``` + +## Testing a Pull Request +You can also test and review a particular Pull Request. Here are two useful ways. + +* Using a utility provided from Zeppelin. + + ``` + dev/test_zeppelin_pr.py [# of PR] + ``` + + For example, if you want to test `#513`, then the command will be: + + ``` + dev/test_zeppelin_pr.py 513 + ``` + +* Another way is using [github/hub](https://github.com/github/hub). + + ``` + hub checkout https://github.com/apache/zeppelin/pull/[# of PR] + ``` + +The above two methods will help you test and review Pull Requests. + +## Source Control Workflow +Zeppelin follows [Fork & Pull] (https://github.com/sevntu-checkstyle/sevntu.checkstyle/wiki/Development-workflow-with-Git:-Fork,-Branching,-Commits,-and-Pull-Request) model. + +## The Review Process + +When a Pull Request is submitted, it is being merged or rejected by following review process. + +* Anybody can be a reviewer and may comment on the change and suggest modifications. +* Reviewer can indicate that a patch looks suitable for merging with a comment such as: "Looks good", "LGTM", "+1". +* At least one indication of suitable for merging (e.g. "LGTM") from committer is required to be merged. +* Pull request is open for 1 or 2 days for potential additional review, unless it's got enough indication of suitable for merging. +* Committer can initiate lazy consensus ("Merge if there is no more discussion") and the code can be merged after certain time (normally 24 hours) when there is no review exists. +* Contributor can ping reviewers (including committer) by commenting 'Ready to review' or suitable indication. + +## Becoming a Committer + +The PPMC adds new committers from the active contributors, based on their contribution to Zeppelin. The qualifications for new committers include: + +1. Sustained contributions: Committers should have a history of constant contributions to Zeppelin. +2. Quality of contributions: Committers more than any other community member should submit simple, well-tested, and well-designed patches. +3. Community involvement: Committers should have a constructive and friendly attitude in all community interactions. They should also be active on the dev, user list and reviewing patches. Also help new contributors and users. + + +## Setting up +Here are some things you will need to build and test Zeppelin. + +### Software Configuration Management (SCM) + +Zeppelin uses Git for its SCM system. `http://git.apache.org/zeppelin.git` you'll need git client installed in your development machine. +For write access, `https://git-wip-us.apache.org/repos/asf/zeppelin.git` + +### Integrated Development Environment (IDE) + +You are free to use whatever IDE you prefer, or your favorite command line editor. + +### Project Structure + +Zeppelin project is based on Maven. Maven works by convention & defines [directory structure] (https://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html) for a project. +The top-level pom.xml describes the basic project structure. Currently Zeppelin has the following modules. + + zeppelin-interpreter + zeppelin-zengine + spark + markdown + angular + shell + flink + ignite + lens + cassandra + zeppelin-web + zeppelin-server + zeppelin-distribution + +### Code convention +We are following Google Code style: + +* [Java style](https://google.github.io/styleguide/javaguide.html) +* [Shell style](https://google.github.io/styleguide/shell.xml) + +Check style report location are in `${submodule}/target/site/checkstyle.html` +Test coverage report location are in `${submodule}/target/site/cobertura/index.html` + +## Getting the source code +First of all, you need the Zeppelin source code. The official location for Zeppelin is [http://git.apache.org/zeppelin.git](http://git.apache.org/zeppelin.git). + +### git access + +Get the source code on your development machine using git. + +``` +git clone git://git.apache.org/zeppelin.git zeppelin +``` + +You may also want to develop against a specific branch. For example, for branch-0.5.6 + +``` +git clone -b branch-0.5.6 git://git.apache.org/zeppelin.git zeppelin +``` + +or with write access + +``` +git clone https://git-wip-us.apache.org/repos/asf/zeppelin.git +``` + +### Fork repository + +If you want not only build Zeppelin but also make change, then you need fork [Zeppelin github mirror repository](https://github.com/apache/zeppelin) and make a pull request. + + +## Build + +### Build Tools + +To build the code, install + + * Oracle Java 7 + * Apache Maven + +### Building the code + +``` +mvn install +``` + +To skip test + +``` +mvn install -DskipTests +``` + +To build with specific spark / hadoop version + +``` +mvn install -Phadoop-2.2 -Dhadoop.version=2.2.0 -Pspark-1.3 -Dspark.version=1.3.0 +``` + +## Tests +Each new File should have its own accompanying unit tests. Each new interpreter should have come with its tests. + + +Zeppelin has 3 types of tests: + + 1. Unit Tests: The unit tests run as part of each package's build. E.g. SparkInterpeter Module's unit test is SparkInterpreterTest + 2. Integration Tests: The integration tests run after all modules are build. The integration tests launch an instance of Zeppelin server. ZeppelinRestApiTest is an example integration test. + 3. GUI integration tests: These tests validate the Zeppelin UI elements. These tests require a running Zeppelin server and launches a web browser to validate Notebook UI elements like Notes and their execution. See ZeppelinIT as an example. + +Currently the GUI integration tests are not run in the Maven and are only run in the CI environment when the pull request is submitted to github. Make sure to watch the [CI results] (https://travis-ci.org/apache/zeppelin/pull_requests) for your pull request. + +## Continuous Integration + +Zeppelin uses Travis for CI. In the project root there is .travis.yml that configures CI and [publishes CI results] (https://travis-ci.org/apache/zeppelin/builds) + + +## Run Zeppelin server in development mode + +``` +cd zeppelin-server +HADOOP_HOME=YOUR_HADOOP_HOME JAVA_HOME=YOUR_JAVA_HOME mvn exec:java -Dexec.mainClass="org.apache.zeppelin.server.ZeppelinServer" -Dexec.args="" +``` + +or use daemon script + +``` +bin/zeppelin-daemon start +``` + + +Server will be run on http://localhost:8080 From 5f7e2bd47c2b027c9d63fb4d0ebc6c9e2b8b2e33 Mon Sep 17 00:00:00 2001 From: Damien CORNEAU Date: Mon, 22 Aug 2016 17:13:14 +0900 Subject: [PATCH 04/22] Add a side menu template --- _includes/sub-views/community/contribute.md | 2 +- _includes/themes/zeppelin/sideMenu.html | 10 +++++ _layouts/sideMenu.html | 5 +++ contribute.md => contribution/general.md | 43 ++++++++++----------- contribution/webapplication.md | 22 +++++++++++ 5 files changed, 58 insertions(+), 24 deletions(-) create mode 100644 _includes/themes/zeppelin/sideMenu.html create mode 100644 _layouts/sideMenu.html rename contribute.md => contribution/general.md (90%) create mode 100644 contribution/webapplication.md diff --git a/_includes/sub-views/community/contribute.md b/_includes/sub-views/community/contribute.md index 3e080b99d73..b91aee1485d 100644 --- a/_includes/sub-views/community/contribute.md +++ b/_includes/sub-views/community/contribute.md @@ -26,7 +26,7 @@ Don't hesitate to report new bugs and improvements ideas. We like to get code contributions through Pull Requests on our [Github Mirror](https://github.com/apache/zeppelin). -But before starting, please read our [Contribution guidelines](/contribute.html), it will give +But before starting, please read our [Contribution guidelines](/contribution/general.html), it will give you important information about our review process, and pointers on how to make a good code contribution. You can visit our [issue Tracker](https://issues.apache.org/jira/browse/ZEPPELIN) to find issues to resolve, diff --git a/_includes/themes/zeppelin/sideMenu.html b/_includes/themes/zeppelin/sideMenu.html new file mode 100644 index 00000000000..b7246ad27bc --- /dev/null +++ b/_includes/themes/zeppelin/sideMenu.html @@ -0,0 +1,10 @@ +
+
+ {% assign pages_list = site.pages %} + {% assign group = page.menu %} + {% include JB/pages_list %} +
+
+ {{ content }} +
+
diff --git a/_layouts/sideMenu.html b/_layouts/sideMenu.html new file mode 100644 index 00000000000..96fc5c2beea --- /dev/null +++ b/_layouts/sideMenu.html @@ -0,0 +1,5 @@ +--- +layout: default +--- +{% include JB/setup %} +{% include themes/zeppelin/sideMenu.html %} diff --git a/contribute.md b/contribution/general.md similarity index 90% rename from contribute.md rename to contribution/general.md index fa7ab9422d7..51d057cee25 100644 --- a/contribute.md +++ b/contribution/general.md @@ -1,7 +1,8 @@ --- -layout: page -title: "Contribute" +layout: sideMenu +title: "General" description: "" +menu: nav-contrib --- -{% include JB/setup %} # Contribution Guidelines @@ -35,31 +35,29 @@ For those of you who are not familiar with Apache project, understanding [How it When creating a Pull Request, you will automatically get the template below. Filling it thoroughly can improve the speed of the review process. -``` -### What is this PR for? -A few sentences describing the overall goals of the pull request's commits. -First time? Check out the contribution guidelines - https://zeppelin.apache.org/contribute.html + ### What is this PR for? + A few sentences describing the overall goals of the pull request's commits. + First time? Check out the contribution guidelines - https://zeppelin.apache.org/contribute.html -### What type of PR is it? -[Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring] + ### What type of PR is it? + [Bug Fix | Improvement | Feature | Documentation | Hot Fix | Refactoring] -### Todos -* [ ] - Task + ### Todos + * [ ] - Task -### What is the Jira issue? -* Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN/ -* Put link here, and add [ZEPPELIN-*Jira number*] in PR title, eg. [ZEPPELIN-533] + ### What is the Jira issue? + * Open an issue on Jira https://issues.apache.org/jira/browse/ZEPPELIN/ + * Put link here, and add [ZEPPELIN-*Jira number*] in PR title, eg. [ZEPPELIN-533] -### How should this be tested? -Outline the steps to test the PR here. + ### How should this be tested? + Outline the steps to test the PR here. -### Screenshots (if appropriate) + ### Screenshots (if appropriate) -### Questions: -* Does the licenses files need update? -* Is there breaking changes for older versions? -* Does this needs documentation? -``` + ### Questions: + * Does the licenses files need update? + * Is there breaking changes for older versions? + * Does this needs documentation? ## Testing a Pull Request You can also test and review a particular Pull Request. Here are two useful ways. @@ -232,5 +230,4 @@ or use daemon script bin/zeppelin-daemon start ``` - Server will be run on http://localhost:8080 diff --git a/contribution/webapplication.md b/contribution/webapplication.md new file mode 100644 index 00000000000..2e4dc67f30a --- /dev/null +++ b/contribution/webapplication.md @@ -0,0 +1,22 @@ +--- +layout: sideMenu +title: "Web Application" +description: "" +group: nav-contrib +menu: nav-contrib-front +--- + + +# About the Web App From c933c194b3190961e799b8c9a75071718290c6dd Mon Sep 17 00:00:00 2001 From: Damien CORNEAU Date: Mon, 22 Aug 2016 17:43:34 +0900 Subject: [PATCH 05/22] Add zeppelin-web contribution page --- contribution/webapplication.md | 130 ++++++++++++++++++++++++++++++++- 1 file changed, 129 insertions(+), 1 deletion(-) diff --git a/contribution/webapplication.md b/contribution/webapplication.md index 2e4dc67f30a..3997d629eba 100644 --- a/contribution/webapplication.md +++ b/contribution/webapplication.md @@ -19,4 +19,132 @@ See the License for the specific language governing permissions and limitations under the License. --> -# About the Web App +# Contributing to Zeppelin-Web + +## Dev Mode +When working on Zeppelin's WebApplication, it is recommended to run in dev mode. + +For that, start Zeppelin server normally, then use ``./grunt serve`` in _zeppelin-web_ directory. + +This will launch a Zeppelin WebApplication on port **9000** that will update on code changes. + +## Technologies + +Zeppelin WebApplication is using **AngularJS** as main Framework, and **Grunt** and **Bower** as helpers. + +So you might want to get familiar with it. +[Here is a good start](http://www.sitepoint.com/kickstart-your-angularjs-development-with-yeoman-grunt-and-bower/) +(There is obviously plenty more ressources to learn) + +## Coding style + +* We follow mainly the [Google Javascript Guide](https://google-styleguide.googlecode.com/svn/trunk/javascriptguide.xml) +* We use a 2 spaces indentation +* We use single quotes + +But don't worry, Eslint and Jscs will make you remember it for the most part. + +We try not to have **JQuery except in directives**, If you want to include a library, +please search for its **angularJS** directive first. + +If you still need to use it, then please use ``angular.element()`` instead of ``$()`` + +## Folder Structure & Code Organization + +* `src` folder: Contains the Source code for Zeppelin WebApplication +* `dist` folder: Contains the compiled code after using **grunt build** + +### Src and Code Organization + +The `src` folder is organized as such: + +
+ src/
+ ├──  app/
+ │   ├──  name/
+ │   │    ├──  name.controller.js
+ |   |    ├──  name.html
+ |   |    ├──  subComponent1/
+ |   |    |    ├──  subComponent1.html
+ |   |    |    ├──  subComponent1.css
+ │   |    |    └──  subComponent1.controller.js
+ │   │    └──  name.css
+ │   └──  app.js
+ ├──  assets/
+ │   ├──  images/
+ │   └──  styles/
+ |        ├──  looknfeel/
+ │        └──  printMode.css
+ ├──  components/
+ │   ├──  component1/
+ |   |    ├──  component1.html
+ │   |    └──  component1.controller.js
+ │   └──  component2/
+ ├──  fonts/
+ |    ├──  *.{eot,svg,ttf,woff,otf}
+ │    └──  *.css
+ ├──  favico.ico
+ ├──  index.html
+ └──  404.html
+
+ +The code is now organized in a component type of architecture, where everything is logically grouped. + +#### File type name convention + +In order to understand what is contained inside the .js files without opening it, we use some name conventions: + +* .controller.js +* .directive.js +* .service.js + +### Component Architecture + +When we talk about Component architecture, we think about grouping files together in a logical way. + +A component can then be made of multiple files like `.html`, `.css` or any other file type mentioned above. + +Related components can be grouped as sub-component as long as they are used in that component only. + + +#### App folder + +Contains the application `app.js` and page related components. + +* Home Page +* Interpreter Page +* Notebook Page +etc... + +The only resctiction being that a component in the `app` folder is **not used anywhere else** + +#### Components folder + +The `components` folder is here to contains any reusable component (used more than once) + +### Fonts + +Fonts files and their css are mixed together in the `fonts` folder + +## New files includes + +As we do not use yeoman to generate controllers or other type of files with this new structure, +we need to do some includes manually in `index.html` in order to use dev mode and compile correctly. + +* Non-bower `.js` files needs to be injected between the tag `` +* Css files needs to be injected between the tag `` + +## Add plugins with Bower +``` +bower install --save +``` +The file index.html will automatically update with the new bower_component + +
+ +**Example**: `./bower install angular-nvd3` + +You should find that line in the index.html file +``` + +```` From 306ae06724a870b2da454283afc851f20b65925f Mon Sep 17 00:00:00 2001 From: Damien CORNEAU Date: Mon, 22 Aug 2016 18:05:32 +0900 Subject: [PATCH 06/22] Add zeppelin-web GPG menu --- contribution/webapplication.md | 9 +++++++++ contribution/zeppelinweb/GPG01.md | 22 ++++++++++++++++++++++ 2 files changed, 31 insertions(+) create mode 100644 contribution/zeppelinweb/GPG01.md diff --git a/contribution/webapplication.md b/contribution/webapplication.md index 3997d629eba..a03f85b151a 100644 --- a/contribution/webapplication.md +++ b/contribution/webapplication.md @@ -36,6 +36,15 @@ So you might want to get familiar with it. [Here is a good start](http://www.sitepoint.com/kickstart-your-angularjs-development-with-yeoman-grunt-and-bower/) (There is obviously plenty more ressources to learn) +## Good Practices + +On the side menu of this page, you will find documentation about some of the best practices we want to apply +in this project. + +This is a great way for people to learn more about angularJS, and to keep the code clean and optimized. + +Please try to follow those __Good Practices Guides__ when making a PR or reviewing one. + ## Coding style * We follow mainly the [Google Javascript Guide](https://google-styleguide.googlecode.com/svn/trunk/javascriptguide.xml) diff --git a/contribution/zeppelinweb/GPG01.md b/contribution/zeppelinweb/GPG01.md new file mode 100644 index 00000000000..c9070ee9908 --- /dev/null +++ b/contribution/zeppelinweb/GPG01.md @@ -0,0 +1,22 @@ +--- +layout: sideMenu +title: "#1 - Using IIFE" +description: "" +group: nav-contrib-front +menu: nav-contrib-front +--- + + +# Using IIFE From 666f3ec95ed4f7d6aed5efefec3b40269d022bf0 Mon Sep 17 00:00:00 2001 From: Damien CORNEAU Date: Tue, 23 Aug 2016 14:31:29 +0900 Subject: [PATCH 07/22] Add first Zeppelin-web best practices page --- contribution/zeppelinweb/GPG01.md | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/contribution/zeppelinweb/GPG01.md b/contribution/zeppelinweb/GPG01.md index c9070ee9908..7291e45a691 100644 --- a/contribution/zeppelinweb/GPG01.md +++ b/contribution/zeppelinweb/GPG01.md @@ -1,6 +1,6 @@ --- layout: sideMenu -title: "#1 - Using IIFE" +title: "Defining Components" description: "" group: nav-contrib-front menu: nav-contrib-front @@ -19,4 +19,28 @@ See the License for the specific language governing permissions and limitations under the License. --> -# Using IIFE +# Defining Angular Components + +
+We should have only one Angular Component per file, and it should look like this: + +``` +(function() { + 'use strict'; + + angular.module('zeppelinWebApp').controller('HomeCtrl', HomeCtrl); + + HomeCtrl.$inject = ['$location']; + + function HomeCtrl($location) { + ... + } + +})(); +``` + +#### Explanations + +* The component function and the component's dependency injection is separated from the component definition +* We apply an Immediately Invoked Function Expression (IIFE) to each component, You can learn more about it +in this [nice post](https://github.com/johnpapa/angular-styleguide/tree/master/a1#iife) of John Papa. From 81493abd9ecbf07cf864f5509e96c1bbabdc72ce Mon Sep 17 00:00:00 2001 From: Damien CORNEAU Date: Tue, 23 Aug 2016 15:45:29 +0900 Subject: [PATCH 08/22] Change Side Menu style --- _includes/themes/zeppelin/sideMenu.html | 2 +- assets/themes/zeppelin/css/style.css | 25 +++++++++++++++++++++++++ 2 files changed, 26 insertions(+), 1 deletion(-) diff --git a/_includes/themes/zeppelin/sideMenu.html b/_includes/themes/zeppelin/sideMenu.html index b7246ad27bc..7534e207c1d 100644 --- a/_includes/themes/zeppelin/sideMenu.html +++ b/_includes/themes/zeppelin/sideMenu.html @@ -1,5 +1,5 @@
-
+
{% assign pages_list = site.pages %} {% assign group = page.menu %} {% include JB/pages_list %} diff --git a/assets/themes/zeppelin/css/style.css b/assets/themes/zeppelin/css/style.css index 482847bc79f..038261c64a9 100644 --- a/assets/themes/zeppelin/css/style.css +++ b/assets/themes/zeppelin/css/style.css @@ -173,6 +173,31 @@ body { outline-width: 0; } +/* SideMenu */ + +.sideMenu li { + list-style: none; + border: 1px solid #c2c2c2; + border-bottom: none; + padding: 5px 10px; +} + +.sideMenu li a { + text-decoration: none; + color: #3071a9; +} + +.sideMenu li:first-of-type { + border-top-left-radius: 3px; + border-top-right-radius: 3px; +} + +.sideMenu li:last-of-type { + border-bottom: 1px solid #c2c2c2; + border-bottom-left-radius: 3px; + border-bottom-right-radius: 3px; +} + /* CUSTOMIZE THE CAROUSEL -------------------------------------------------- */ From 94d653b26eeb44538ef5f07f969de912e324478a Mon Sep 17 00:00:00 2001 From: Damien CORNEAU Date: Wed, 24 Aug 2016 11:49:35 +0900 Subject: [PATCH 09/22] Fix typos --- _includes/sub-views/community/contribute.md | 8 ++++---- contribution/general.md | 5 +++-- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/_includes/sub-views/community/contribute.md b/_includes/sub-views/community/contribute.md index b91aee1485d..e34a5966391 100644 --- a/_includes/sub-views/community/contribute.md +++ b/_includes/sub-views/community/contribute.md @@ -14,10 +14,10 @@ limitations under the License. ## How to contribute -There is multiple ways you can contribute to the project. +There are multiple ways you can contribute to the project. And help is always welcome! -#### Issue Tracker +### Issue Tracker Apache Zeppelin uses JIRA as an [Issue Tracker](https://issues.apache.org/jira/browse/ZEPPELIN). Don't hesitate to report new bugs and improvements ideas. @@ -29,12 +29,12 @@ We like to get code contributions through Pull Requests on our [Github Mirror](h But before starting, please read our [Contribution guidelines](/contribution/general.html), it will give you important information about our review process, and pointers on how to make a good code contribution. -You can visit our [issue Tracker](https://issues.apache.org/jira/browse/ZEPPELIN) to find issues to resolve, +You can visit our [Issue Tracker](https://issues.apache.org/jira/browse/ZEPPELIN) to find issues to resolve, and if your are a newcomer and don't know where to get started, we have set some [beginner issues](https://issues.apache.org/jira/browse/ZEPPELIN-1245?jql=project%20%3D%20ZEPPELIN%20AND%20status%20%3D%20Open%20AND%20labels%20%3D%20beginner). #### Other contributions -Not much of a coder? There is other ways to help out: +Not much of a coder? There are other ways to help out: * Documentation and website improvements are always welcome * Helping each other by answering questions on the Mailing List diff --git a/contribution/general.md b/contribution/general.md index 51d057cee25..0a9106d4914 100644 --- a/contribution/general.md +++ b/contribution/general.md @@ -98,7 +98,7 @@ When a Pull Request is submitted, it is being merged or rejected by following re ## Becoming a Committer -The PPMC adds new committers from the active contributors, based on their contribution to Zeppelin. The qualifications for new committers include: +The PMC adds new committers from the active contributors, based on their contribution to Zeppelin. The qualifications for new committers include: 1. Sustained contributions: Committers should have a history of constant contributions to Zeppelin. 2. Quality of contributions: Committers more than any other community member should submit simple, well-tested, and well-designed patches. @@ -208,10 +208,11 @@ Zeppelin has 3 types of tests: 1. Unit Tests: The unit tests run as part of each package's build. E.g. SparkInterpeter Module's unit test is SparkInterpreterTest 2. Integration Tests: The integration tests run after all modules are build. The integration tests launch an instance of Zeppelin server. ZeppelinRestApiTest is an example integration test. - 3. GUI integration tests: These tests validate the Zeppelin UI elements. These tests require a running Zeppelin server and launches a web browser to validate Notebook UI elements like Notes and their execution. See ZeppelinIT as an example. + 3. GUI integration tests: These tests validate the Zeppelin UI elements. These tests require a running Zeppelin server and launches a web browser to validate Notebook UI elements like Notes and their execution. See ZeppelinIT as an example. Currently the GUI integration tests are not run in the Maven and are only run in the CI environment when the pull request is submitted to github. Make sure to watch the [CI results] (https://travis-ci.org/apache/zeppelin/pull_requests) for your pull request. + ## Continuous Integration Zeppelin uses Travis for CI. In the project root there is .travis.yml that configures CI and [publishes CI results] (https://travis-ci.org/apache/zeppelin/builds) From e7209eac8487d5cb7910e5553b237120aead5d4a Mon Sep 17 00:00:00 2001 From: AhyoungRyu Date: Wed, 24 Aug 2016 11:58:44 +0900 Subject: [PATCH 10/22] Add docs contribution guide --- contribution/documentation.md | 151 ++++++++++++++++++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100644 contribution/documentation.md diff --git a/contribution/documentation.md b/contribution/documentation.md new file mode 100644 index 00000000000..c7e00f77148 --- /dev/null +++ b/contribution/documentation.md @@ -0,0 +1,151 @@ +--- +layout: sideMenu +title: "Documentation" +description: "" +group: nav-contrib +menu: nav-contrib-front +--- + + +# Contributing to Documentation + +## Dev Mode +Apache Zeppelin is using [Jekyll](https://jekyllrb.com/) which is a static site generator and [Github Pages](https://pages.github.com/) as a site publisher. For the more details, see [help.github.com/articles/about-github-pages-and-jekyll/](https://help.github.com/articles/about-github-pages-and-jekyll/). + +### Requirements + +``` +# ruby --version >= 2.0.0 +# Install Bundler using gem +gem install bundler + +cd $ZEPPELIN_HOME/docs +# Install all dependencies declared in the Gemfile +bundle install +``` + +For the further information about requirements, please see [here](https://help.github.com/articles/setting-up-your-github-pages-site-locally-with-jekyll/#requirements). + +On OS X 10.9, you may need to do + +``` +xcode-select --install + +``` +### Run the website locally + +If you don't want to encounter uglily rendered pages, run the documentation site in your local first. +In `$ZEPPELIN_HOME/docs`, + +``` +bundle exec jekyll serve --watch +``` + +Using the above command, Jekyll will start a web server at `http://localhost:4000` and watch the `/docs` directory to update. + + +## Folder Structure & Components +`docs/` folder is organized like below: + +``` +docs/ + ├── _includes/themes/zeppelin + │ ├── _navigation.html + │ └── default.html + ├── _layouts + ├── _plugins + ├── assets/themes/zeppelin -> {ASSET_PATH} + │ ├── bootstrap + │ ├── css + │ ├── img + │ └── js + ├── development/ *.md + ├── displaysystem/ *.md + ├── install/ *.md + ├── interpreter/ *.md + ├── manual/ *.md + ├── quickstart/ *.md + ├── rest-api/ *.md + ├── security/ *.md + ├── storage/ *.md + ├── Gemfile + ├── Gemfile.lock + ├── _config.yml + ├── index.md + └── ... +``` + + - `_navigation.html`: the dropdown menu in navbar + - `default.html` & `_layouts/`: define default HTML layout + - `_plugins/`: custom plugin `*.rb` files can be placed in this folder. See [jekyll/plugins](https://jekyllrb.com/docs/plugins/) for the further information. + - `{ASSET_PATH}/css/style.css`: extra css components can be defined + - `{ASSET_PATH}/img/docs-img/`: image files used for document pages can be placed in this folder + - `{ASSET_PATH}/js/`: extra `.js` files can be placed + - `Gemfile`: defines bundle dependencies. They will be installed by `bundle install`. + - `Gemfile.lock`: when you run `bundle install`, bundler will persist all gems name and their version to this file. For the more details, see [Bundle "The Gemfile Lock"](http://bundler.io/v1.10/man/bundle-install.1.html#THE-GEMFILE-LOCK) + - `documentation_group`: `development/`, `displaysystem/`, `install/`, `interpreter/`... + - `_config.yml`: defines configuration options for docs website. See [jekyll/configuration](https://jekyllrb.com/docs/configuration/) for the other available config variables. + - `index.md`: the main page of `http://zeppelin.apache.org/docs//` + + +### Markdown +Zeppelin documentation pages are written with [Markdown](http://daringfireball.net/projects/markdown/). It is possible to use [GitHub flavored syntax](https://help.github.com/categories/writing-on-github/) and intermix plain HTML. + +### Front matter +Every page contains [YAML front matter](https://jekyllrb.com/docs/frontmatter/) block in their header. Don't forget to wrap the front matter list with triple-dashed lines(`---`) like below. +The document page should start this triple-dashed lines. Or you will face 404 error, since Jekyll can't find the page. + +``` +--- +layout: page +title: "Apache Zeppelin Tutorial" +description: "This tutorial page contains a short walk-through tutorial that uses Apache Spark backend. Please note that this tutorial is valid for Spark 1.3 and higher." +group: quickstart +--- +``` + + - `layout`: the default layout is `page` which is defined in `_layout/page.html`. + - `title`: the title for the document. Please note that if it needs to include `Zeppelin`, it should be `Apache Zeppelin`, not `Zeppelin`. + - `description`: a short description for the document. One or two sentences would be enough. This description also will be shown as an extract sentence when people search pages. + - `group`: a category of the document page + +### Headings +All documents are structured with headings. From these headings, you can automatically generate a **Table of Contents**. There is a simple rule for Zeppelin docs headings. + +``` +# Level-1 heading <- used only for the main title +## Level-2 heading <- start with this +### Level-3 heading +#### Level-4 heading <- won't be converted in TOC from this level +``` + +### Table of contents(TOC) + +``` +
+``` + +Add this line below `# main title` in order to generate a **Table of Contents**. Headings until `### (Level-3 heading)` are included to TOC. + + +Default setting options for TOC are definded in [here](https://github.com/apache/zeppelin/blob/master/docs/assets/themes/zeppelin/js/toc.js#L4). + + +### Adding new pages +If you're going to create new pages, there are some spots you need to add the location of the page. + + - **Dropdown menu in navbar**: add your docs location to [_navigation.html](https://github.com/apache/zeppelin/blob/master/docs/_includes/themes/zeppelin/_navigation.html) + - **Main index**: add your docs below [What is the next?](http://zeppelin.apache.org/docs/latest/#what-is-the-next) section in [index.md](https://github.com/apache/zeppelin/blob/master/docs/index.md) with a short description. No need to do this if the page is for **Interpreters**. + From 8f69f283d031fcdb5f5c90426ad01f88f4879adf Mon Sep 17 00:00:00 2001 From: Damien CORNEAU Date: Wed, 24 Aug 2016 12:07:31 +0900 Subject: [PATCH 11/22] Add GUI integration tests CLI --- contribution/general.md | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/contribution/general.md b/contribution/general.md index 0a9106d4914..6baff2af00f 100644 --- a/contribution/general.md +++ b/contribution/general.md @@ -206,12 +206,29 @@ Each new File should have its own accompanying unit tests. Each new interpreter Zeppelin has 3 types of tests: - 1. Unit Tests: The unit tests run as part of each package's build. E.g. SparkInterpeter Module's unit test is SparkInterpreterTest - 2. Integration Tests: The integration tests run after all modules are build. The integration tests launch an instance of Zeppelin server. ZeppelinRestApiTest is an example integration test. - 3. GUI integration tests: These tests validate the Zeppelin UI elements. These tests require a running Zeppelin server and launches a web browser to validate Notebook UI elements like Notes and their execution. See ZeppelinIT as an example. +* __Unit Tests:__ The unit tests run as part of each package's build. E.g. SparkInterpeter Module's unit test is SparkInterpreterTest +* __Integration Tests:__ The integration tests run after all modules are build. The integration tests launch an instance of Zeppelin server. ZeppelinRestApiTest is an example integration test. +* __GUI integration tests:__ These tests validate the Zeppelin UI elements. These tests require a running Zeppelin server and launches a web browser to validate Notebook UI elements like Notes and their execution. See ZeppelinIT as an example. -Currently the GUI integration tests are not run in the Maven and are only run in the CI environment when the pull request is submitted to github. Make sure to watch the [CI results] (https://travis-ci.org/apache/zeppelin/pull_requests) for your pull request. +Currently the __GUI integration tests__ are not run in the Maven and are only run in the CI environment when the pull request is submitted to github. +Make sure to watch the [CI results] (https://travis-ci.org/apache/zeppelin/pull_requests) for your pull request. + +#### Running GUI integration tests locally + +##### All tests, just like the CI: + +``` +PATH=~/Applications/Firefox.app/Contents/MacOS/:$PATH CI="true" mvn verify -Pspark-1.6 -Phadoop-2.3 -Ppyspark -B -pl "zeppelin-interpreter,zeppelin-zengine,zeppelin-server,zeppelin-display,spark-dependencies,spark" -Dtest="org.apache.zeppelin.AbstractFunctionalSuite" -DfailIfNoTests=false +``` + +##### Next to a Running instance of Zeppelin + +This allows you to target a specific __GUI integration test__. + +``` +TEST_SELENIUM="true" mvn package -DfailIfNoTests=false -pl 'zeppelin-interpreter,zeppelin-zengine,zeppelin-server' -Dtest=ParagraphActionsIT +``` ## Continuous Integration From 318c4c19144b47874290660c13a04fe792b081ea Mon Sep 17 00:00:00 2001 From: AhyoungRyu Date: Wed, 24 Aug 2016 12:11:27 +0900 Subject: [PATCH 12/22] Add section for 'For committers only' --- contribution/documentation.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/contribution/documentation.md b/contribution/documentation.md index c7e00f77148..4bdfc38b2d8 100644 --- a/contribution/documentation.md +++ b/contribution/documentation.md @@ -149,3 +149,23 @@ If you're going to create new pages, there are some spots you need to add the lo - **Dropdown menu in navbar**: add your docs location to [_navigation.html](https://github.com/apache/zeppelin/blob/master/docs/_includes/themes/zeppelin/_navigation.html) - **Main index**: add your docs below [What is the next?](http://zeppelin.apache.org/docs/latest/#what-is-the-next) section in [index.md](https://github.com/apache/zeppelin/blob/master/docs/index.md) with a short description. No need to do this if the page is for **Interpreters**. + +## For committers only +### Bumping up version in a new release + +`ZEPPELIN_VERSION` and `BASE_PATH` property in `_config.yml` + +### Deploy to ASF svnpubsub infra + 1. generate static website in `./_site` + + ``` + # go to /docs under Zeppelin source + bundle exec jekyll build --safe + ``` + + 2. checkout ASF repo + ``` + svn co https://svn.apache.org/repos/asf/zeppelin asf-zeppelin + ``` + 3. copy `zeppelin/docs/_site` to `asf-zeppelin/site/docs/[VERSION]` + 4. ```svn commit``` From 98826702a321a6c67e2af030fb7d120560a883d8 Mon Sep 17 00:00:00 2001 From: Damien CORNEAU Date: Wed, 24 Aug 2016 12:25:13 +0900 Subject: [PATCH 13/22] Allows contribution page without side Menu --- _includes/themes/zeppelin/sideMenu.html | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/_includes/themes/zeppelin/sideMenu.html b/_includes/themes/zeppelin/sideMenu.html index 7534e207c1d..05181990f50 100644 --- a/_includes/themes/zeppelin/sideMenu.html +++ b/_includes/themes/zeppelin/sideMenu.html @@ -1,10 +1,16 @@
-
- {% assign pages_list = site.pages %} - {% assign group = page.menu %} - {% include JB/pages_list %} -
-
- {{ content }} -
+ {% if page.menu %} +
+ {% assign pages_list = site.pages %} + {% assign group = page.menu %} + {% include JB/pages_list %} +
+
+ {{ content }} +
+ {% else %} +
+ {{ content }} +
+ {% endif %}
From e0b4c8d30ffe4df95e3cb1cbc1fe3205191f0266 Mon Sep 17 00:00:00 2001 From: Damien CORNEAU Date: Wed, 24 Aug 2016 12:27:26 +0900 Subject: [PATCH 14/22] change name of good practice file --- contribution/zeppelinweb/{GPG01.md => goodPracticeGuide01.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename contribution/zeppelinweb/{GPG01.md => goodPracticeGuide01.md} (100%) diff --git a/contribution/zeppelinweb/GPG01.md b/contribution/zeppelinweb/goodPracticeGuide01.md similarity index 100% rename from contribution/zeppelinweb/GPG01.md rename to contribution/zeppelinweb/goodPracticeGuide01.md From a4c924bcc9d654e33e924310b480773ee486db29 Mon Sep 17 00:00:00 2001 From: AhyoungRyu Date: Wed, 24 Aug 2016 12:31:29 +0900 Subject: [PATCH 15/22] Remove 'menu' property --- contribution/documentation.md | 1 - 1 file changed, 1 deletion(-) diff --git a/contribution/documentation.md b/contribution/documentation.md index 4bdfc38b2d8..f049370929e 100644 --- a/contribution/documentation.md +++ b/contribution/documentation.md @@ -3,7 +3,6 @@ layout: sideMenu title: "Documentation" description: "" group: nav-contrib -menu: nav-contrib-front --- {% endcomment %} @@ -25,15 +25,17 @@ Usage: {% else %} {% for node in pages_list %} {% if node.title != null %} - {% if group == null or group == node.group %} - {% if page.url == node.url %} -
  • {{node.title}}
    • - {% else %} -
  • {{node.title}}
    • - {% endif %} - {% endif %} + {% for cat in node.group %} + {% if cat == group %} + {% if page.url == node.url %} +
  • {{node.title}}
    • + {% else %} +
  • {{node.title}}
    • + {% endif %} + {% endif %} + {% endfor %} {% endif %} {% endfor %} {% endif %} {% assign pages_list = nil %} -{% assign group = nil %} \ No newline at end of file +{% assign group = nil %} diff --git a/contribution/webapplication.md b/contribution/webapplication.md index a03f85b151a..8010e1039ab 100644 --- a/contribution/webapplication.md +++ b/contribution/webapplication.md @@ -2,7 +2,9 @@ layout: sideMenu title: "Web Application" description: "" -group: nav-contrib +group: +- nav-contrib +- nav-contrib-front menu: nav-contrib-front ---