Documentation for grunt/gulp tasks

[Reinmar]

Followup for #43 and #36.

I'm not sure yet where we want to keep it (in the README or in ckeditor5-design and later copy somewhere... dunno). But we definitely need to have a list of them with short descriptions what they are needed for.

Second part is a documentation of a typical workflow (based on #43).

[Reinmar]

One thing that would definitely need to be clarified in ckeditor5-design is what we encountered while working on #43. Like a more high-level description of the situation. Without it the behaviour of our tools may seem to be weird.

PS. We need to link to npm/npm#7742 (comment)

[szymonkups]

For now, I will put tasks description here. Later on we can move them to more suitable place.

Development workflow

Initializing CKEditor 5 repository

To start developing CKEditor 5 packages, first step is to clone CKEditor 5 GitHub repository. It can be done by calling:

git clone git@github.com:cksource/ckeditor5.git

Next, install all project dependencies:

npm install

At this point all packages are installed inside node_modules directory and can be used for building CKEditor 5. Recommended approach is to store the main CKEditor 5 repository under separate directory. It will be later referred as workspace directory:

+---workspace
    +---ckeditor5
        +---node_modules
            +---ckeditor5-core
            +---ckeditor5-toolbar

Package development mode

Putting package in development mode means that it is present in package.json as dependency, its repository is cloned into workspace directory and linked (using symbolic link) to node_modules inside CKEditor 5:

+---workspace
    +---ckeditor5
    |   +---node_modules
    |       +---ckeditor5-core     ---symlink-->  ../../ckeditor5-core
    |       +---ckeditor5-toolbar  ---symlink-->  ../../ckeditor5-toolbar
    |
    +---ckeditor5-core
    +---ckeditor5-toolbar          

In this example both ckeditor5-core and ckeditor5-toolbar are in development mode. New commits can be made inside their corresponding repositories and at the same time these packages can be used in CKEditor 5 build process.

Note

There is a problem with this approach: each package in development mode has its own node_modules directory with own copy of all dependencies (after running npm install there). Some packages will use same dependencies, so ideally these dependencies should be stored once. Example below shows a setting with one shared package:

+---workspace
    +---ckeditor5
    |   +---node_modules
    |       +---package-a  ---symlink-->  ../../package-a
    |       +---package-b  ---symlink-->  ../../package-b
    |       +---package-c  
    |
    +---package-a
    |   +---node_modules
    |       +---package-c
    |
    +---package-b
        +---node_modules
            +--package-c       

In this situation package-c is shared between package-a and package-b and is also used in CKEditor 5 repository. Our first approach was to use npm dedupe command, but it fails on symlinked directories. Other than that, packages in development mode need their own dev-dependencies for testing and using JSHint/JSCS. We decided (after trying different approaches #36, #43) that packages in development mode will be treated as autonomous repositories and we will not depend on npm dedupe. Dealing with duplicated dependencies is a task for CKEditor 5 builder. (TODO: how builder works).

Developing packages

Selecting one package for development

Initial state is described in initializing CKEditor 5 repository. To use development version of a chosen package use dev-install task provided in CKEditor 5 repository. For example, let's use ckeditor5-core package in development mode:

gulp install --package ckeditor/ckeditor5-core

After the task is executed, ckeditor5-core package will be cloned inside workspace directory and linked to CKEditor 5 node_modules:

+---workspace
    +---ckeditor5
    |   +---node_modules
    |       +---ckeditor5-core  ---symlink-->  ../../ckeditor5-core
    |       +--- (...)
    |
    +---ckeditor5-core          

Package is now ready for further development and at the same time will be used for building CKEditor 5.

Using all CKEditor 5 packages in development mode

If there is a need to put all CKEditor 5 packages in the development mode, it can be done by using dev-init task. This task takes all packages included in package.json that starts with ckeditor5- prefix and puts them in development mode.

gulp init

Creating new package

Task that simplifies process of creating new CKEditor 5 packages can be run by:

gulp create-package

Task will ask for package name, initial version and GitHub URL. It will initialize package repository and put it in development mode.

Tasks operating on all packages in development mode

These task operate on all packages that are currently in development mode. This means they will be executed on packages that are present as dependencies inside package.json file in main CKEditor 5 repository.

Tasks Git status

To get short Git status from all repositories:

gulp status

Sample output for two packages in development mode:

ckeditor5-core
## master...origin/master
ckeditor5-plugin-devtest
## master...origin/master

Updating packages

To update Git repositories of packages in development mode:

gulp update

This task will checkout package to desired branch and call git pull to download changes from remote repository. Additionally this task can call npm update in each package repository and main CKEditor repository. This is optional since updating may take a lot of time in some cases:

gulp update --npm-update

Tasks description

install --package < path|GitHub URL|name >

Task description

Installs selected dependency in development mode. Dependency will be cloned into workspace directory (if it is not already), linked ( using symlink ) to node_modules and added to package.json inside CKEditor5 main repository.

Examples

• when providing local path to repository on disk:

gulp install --package /home/user/workspace/ckeditor5-core

• when providing GitHub URL - it can be either short or full URL, also with branch information:

gulp install --package ckeditor/ckeditor5-core
gulp install --package https://github.com/ckeditor/ckeditor5-core.git
gulp install --package git@github.com:ckeditor/ckeditor5-core.git
gulp install --package ckeditor/ckeditor5-core#branch-name

• when providing module name used in NPM. In this situation task will try to reach NPM to obtain GitHub URL from package information and clone it to workspace directory:

gulp install --package ckeditor5-core

Task steps

1. Uses either local path, GitHub URL or NPM name to recognize dependency.
2. If dependency directory does not exists - clones dependency repository into workspace.
3. If branch information is provided - checkout repository to desired branch.
4. Runs npm install inside newly created repository to install all dependencies there.
5. Links repository to node_modules inside CKEditor5 repository.
6. Adds dependency to package.json file inside CKEditor5 repository.

init

Task description

Installs all CKEditor 5 dependencies in development mode. All dependencies from ckeditor5/package.json that starts with ckeditor5- prefix will be installed using dev-install task.

Examples

gulp init

Task steps

1. Gets all dependencies from ckedtior5/package.json that starts with ckeditor5- prefix.
2. Executes install task with each dependency found.

update [--npm-update]

Task description

Updates all dependencies that are currently in development mode. Checks them out to branch provided in ckeditor5/package.json and pulls changes from the remote repository. Optionally it can run npm update inside each dependency and main CKEditor 5 repository.

Examples

• run without npm update

gulp update

• run with npm update

gulp update --npm-update

Task steps

1. Gets all dependencies from ckedtior5/package.json that starts with ckeditor5- prefix.
2. For each dependency that is currently in development mode:
   1. Checks out repository to branch provided in ckeditor5/package.json (to master if no branch is provided)
   2. Pulls changes from remote.
   3. Runs npm update if --npm-update option was provided.
3. Runs npm update inside CKEditor 5 repository if --npm-update option was provided.

status

Task description

Prints repository information using git status --porcelain -sb command for all dependencies in development mode.

Examples

gulp status

Task steps

1. Gets all dependencies from ckedtior5/package.json that starts with ckeditor5- prefix.
2. Executes git status --porcelain -sb for each dependency that is currently in development mode.

create-package

Task description

Creates new package in development mode with provided information.

Example

gulp create-package

Task steps

1. Gathers information that will be included in the new package's package.json
   1. Asks for package name.
   2. Asks for initial package version.
   3. Asks for GitHub URL.
2. Initializes Git repository in workspace directory by executing git init command.
3. Copies template files from ckeditor5/dev/tasks/templates, and other files needed for development:
   1. Default editor configuration: .editorconfig
   2. JSHint configuration files: .jshintrc, tests/.jshintrc, dev/.jshintrc
   3. JSCS configuration: .jscsrc
   4. Git attributes file: .gitattributes
   5. JSHint and JSCS gulp tasks: dev/tasks/lint
4. Updates package's package.json file.
5. Adds package to ckeditor5/package.json.
6. Commits changes in package's repository.
7. Links package directory to ckeditor5/node_modules.
8. Runs npm install inside package's directory.

relink

Task description

Recreates symbolic links for packages in development mode.

Examples

gulp relink

Task steps

1. Gets all dependencies from ckedtior5/package.json that starts with ckeditor5- prefix.
2. Creates symbolic link in ckeditor5/node_modules for each dependency that is currently in development mode.

[Reinmar]

Important note: you use the term "Plugin development mode" and later use the word "plugin" many times. The thing you're referring to is a "package" (aka "node module").

The important of a correct naming is well visible e.g. when you work on ckeditor5-core which is not a plugin – it's simply a package with some modules inside.

Relevant discussion is here: ckeditor/ckeditor5-design#92

[Reinmar]

PS. #66

[szymonkups]

Updated my comment to use gulp tasks, also replaced "plugin" with "package".

[Reinmar]

Published on https://github.com/ckeditor/ckeditor5/wiki/Development-Workflow. Great job.