crystal-nodejs

Node.js engine for crystal-lang. JS code and npm module executes on crystal-nodejs, And don't need to install Node.js binary.Explain about architecture, Compiled as a shared object Node.js(i.e: libnodejs) and execute as one process on crystal-lang. So call C execvp() system call through Process.run() method, execute in this c function.

Process.run method is low overhead, Compare pure Node.js js code and crystal-nodejs js code, There was no difference in performance(see Benchmark heading about performance detail).

The biggest merit is that over 11,000,000 npm modules can be used in crystal and Will expand the Crystal Lang ecosystem

Using crystal-nodejs can pass through Node.js result of crystal-lang. Happy Crystaling!! 🎉

Architecture image

In the above architecture image, crystal-nodejs provides Node.js, LibC, lib that was created crystal-lang

Supported OS

• Linux(Ubuntu18.04, 20.04, Arch Linux...Maybe will work other distributions)
• MacOSX(High Sierra... Maybe will work other os version)
• Windows(Unfortunately yet not supported on crystal-lang...)

Supported Node.js

• Look at this directory(e.g 12.20.1, 14.15.4) for supported versions.crystal-nodejs will be supported until latest 2 version in Node.js

• Current Node.js latest version: 14.15.4

Installation

1. Add the dependency to your shard.yml:

  dependencies:
     nodejs:
       github: fukaoi/crystal-nodejs

2. Run shards install

3. libnodejs install, init js directory make or make all.So created $HOME/.crystal-nodejs/

$HOME/.crystal-nodejs/
├── bin        // Use binary in crystal-nodejs
│   ├── node  
│   ├── npm 
│   └── npx
├── js         // Origina JS code, package.json. Enable the subdirectories below js/*/a.js
└── lib        // shared object for Node.js
    └── libnode.so.64(libnode.64.dylib)

Using Docker

alpine

FROM crystallang/crystal:0.35.1-alpine as builder

RUN apk add --update --no-cache \
    curl \
    make \
    python \
    g++ \ 
    gcc \
    gcc-doc \
    linux-headers \
    libc6-compat

RUN ln -s /lib/libc.musl-x86_64.so.1 /lib/ld-linux-x86-64.so.2
RUN git clone https://github.com/fukaoi/crystal-nodejs.git

WORKDIR /crystal-nodejs

RUN shards install

Ubuntu20

FROM ubuntu:20.04 as builder

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update
RUN apt-get upgrade -y 
RUN apt-get install curl gnupg make g++ libyaml-dev -y
RUN curl -sL "https://keybase.io/crystal/pgp_keys.asc" | apt-key add -
RUN echo "deb https://dist.crystal-lang.org/apt crystal main" | tee /etc/apt/sources.list.d/crystal.list
RUN apt-get update
RUN apt-get install crystal -y

RUN git clone https://github.com/fukaoi/crystal-nodejs.git

WORKDIR /crystal-nodejs

RUN shards install

Usage

Bacis usage

Output Hello crystal-nodejs !! on terminal

• Simple JS code, output 'Hello crystal-nodejs'

require "nodejs"

Nodejs.eval("console.log('Hello crystal-nodejs !!')")

Use setTimeout execute lazy code

• below code is output 'Hello crystal' after 2sec

require "nodejs"

code = <<-CODE
  let user = {
    sayHi(firstName) {
      console.log(`Hello, ${firstName}!`);
    }
  };
  setTimeout(() => {user.sayHi('crystal')}, 2000);
CODE

Nodejs.eval(code)

How to send Node.js result to crystal-lang

• Use special toCrystal() method.toCrystal() is only function in crystal-nodejs.Can response various type in JS, and all JS type is converted JSON::Any in crystal

require "nodejs"

code = <<-CODE
  toCrystal({data:"spec"})
CODE

res = Nodejs.eval(code)

puts res           # {"data" => "spec"}
puts typeof(res)   # JSON::Any

• The below example, all JS type(Number, Boolean, String) is converted JSON::Any.

require "nodejs"

code = <<-CODE
  toCrystal({123456})
CODE

res = Nodejs.eval(code)

puts res           # 123456
puts typeof(res)   # JSON::Any

Use NPM(node modules)

Node modules is installed in $HOME/.crystal-nodejs/js/

npm install method

• Use Nodejs::Npm.install('package name') method, can install need package.this below example is for use mathjs module.

require "nodejs"
Nodejs::Npm.install("mathjs")

code = <<-CODE
  const math = require("mathjs"); // Note: JS side require
  toCrystal(math.log(10000, 10));
CODE

puts Nodejs.eval(code) # 4

package.json

• a package.json copy to $HOME/.crystal-nodejs/js/ direcotry, and call NodeJs::Npm.install method

[package.json]

"dependencies": {
  "mathjs": "^6.0.2"
}

require "nodejs"
Nodejs::Npm.install

code = <<-CODE
  const math = require("mathjs"); // Note: JS side require
  toCrystal(math.log(10000, 10));
CODE

puts Nodejs.eval(code) # 4

Use existing the JS file

loading JS file and execute

• existing js file to, use Node Js.file run method, can run on crystal-lang

[file_run.js]

const fs = require('fs');
fs.readFile('spec/nodejs_spec.cr', 'utf8', (err, text) => {
  console.log('text file!');
  toCrystal({text: text});
});

require "nodejs"
puts Nodejs.file_run("./file_run.js")

Replace parameters in existing JS file

• Use Nodejs.replace_params() method, can set the customization value to the parameter of the existing js file from crystal-lang. this method merit don't have to modify existing js files and can use on crystal-lang

• replace parameter type can be only const xxxxx.

[calc.js]

const a = 10;
const b = 20;

const calc = (left_side, right_side) => {
  return left_side + right_side;
}

const res = calc(a, b);
console.log(res);

require "nodejs"

code = File.read("./calc.js")
hash = {
  "a" => 100,  # const a = 100
  "b" => 200   # const b = 200
}

replaced_code = Nodejs.replace_params(code, hash)
Nodejs.eval(replaced_code)  # 300

Catch JS Exception

throw errors directly

• throw Error in JS, it is converted as JSSideExceptin in crystal-lang. So can catch JSSIDEExcetpion in crystal-lang.

require "nodejs"

code = <<-CODE
  throw new Error('Error raise!');
CODE

Nodejs.eval(code)  

Error: Error raise!
    at [eval]:10:9
    at Script.runInThisContext (vm.js:122:20)
    at Object.runInThisContext (vm.js:329:38)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (internal/modules/cjs/loader.js:776:30)
    at evalScript (internal/bootstrap/node.js:589:27)
    at startup (internal/bootstrap/node.js:265:9)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)
 (Nodejs::JSSideException)

Error object throw after catch the JS Error with try catch

• can pass an error object to crystal-lang by using a specific toCrystalErr method.

require "nodejs"

code = <<-CODE
  try {
    throw new Error('Error raise!');
  } catch(e) {
    toCrystalErr(e);
  }
CODE

Nodejs.eval(code)  

Error: Error raise!
    at [eval]:10:8
    at Script.runInThisContext (vm.js:122:20)
    at Object.runInThisContext (vm.js:329:38)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (internal/modules/cjs/loader.js:776:30)
    at evalScript (internal/bootstrap/node.js:589:27)
    at startup (internal/bootstrap/node.js:265:9)
    at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)
 (Nodejs::JSSideException)

More usages look at spec/nodejs/npm/

Benchmark

This benchmark is fibonacci and binary-search results, As can see from the result, crystal-nodejs is high performance.below at bar chart is response average time that executes 5 times fibonacci and binary-search. benchmark source code detail is can see from URL link

• Machine spec:
  • cpu: Intel(R) Core(TM) i7-7820HK CPU @ 2.90GHz 4core
  • memory: 16Gbyte

• Benchmark code:
  • fibonacci source: Node.js crystal-lang
  • binary-search source: Node.js crystal-lang

Raw data(Benchmark result)

• fibonacci

-	Node.js	crystal-nodejs
1	real 2m26.968s	real 2m27.337s
2	real 2m27.230s	real 2m27.263s
3	real 2m26.987s	real 2m28.058s
4	real 2m27.582s	real 2m26.672s
5	real 2m27.262s	real 2m26.831s

• binary-search

-	Node.js	crystal-nodejs
1	real 1m35.699s	real 1m35.962s
2	real 1m35.082s	real 1m32.018s
3	real 1m33.309s	real 1m35.212s
4	real 1m34.423s	real 1m33.830s
5	real 1m32.050s	real 1m33.530s

Development

Extension directory tree

crystal-nodejs/
ext/
├── libnode.cc     // main function for libnode.so
├── node_path.cr   // script for rewrite node path of npm
├── obj            // shared object of Linux, Mac OSX
│   └── 10.16.0
└── 10.16.0        // shared object dependency file and directory
    ├── bin
    ├── include
    └── lib

Create json class

May want to convert crystal-lang json mapper type because response parameter from Nodejs.eval() method is all JSON::Any type. So the recommended tool is jsontocr, very simple and easy.

Exception

• CrystalSideException ... Be thrown error in crystal-lang code
• JSSideException ... Be thrown error in JS code

Make tasks

• make build ... build libnode and deploy $HOME/.crystal-nodejs/
• make install ... using package.json, npm install of dependency modules
• make nodejs ... git clone Node.js and build
• make (all) ... Same task make build && make install
• make clean ... delete $HOME/.crystal-nodejs/ and temporaty directory
• make audit ... do static analysis for JS

Raw JS

If will do mixing crystal-lang code and JS code, Debug is hard. But crystal-nodejs can export Raw JS, Can easily see the JS code that is actually executed.

this function is enable below

RAW_JS=true crystal run(or spec)

RAW_JS=true best timing is when execute spec, because must call Nodejs.eval() method.Export of most Raw JS if coverage of spec code is high. To see exported the code, /tmp/raw_js/

/tmp/raw_js/
├── 0cc6107a79be7b828321e57a81a5f828.js
├── 124d1032b2bc7c02cd59430dd8d1ba65.js
├── 170fa95978969d76eca70d2947da4e2f.js
├── 31df5239bef6b4510b8b6e3840627a58.js
├── eaaa61190908871f1571bc5eca89b87a.js
├── ff3c96e301d7d34c34c4a86fef92e09b.js
└── ff4d0b01de3d93d612bb5a09ed381711.js

Safety

Scan for vulnerabilities when build project. If be found vulnerabilities, build is supposed to fail.this logic execute npm audit command in Makefile.execute npm audit --fix command and build continue

use make audit, export raw JS to /tmp/raw_js/ and do static analysis with eslint-plugin-security

Case study

Now writting...

Contributing

1. Fork it (https://github.com/fukaoi/crystal-nodejs/fork)
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create a new Pull Request

Contributors

• fukaoi - creator and maintainer