ES Module Scripts

[willeastcott]

The current PlayCanvas script format was introduced in July 2016. PlayCanvas scripts are snippets of JavaScript that are (asynchronously) loaded and then, once loaded, immediately executed in global application scope.

The current PlayCanvas script format has a number of problems:

1. The current default script template is still generated based on the ES5 feature set of JavaScript. This is looking increasingly antiquated in a world where ES Modules have become dominant. JavaScript has moved on considerably over the last 10 or so years.
2. Developers want to leverage tree shaking to generate an application bundle that only includes the code necessary to execute the application. The current script format is not compatible with tree shaking because it does not specify the specific symbols on which the code relies. ESM does this with the import statement.
3. Developers want to be able to run a build system (like Rollup, Webpack etc). This would allow them to run transpilers (e.g. Babel), minifiers (e.g. Terser) and so on. These build tools (or bundlers) tend to take ES Modules as input. Our current script format does not play nice with these bundlers.
4. While you can run PlayCanvas in Node if you use the ES5 build of the engine, the current script system cannot be used with the module build (or if you run the source directly in Node, as with our engine unit tests). When the scripts are loaded (by doing a dynamic import call), the pc namespace doesn't exist. The script would need to import what it needs.
5. Intellisense in VS Code (or the Code Editor's Monaco) does not work too well with ScriptType-based scripts. For example, attributes don't resolve correctly. An ES Module containing a class might be a better approach.

So I propose that we investigate the development of a new script format that:

• Is based on ES Module format.
• Is based on ES6 class syntax.
• Supports dynamic loading (say when running the Launch tab from the PlayCanvas Editor).
• Supports easy-to-use hot reloading.
• Supports being built into a bundle (by the likes of Rollup etc).
• Supports Intellisense well in VS Code/Monaco.
• Can be run in Node when running the engine as a module or from source modules (this is critical for unit testing in Node, say).
• Is compatible with TypeScript.

I welcome comments from the community on this proposal.

[Maksims]

Would be great to see an example of most minimal script, as well as a script that uses attributes, and some few other features in the code.

[nikitaosyak]

Hi Will!

I'm thrilled to see open discussion about it. Let me share how we do stuff, working around pc quirks.

We use webpack bundling and imports inside webstorm IDE, which works very good with jsdoc, so we have full intellisense as if working with typescript (well, almost)

We also use inheritance in rare cases, also with attributes, and here is how:

// imports work fine with webpack
import UtilShader from "../UtilShader.js"; 

//
// for attributes to make it into intellisense, we have to annotate 
// them with @property keyword. that's a chore, but works fine. 
// Would like to see it in some better form
/**
 * @class {PostEffectBase}
 * @property {boolean} skip
 */
//
// in webstorm, all playcanvas classes are available from 
// pc.* "namespace", so we can intellisense any type, 
// including entities, colors, curves, etc
export default class PostEffectBase extends pc.ScriptType {

    //
    // this is a string for script registry, for better 
    // refactoring purposes, also we can have generic 
    // getComponent with ClassName.pcName using this 
    static pcName = 'postEffectBase' 

    //
    // here we assign attributes in such a way that it can
    // be both added to descendant of current class, otherwise
    // attributes would not be able to propagate
    /**
     * @param {pc.ScriptType} v
     */
    static registerAttributes(v) {
        v.attributes.add('skip', {type : 'boolean', default : false})
    }

    //
    // we can have some class members, protected or private,
    // based on _ or __ notation, but in most cases I don't
    // care for access modifiers :)
    /** @type {pc.RenderTarget} */
    targetInput = null
    
    //
    // ofc we will have script lifecycle methods, which works fine
    initialize() {
        // do something here
    }
    
    //
    // we can have methods with typed arguments
    // and return values, which works fine
    /**
     * @param {pc.RenderTarget} v
     */
    setRenderTarget(v) {
        this.targetInput = v
    }
}

//
// finally, register resulting scripts in playcanvas registry
pc.registerScript(PostEffectBase, PostEffectBase.pcName)

// register attributes for this specific class,
// descendant will use both base class register attributes, and
// own register attributes
PostEffectBase.registerAttributes(PostEffectBase)```

[nikitaosyak]

we also don't use script hotswap, since it's a chore to write around, since I have to lay off my current state, save it and return to new script, but we do use swap method to live-reload whole app after any change, since we're using file watchers and rest API to load bundle into playcanvas editor. for that, we have reload on swap script:

const ReloadByswap = pc.createScript('reloadByswap');

ReloadByswap.prototype.swap = function(old) {
    window.location.reload();
};

export default ReloadByswap```

since it's included in bundle which uploads to editor, every time we change stuff, dev build is being reloaded :)

[nikitaosyak]

regarding vs code or monaco I don't really know why use them. I would better look at IDE headliner, like intellij, since they are just better IDE's, and they have better intellisense. but that's like my humble opinion.

[yaustar]

Be able to have the script attributes auto complete with type in any code editor. The way they are currently, doesn't allow code editors to infer the property or the type

[yaustar]

There is another issue with the Editor that we would need to consider, there's no real folder structure. Each asset is in a folder with the name as the resource id. This would make importing other scripts/files nearly impossible unless we change this somehow or have a build step for each launch

[willeastcott]

@yaustar That's a really good point. Obviously, modules need to specify import paths. So I'm not sure how we'd support that...

[yaustar]

Did the old Scripts V1 workflow and the 'scripts folder' specialised in that respect where they were stored differently? Not entirely sure if that's something we want to go back to.

[Maksims]

For imports, worth looking at import maps.

If writing a script would require to import all dependencies instead of current approach, this will have worse learning and usability curve.

[marklundin]

This is really exciting to see. I've spent a bit of time thinking about this so here's a braindump...

Ideas for a Module Type

Just as a thought starter, I've always thought Decorators are a good candidate for this kind of thing. Similar to Unity Attributes Consider the following:

import { attribute, inspectable } from 'playcanvas'

@inspectable("My Awesome Class", "This description is surfaced to the editor")
class MyClass {

  @attribute("This attribute title is exposed to the editor")
  nSomeValue= "This is a default value"

  initialize(){}
  destroy(){}
  update(){}
}

Of course this is still a Stage 3 Proposal so without TypeScript or other transpiler the same code above would reduce to a slightly more clunky:

import { attribute, inspectable } from 'playcanvas'

class MyClass {

  nSomeValue= 'This is a default value'

  initialize(){}
  destroy(){}
  update(){}
}

inspectable("My Awesome Class", "This description is surfaced to the editor")(MyClass)
attribute('This title is exposed to the editor')(MyClass.prototype.nSomeValue)

The decorators are composable so you can includes things like constraints. The nice thing about this is it's self documenting.

Thoughts on ES Modules

Module resolution Problem
Module paths should resolve in a way synonymous to the asset registry. Unfortunately Import Maps won't work here as it apparently only allows you to map an import to a url so that:

import moment from 'moment'

becomes

import moment from 'https://unpkg.com/moment'

It doesn't allows you to modify the path resolution algorithm so that a relative path like '../../path/to/module' resolves to 'launch.playcanvas.com/api/assets/files/asset/script.js?id=123&branchId=231'

How to solve this:

1. HTTP Server. Update the backend to serve scripts in the same virtual filesystem as the asset registry. This may be the most ideal scenario as it removes the need to do any import re-mappings, however there's likely legitimate reasons why this isn't already the case.

2. Pre-Processor. A build step that resolves and rewrites paths ahead-of-time or at run time. Doing this at runtime though is only likely to increase app startup time as you'd need to traverse the dependency tree. So there is an alternative argument for doing this ahead-of-time as a pre-processing step using a well supported transpiler. This also opens the door for things like TypeScript and JSX and other goodness.

Transpilers
Bundling may not be strictly necessary when using ES modules and HTTP2, however a pre-processing step has other benefits. For example you can also support things like TypeScript or JSX, inject environment variables, strip out non production code etc. There are obvious complexities to this and it's likely to be outside the scope of the current issue, but in our experience developing the PC Package Manager it's still very much desirable and doable at edit time.

Potential issues with ES Modules

• Module scope
  Some scripts in existing projects may depend upon implicitly defined global variables in other scripts. Scripts all share the same global scope however modules have their own scope so adding type='module' to existing project scripts may break things. Users may have to explicitly opt-in to modules

• Error Handling
  In the PC launcher there is a global error handler that catches any error and provides a useful mechanic to link back to the source code in the Code Editor. IIRC this depends on the assetID of the script and there may be issues retrieving this if it's not explicitly defined part of the URL query string. I may be totally wrong on this

• CDN
  This is more of a dev consideration, but ES modules will allow developers to import modules from a CDN using:

import moment from 'https://unpkg.com/moment'

There are obvious pro's/cons in relying on 3rd party CDN's at run time. Some issues being outages and also allowing non version locked modules which can suddenly be upgraded and subsequently introduce breaking changes for your app. This is not obviously something specific to ES modules per-se but the ability to import modules directly is likely to make the problem more prevalent. This is a persuasive argument for bundling 3rd party dependencies so you always have a static shippable build