Standard compliant ES Modules format

[Buslowicz]

Feature Request

RxJS should come in a standard compliant (browser-friendly) ES Modules format.

Describe the solution you'd like
The _esm5 and _esm2015 are close to what we need, but still far from standards. First of all, ES Modules means the browser definitely supports ES6, so no need for esm5 output to be honest.
Next, the _esm2015 format still misses .js file extensions and without them it is impossible for the browser to know what are we trying to load (it works literally as with script src or css link, it needs to point to the exact file, browser will not check for multiple locations as each check has a network latency cost). Furthermore, the _esm2015/internal/Rx.js file has bunch of imports (rxjs-compat) based on Bare Module Specifier (which is strictly a node.js resolution system, not existing in the browser).
In order for the format to be standard compliant, it needs to point to exact file on the server.

Describe alternatives you've considered
So far the only 2 ways of dealing with the issue is to have the server that rewrites the path or bundle the output. As bundling is often a must on production anyway, it is not needed on many development workflows, as it would only slow down the browser reactivity to changes. Path rewrites on-the-fly however work only for custom servers. Some projects use existing tools and writing custom logic just makes more code to maintain, not to mention possible bugs slowing down the development. It would be great to just fetch the file from cdn and not care about further post processing.

This issue has been raised already in #2858 and #3858 but so far both issues are closed and locked as resolved (which is not how it is). Back then browser support for ES Modules could have been lower, but at this moment we have all of the major browsers having a stable implementation enabled by default which gives over 80% of the browsers supporting the feature (node.js also supports it now behind the flag). I really think supporting this format should be a high priority for rxjs.

P.S. Adding module field to package.json would also be a nice addition to the above.

[kwonoj]

Yes, one of issue closed unexpectedly and new issue would be ok to track down further.

It is no doubt we will support native es module, but there are some things need to be resolved including issues you described already. In major, we probably will visit this with upstream support arrives microsoft/TypeScript#16577 , rather than trying to having custom postbuild processing.

[Buslowicz]

The provided issue on TypeScript GH doesn't really bring any hope, quite opposite actually. Is there any chance to push it forward with rxjs itself? If not with this repo, maybe with a separate one (like it used to be with rxjs-es)?

Side note: just thought of how nice (in my personal opinion) would it be to have separate format packages like @reactivex/esm, @reactivex/umd etc :).

[kwonoj]

Rxjs-es was not native module, and from experiences of 5 having separate packages we decided not to separating packages. This is obviously different scope to this issue though.

For supporting es modules, I'd like to understand how urgent this is: yes, major browser supports it, it's capability allows we can support this further. But internal organization of our export / import, and surroundings like compat package is non trivial work to make this happen in real world.

In short,

• what's specific usecases of native module?
• and what prevents to use current module system for those?

As said it is no doubt we will / have to support this eventually but this is sinilar to node side mjs support as well: feature is (somewhat) there, but lacks of toolling and surroundings. If we have strong reason we have to support this in like next major (7) we may need to think how we can do it out of current tooling support, but I am bit doubt about those yet.

[Buslowicz]

Well, the reasoning is dev environment without tools.
With the current state of ES features implemented across modern browsers, projects could be built without any sort of build process which simplifies maintenance by a lot, also reducing number of possible bug sources. Obviously production will heavily benefit from building (due to tree shaking and other code optimizations), but development could be simplified by a lot (even allowing to edit source files directly via chrome dev tools).
Moreover having native modules would allow the use of custom cdn (unpkg sometimes has performance issues).

Current module system aren't native and require 3rd party libraries to make it work. For a project using es modules, the only way for development is to use custom servers (or write additional custom logic for own server if project has one) or use bundlers like webpack, which requires additional configuration and thus consumes project time + makes it more complex/error prone.

I understand many (if not most) projects currently is based on bundlers, but it is because we didn't have native modules before. Now we have them and development cycle will start to change. Having more big libraries (like RxJS) be compatible with it would make the transition faster.

[kwonoj]

but it is because we didn't have native modules before. Now we have them and development cycle will start to change

100% agreed. It's just matter of when. Still, I feel it's bit rough & edgey to make changes right away. All details you mentioned are good, but I doubt you can immediately changes your dev pipeline even if rx ships with es module right away - there are other dependencies doesn't have es module right now that you can't use, isn't it? I agree having some big player will give an acceleration of those trends though.

In short, clarifying, I am not opposed to this implementation itself but I would like to have proper support from upstream & release proper module build doesn't potentially breaking / regresses by having internal tooling to workaround.

[Buslowicz]

As to me, I am currently working on an RxJS based Web Components base class where I am trying to limit the npm dependencies to absolute minimum and try to utilize the browser native api as much as possible (currently got literally 4 folders in node_modules aside of dev dependencies like karma etc). Things will probably grow, but so far everything all I needed was RxJS (1 dep), lit-html (0 deps) and immutable (0 deps) to build a full app. Aside of RxJS, everything works natively in the browser, my custom server fixes paths just for RxJS and I would prefer to not maintain its code :).

I fully understand you prefer to wait a bit, was just hoping anything can be done as a temporary solution. If not, well I guess all we can do is wait.

[Buslowicz]

Hi there, a little update on the TS side of this: microsoft/TypeScript#16577 (comment)
According to a Program Manager of TypeScript they will most likely not implement this, so it's all up for rxjs (and others) to handle this.
If you need more hands on this (either to plan/discuss the functionality or with pure coding), I am here to help, just no rush, I understand this is not a highest priority.

[Buslowicz]

Hi @kwonoj did anything change on your side about the above? I've hit this again as a partial blocker and am wondering how far from resolving it is.

[frehner]

Ah, just ran into this trying to use the ESM version of the build. Seems... weird to have an esm build but not actually have it work? 🤷‍♀

[Buslowicz]

This should really be fixed on rxjs level, typescript will not implement it on their side, and it would be good to have rxjs browser friendly.

[kwonoj]

@frehner minor nit but we never provided native esm, our pkg clarifies it's es-import syntax module only. those 2 are different.

---

I'm keeping an eye on this issue and with node.js's native esm supports without .mjs extension give a few attempts. Unfortunately, until several upstream issues resolved it is not possible to publish our package correctly can be consumed in browser / node.js side.

When issue first created I assume tsc extension issue is only problem but it wasn't - node.js side module property to allow module resolutions, and properly importing esm across our references, make testing work (node.js won't allow native esm module import via require cjs), etcs are all blockers to proceed. That's only coming from quick poc attempts, so maybe there should be more issues coming after even.

Once all these blockers are good to go, we'll make attempt to create major breaking semver with alpha to try out. Sadly, it won't happen soon in current state in my opinion.

[Buslowicz]

That's understandable. Well, fingers crossed to have it fixed as soon as possible, even though it's not going to be soon.

[frehner]

@frehner minor nit but we never provided native esm, our pkg clarifies it's es-import syntax module only. those 2 are different.

Hm, I'm probably reading this wrong then:

Published along with rxjs 5.5 is builds of rxjs in ECMAScript Module format (imports and exports) with both ES5 and ES2015 language level.

But in any case, I understand it may require some work. Thank you for your response.

[Grunet]

For anyone looking for a more explicit workaround to this like I was, I made the following changes based on the comments above so I could test without needing to bundle.

1. I switched my imports to point directly at the index.js under the _esm2015 folder

import { Observable, Subject } from "../../node_modules/rxjs/_esm2015/index.js";

2. I setup a local web server with express (the Hello World example modified to serve up my static files) and added in this piece of middleware to rewrite the extensions

const trimEnd = require("lodash.trimEnd");
...
app.use("/node_modules/rxjs/", (req, res, next) => {
  if (!req.url.includes(".js")) {
    req.url = trimEnd(req.url, "/") + ".js";
  }

  next();
});

That seems to work (for my use case at least) from what I can tell. Would be curious to know of any other/better workarounds I could/should be using in the meantime

[Grunet]

For Node (v13.3.0 at least), I found a (local) workaround that seems to work ok for me without requiring any modifications to rxjs's code's imports. In addition to step 1 above, I did the following:

2. Update rxjs's package.json to declare itself as a module + indicate what the es6 import entry point should be (based on guidance from the node es6 module docs here) in addition to the "module" field pointing at the es5 build that's already there for bundlers

"type": "module",
"exports": {
  "import": "./_esm2015/index.js"
},

3. Run node with the --es-module-specifier-resolution flag set to "node" (based off of this part of the node es6 module docs, but with the renamed flag)

node --es-module-specifier-resolution=node moduleScriptToRun.js 

At the very least that seemed to resolve the Node module loader errors I was running into. But from reading through this issue's discussion, I think I understand now that it's potentially not as simple as updating the package.json and that more testing in a Node es6 module context might be needed to make sure rxjs works without issue.

[joeldenning]

FYI for anyone looking for a version of rxjs that works in browsers with a standard import, check out https://github.com/esm-bundle/rxjs. It autopublishes new versions as rxjs is published. It provides only a few bundles, instead of individual javascript files. But for those that find that interesting, I've used it for a while now and it works 😄.

Example usage (try it in the browser console!):

(async () => {
  const rxjs = await import('https://unpkg.com/@esm-bundle/rxjs/esm/es2015/rxjs.min.js')
  const operators = await import('https://unpkg.com/@esm-bundle/rxjs/esm/es2015/rxjs-operators.min.js')

  console.log(rxjs, operators)
})()

Code Sandbox demonstration

There's even both es2015 and es5 versions for all the Angular differential loading people :)

[vwkd]

This is really unfortunate. They should switch to ESM modules.

Also because their README doesn't work as described. Right now it says npm install rxjs and a few lines later import { range } from "rxjs"; which as of now doesn't work, because RxJS still uses the old exports object

[csvn]

This issue is not only for browsers. Node v14 has unflagged native ESM modules, and v14 is nearing LTS release. It would be cool if RxJS was to work with both browsers and node out of the box with ESM syntax, with the additions of file extensions on files and imports. It's probably possible to do as a build step.

[kwonoj]

We have internal goals to make this happen indeed, but we can't guarantee to align with node 14's LTS or provide specific timeframe. There were couple of attempt to make pkg isomorphic to provide cjs / esm compliant both but it was way tricker with how rxjs is currently packaged.

[csvn]

Alright, thanks for your response! RxJS is indeed a large project with many different build targets. It's nice to hear that it's under consideration ✌

[chuanqisun]

Another work around is using a service like Skypack. (just open https://cdn.skypack.dev/rxjs in your browser, you'll know what I mean).

It's basically the hack from @Grunet but the service resolves the paths and caches the result for you on a CDN.

You might need some additional manual type stitching to get intellisense working.

@Draccoz , I came across this thread as I was trying to pull off a build-less project with web components and lit-html. I feel the web community is near a turning point to embrace native esm imports. I agree with @kwonoj that other libraries probably don't have native esm support either. But that's a chicken and egg problem for bootstrapping any new standard. Pika CDN (now Skypack) seems to be doing meaningful work to auto convert as many modules as possible. Given that rxjs (at least v5) had the spirit of pushing Observable into JavaScript Standards with TC39, it would make total sense to also embrace other standards such as native esm modules. Fingers crossed we can get rxjs running as native modules in the near future. And thank you for the amazing work rxjs community!

[Buslowicz]

Meh, closed by accident.
Yea, had the identical project idea (web components + lit + RxJS), native modules would be fantastic. For now I'll stick to my simple dev server with path rewriting. Will see which direction to head towards later.

[benlesh]

Ugh... accidentally refiled here: #6030

[flash-me]

With import-maps being landet in Chrome 89, there is higher interest for ESM.
TBH, rxjs is the only package in the angular toolchain that doesn't work out of the box with import-maps.
The workaround is to rollup rxjs and rxjs/operatos and provide that bundle (like FESM2015).

I'm not getting what the real challenge here is.
Adding a .js file extension to the imports shouldn't be that hard, or am I missing sth?

cheers
flash ⚡

[Grunet]

This issue seems to hold most of the discussion around adding js file extensions. It still seems to be ongoing but this maybe relevant comment was the last one before it was closed.

[Grunet]

Also FYI for anyone else in the "browser dev environment" + "doesn't want to bundle" (and not using an ES module CDN tool like Skypack) boat, but the "use a web server to rewrite the paths to include the file extensions" trick doesn't seem to work as of recent versions (at least 6.5.5 according to this issue) because of a new dependency on tslib.

I went ahead and updated my express middleware workaround mentioned above to also rewrite all the from "tslib" statements into from <relative path to the tslib ESM bundle>, and it seems to be working okay now

const path = require("path");
const fs = require("fs");
const slash = require("slash");
const trimEnd = require("lodash.trimEnd");
...
app.use("/node_modules/rxjs/", (req, res, next) => {
  if (!req.url.includes(".js")) {
    req.url = trimEnd(req.url, "/") + ".js";
  }

  const absPathToRxJsFile = path.resolve(
    path.join(staticAssetsRootDir, "node_modules/rxjs", req.url)
  );

  fs.readFile(absPathToRxJsFile, (err, data) => {
    if (err) {
      next(err);
      return;
    }

    const absPathToTsLibFile = slash(
      path.join(
        path.relative(absPathToRxJsFile, staticAssetsRootDir),
        "node_modules/tslib/tslib.es6.js"
      )
    );

    const modifiedRxJsFileText = data
      .toString()
      .replace(/from "tslib"/g, `from "${absPathToTsLibFile}"`);

    res.setHeader("content-type", "text/javascript");
    res.send(Buffer.from(modifiedRxJsFileText));
  });
});

("staticAssetsRootDir" is the absolute path for the folder containing the "node_modules" folder)

Also related FYI but the folder containing ES module versions of rxjs moved in v7 (it seems to be under dist/esm/ for ES6 or dist/esm5/ for ES5 now) so all the import paths I mentioned in my previous workaround comment would also need to be updated

[benlesh]

Core Team: We've decided to wait to see how the ecosystem evolved around this for now. That's not to say it's not going to happen. Just we're not ready to deal with this just yet.

[Buslowicz]

On over 2 years ticket :D but yea, I get it, it's better to wait than to get into unnecessary maintenance and/or standard compatibility issues.

[oldmansutton]

It's been over a year since the last comments, so... any update? I was using the UMD bundle for the longest time, but in efforts to modify our code base, everything is going ES6. I resisted for a long time, but here we are. Rxjs is the last piece of my puzzle (but arguably the most important).

[Buslowicz]

Luckily I'm stuck with Angular now, so don't need to wait for it, but would be good to at least know some estimates. 3 years is quite some time for that to implement, especially that all it needed (back then, didn't have time to check now), was remove the tslib and add file extensions, which could be done with a prepublish npm script. If help is needed, I'll be happy to contribute.

[imcotton]

There is one installation guide¹ on mentioning how to fit with bundle tooling regards to es2015 (and ESM) been added in PR #6637, at least it works as a workaround for now.

Footnotes

1. https://github.com/ReactiveX/rxjs/blob/master/docs_app/content/guide/installation.md ↩

[oldmansutton]

I've been using pure JavaScript for over 25 years now (no need to transpile), and don't use bundlers (less need to since http/2 came out as well), so that doesn't really help. I normally try to avoid Node like the plague. ALL JS I write is for browser use only.

@joeldenning comment (#4416 (comment)) pointed out a very great solution, however. The esm-bundle/rxjs repo works perfectly.

EDIT: esm-bundle version websocket doesn't always connect, it's hit or miss. Being able to just do
import { webSocket } from 'rxjs/webSocket';
in a es6 browser module without node shenannigans, transpilation from typescript or having to use a bundler should be first thing that works. That's like, web 101.

[muratcorlu]

Do you consider using esbuild to create ESM bundle? It support TS, it's fast, it's a single command so easy to implement. If that's an option, I can try to make a PR.

[braebo]

I was wondering why rxjs is blowing up my project with exports is not defined.

[oldmansutton]

@muratcorlu ECMAScript Modules don't need to be bundled, the browsers have native support now for importing modules without all these unnecessary bundling steps and libraries. I feel like that point is largely lost on the rxjs team, who seems to be sticking with a 2010's based idea of web development, but marketing it as real ES Modules. But.... they're not, because you can't just import the module in a browser, because it's still relying on bundlers/packagers/Node, whatever. It's not a native ESM yet.

The good news is that modern browsers have started to support module functionality natively, and this is what this article is all about. This can only be a good thing — browsers can optimize loading of modules, making it more efficient than having to use a library and do all of that extra client-side processing and extra round trips.

[SamMousa]

There is a 3rd party browser ESM bundle that we use.

https://www.npmjs.com/package/@esm-bundle/rxjs

Read more stuff here: https://medium.com/@joeldenning/an-esm-bundle-for-any-npm-package-5f850db0e04d