[Perf] Introducing static bundle

[arunoda]

This is a performance specific configuration which gives us two major performance benefits:

1. Reduce webpack's dev re-build time by 2x (or more)

Webpack's rebuild time usually depend on the amount of code it needs to handle. Basically, most of those code is from NPM modules but not from your app's source code. With static modules, we are moving those NPM modules out of the webpack builder and serves directly.

With that change, webpack only handles your app's code. So, that's where we could gain the re-build performance.

2. Explicitly define common modules

Our code splitting logic moves any common modules into a common bundle. But in order to do that, that module needs to be used in every page in our app. Practically, that's something hard to do.

Because of that, some of the most used modules won't move into the common bundle and keep a copy of that module in each and every page in your app. That increases the overall size of the app.

In that case, we could mark that module as a "static module" and then it'll move into the static bundle.

Usage

Add following content into your next.config.js.

module.exports = {
  staticModules: function(options) {
    // use options.dev flag to add modules to the bundle based on the env.
    return [
      'material-ui',
      'material-ui/svg-icons'
    ]
  }
}

You could use this app to benchmark this PR.
(Thanks @tgoldenberg for the sample repo)

[timneutkens]

This is amazing @arunoda 😮 checking now.

[timneutkens]

usually depends

Instead of:

Basically, most of those code is from NPM modules but not from your app's source code. With static modules, we are moving those NPM modules out of the webpack builder and serves directly.

Maybe this:

Most of the code it needs to handle is not your app's source code, but NPM modules. With static modules we are serving those NPM modules directly, instead of passing them through webpack.

[timneutkens]

From:

Practically, that's something hard to do.

To:

From experience, that is something that almost never happens.

[timneutkens]

Apart from the few small things I noticed in the readme this looks really good. Great job! 👍

[nkzawa]

The Idea seems great 👍 I wonder if we can use DLL plugin of webpack for this purpose 🤔

https://github.com/webpack/docs/wiki/list-of-plugins#dllplugin

I'm not fully understanding how it works yet though.

[arunoda]

@nkzawa DDL plugin uses something similar. (It complicated things a bit)
Anyway even if we do that, we need to do a webpack build for that.

We don't need that and this is something we've already done with our CDN bundle and
we know it's working pretty well.

[arunoda]

Thanks @timneutkens. I pick your changes.

[nkzawa]

@arunoda What I expect is cache of webpack works and don't rebuild dll files at all when the plugin is enabled. Then basically it would be the same with what this PR does ?

Anyway I wonder if this can be a webpack plugin since this is a workaround for webpack. I feel t's a bit weird to have settings on next.js.

[nkzawa]

It seems DLL plugin works a bit differently than what I expected, but using the plugin is a recommended way anyway ?

[arunoda]

It seems DLL plugin works a bit differently than what I expected, but using the plugin is a recommended way anyway ?

I don't think so. Anyway, it complicate stuff.
A nice option is I provide a list modules like this and webpack create this static bundle itself.

I'm not sure we can do it with a plugin or not.

[frol]

I am not familiar with webpack to judge this, so I will only put my concern here.

Can't static-bundle.js get merged with commons.js and main.js? It looks like all three of these are included on every page anyway, so splitting them up makes 3 separate requests.

[arunoda]

Technically, these three are serving three different purposes. It's possible to merge to all.
But with HTTP2, having multiple files is better.
(Even without that, browsers starts to download them in parallel)

So, it's better to have many files rather one.

[rauchg]

@arunoda since the execution of the files has to be sequential, is it really better to have 3 files?

[arunoda]

@rauchg but the parsing time it so small compare to network time. So, it shouldn't be a much of a problem.

Anyway, I am not disagree for having just a single build file as that's how most of the other frameworks does.

Shall we do this on a different issue/PR?

[frol]

This implementation makes it one step closer to slimmer apps, but how would you recommend to handle the use-case when there are groups of pages having heavy dependencies? Here is an example:

I have an app, which uses UI framework (e.g. React-MD), so it makes sense to bundle it into static. Then, I have a submodule with a few pages, which heavily use charts, so I use a charting library and I want it to be a common dependency for the submodule. And just for the sake of example, there is another submodule with a few pages, which makes heavy map interactions. Thus, charts are only relevant to the first submodule, and maps are only relevant to the second. Putting those into the static ruins the whole point of code-splitting...

[arunoda]

@frol yes. Basically you've to balance between moving to static bundle or keeping them in each page. And we'll have dynamic imports(#728) soon. That'll also helps on this matter.

[frol]

It is unfortunate that this is marked as 2.1 feature because 2.0 hasn't even released yet, so it sounds like a "far future". However, for the "far future" I would love to see a more flexible implementation, e.g. it would be nice if you can specify "sub-static" bundles in addition to static bundles, which would be shared with just a few of the pages (dynamic imports are still nice, but the out of the box solution for intelligent bundling is nicer).