Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(core): Replace Webpack with Rspack - siteConfig.future.experimental_faster.rspackBundler #10402

Draft
wants to merge 55 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
0dc5661
Rspack POC
slorber Aug 14, 2024
b1f4ae2
fix scripts
slorber Aug 14, 2024
ca7fefc
temp remove DefinePlugin
slorber Aug 14, 2024
127845d
fix pwa EnvironmentPlugin + CopyRspackPlugin
slorber Aug 14, 2024
c13d987
Add Rsdoctor plugin
slorber Aug 15, 2024
d08a465
remove RSDOCTOR by defaulkt
slorber Aug 15, 2024
2cfe4cd
remove useless exit + use Rspack minifiers
slorber Aug 15, 2024
8a2ab67
builtin:swc-loader
slorber Aug 15, 2024
6ec8d65
remove alias CssExtractRspackPlugin as MiniCssExtractPlugin
slorber Aug 16, 2024
64fdfcf
remove alias CssExtractRspackPlugin as MiniCssExtractPlugin
slorber Aug 16, 2024
4138492
Revert "remove alias CssExtractRspackPlugin as MiniCssExtractPlugin"
slorber Aug 16, 2024
486ee81
use LightningCssMinimizerRspackPlugin
slorber Aug 16, 2024
69f7d43
use RsdoctorRspackMultiplePlugin
slorber Aug 22, 2024
cbddd79
merge from main
slorber Sep 6, 2024
2e27a8a
upgrade and fix Rsdoctor?
slorber Sep 6, 2024
dacc090
upgrade Rspack to v1 stable
slorber Sep 6, 2024
fc9576b
Try to restore ChunkAssetPlugin for preloading
slorber Sep 6, 2024
a76cb73
Merge branch 'main' into slorber/poc-rspack
slorber Sep 10, 2024
b5c267a
Rspack 1.0.4
slorber Sep 10, 2024
2fff52f
damn, ChunkAssetPlugin still doesn't work
slorber Sep 10, 2024
456cde8
Merge branch 'main' into slorber/poc-rspack
slorber Sep 13, 2024
318c2df
fix merge issues
slorber Sep 13, 2024
3b5dbe5
try to fix Rspack build
slorber Sep 13, 2024
5e3fe51
add todo
slorber Sep 13, 2024
68b4bf6
add todo + disable gca
slorber Sep 13, 2024
fc08c5d
make it work
slorber Sep 13, 2024
e8272c4
build perf, log current bundler name
slorber Sep 13, 2024
0cee6bf
add RspackJsLoaderFactory
slorber Sep 13, 2024
6261ab5
simplify Rspack js loader
slorber Sep 13, 2024
bf2ad51
simplify Rspack js loader
slorber Sep 13, 2024
a06b194
fix lint error + wire rspack minimizer properly
slorber Sep 13, 2024
c8dc795
fix minimizer
slorber Sep 13, 2024
3b6987d
fix theme configureWebpack
slorber Sep 13, 2024
200ced7
fix Prism bundle size increase
slorber Sep 13, 2024
b4469fe
refactor: apply lint autofix
slorber Sep 13, 2024
0739fce
empty
slorber Sep 13, 2024
a608e88
Merge branch 'main' into slorber/poc-rspack
slorber Sep 26, 2024
5de89ab
handle merge conflicts
slorber Sep 26, 2024
d6b34a9
rspack 1.0.7
slorber Sep 26, 2024
c4881e7
attempt to restore ChunkAssetPlugin
slorber Sep 26, 2024
402d3ef
temp disable ChunkAssetPlugin
slorber Sep 26, 2024
74ddd14
refactor: apply lint autofix
slorber Sep 26, 2024
628980e
empty
slorber Sep 26, 2024
c66bfb4
fix createJsLoaderFactory test
slorber Sep 26, 2024
cb84db1
remove Rspack config validation errors
slorber Sep 26, 2024
3a202df
Use Rspack canary, restore ChunkAssetPlugin
slorber Sep 26, 2024
128bfd6
Merge branch 'main' into slorber/poc-rspack
slorber Sep 27, 2024
5bb213e
re-add missing rspack dep + fix lockfile
slorber Sep 27, 2024
4eacf9f
Wire browserslist to LightningCSS built-in targets
slorber Sep 27, 2024
e23b63e
restore dynamic require usage
slorber Sep 30, 2024
097ed54
Rspack 1.0.8
slorber Sep 30, 2024
1c48869
Merge branch 'main' into slorber/poc-rspack
slorber Oct 3, 2024
8093a21
Add useful comment for getBrowserslistQueries
slorber Oct 3, 2024
9eebd7f
Merge branch 'main' into slorber/poc-rspack
slorber Oct 3, 2024
4f40e3b
Merge branch 'main' into slorber/poc-rspack
slorber Oct 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 9 additions & 8 deletions packages/docusaurus-bundler/src/currentBundler.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import webpack from 'webpack';
import WebpackBar from 'webpackbar';
import MiniCssExtractPlugin from 'mini-css-extract-plugin';
import CopyWebpackPlugin from 'copy-webpack-plugin';
import logger from '@docusaurus/logger';
import {importRspack} from './importFaster';
import type {CurrentBundler, DocusaurusConfig} from '@docusaurus/types';

// We inject a site config slice because the Rspack flag might change place
Expand All @@ -32,10 +32,10 @@ export async function getCurrentBundler({
siteConfig: SiteConfigSlice;
}): Promise<CurrentBundler> {
if (isRspack(siteConfig)) {
// TODO add support for Rspack
logger.error(
'Rspack bundler is not supported yet, will use Webpack instead',
);
return {
name: 'rspack',
instance: await importRspack(),
};
}
return {
name: 'webpack',
Expand All @@ -49,7 +49,8 @@ export async function getCSSExtractPlugin({
currentBundler: CurrentBundler;
}): Promise<typeof MiniCssExtractPlugin> {
if (currentBundler.name === 'rspack') {
throw new Error('Rspack bundler is not supported yet');
// @ts-expect-error: this exists only in Rspack
return currentBundler.instance.CssExtractRspackPlugin;
}
return MiniCssExtractPlugin;
}
Expand All @@ -60,9 +61,9 @@ export async function getCopyPlugin({
currentBundler: CurrentBundler;
}): Promise<typeof CopyWebpackPlugin> {
if (currentBundler.name === 'rspack') {
throw new Error('Rspack bundler is not supported yet');
// @ts-expect-error: this exists only in Rspack
return currentBundler.instance.CopyRspackPlugin;
}
// https://github.com/webpack-contrib/copy-webpack-plugin
return CopyWebpackPlugin;
}

Expand Down
11 changes: 11 additions & 0 deletions packages/docusaurus-bundler/src/importFaster.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
* LICENSE file in the root directory of this source tree.
*/

import type webpack from 'webpack';
import type {ConfigureWebpackUtils} from '@docusaurus/types';
import type {
MinimizerOptions as JsMinimizerOptions,
Expand All @@ -29,6 +30,11 @@ async function ensureFaster(): Promise<FasterModule> {
}
}

export async function importRspack(): Promise<typeof webpack> {
const faster = await ensureFaster();
return faster.getRspack() as unknown as typeof webpack;
}

export async function importSwcJsLoaderFactory(): Promise<
ConfigureWebpackUtils['getJSLoader']
> {
Expand All @@ -50,6 +56,11 @@ export async function importSwcHtmlMinifier(): Promise<
return faster.getSwcHtmlMinifier();
}

export async function importBrowserslistQueries(): Promise<string[]> {
const faster = await ensureFaster();
return faster.getBrowserslistQueries();
}

export async function importLightningCssMinimizerOptions(): Promise<
CssMinimizerOptions<CustomOptions>
> {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,16 @@
* LICENSE file in the root directory of this source tree.
*/

import {fromPartial} from '@total-typescript/shoehorn';
import {fromPartial, type PartialDeep} from '@total-typescript/shoehorn';
import {createJsLoaderFactory} from '../jsLoader';

import type {RuleSetRule} from 'webpack';

describe('createJsLoaderFactory', () => {
function testJsLoaderFactory(
siteConfig?: Parameters<typeof createJsLoaderFactory>[0]['siteConfig'],
siteConfig?: PartialDeep<
Parameters<typeof createJsLoaderFactory>[0]['siteConfig']
>,
) {
return createJsLoaderFactory({
siteConfig: {
Expand All @@ -21,7 +23,12 @@ describe('createJsLoaderFactory', () => {
jsLoader: 'babel',
...siteConfig?.webpack,
},
future: fromPartial(siteConfig?.future),
future: fromPartial({
...siteConfig?.future,
experimental_faster: fromPartial({
...siteConfig?.future?.experimental_faster,
}),
}),
},
});
}
Expand Down
33 changes: 32 additions & 1 deletion packages/docusaurus-bundler/src/loaders/jsLoader.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@

import {getBabelOptions} from '@docusaurus/babel';
import {importSwcJsLoaderFactory} from '../importFaster';
import {getCurrentBundler} from '../currentBundler';
import type {ConfigureWebpackUtils, DocusaurusConfig} from '@docusaurus/types';

const BabelJsLoaderFactory: ConfigureWebpackUtils['getJSLoader'] = ({
Expand All @@ -19,6 +20,25 @@ const BabelJsLoaderFactory: ConfigureWebpackUtils['getJSLoader'] = ({
};
};

const RspackJsLoaderFactory: ConfigureWebpackUtils['getJSLoader'] = () => {
return {
loader: 'builtin:swc-loader',
options: {
jsc: {
parser: {
syntax: 'typescript',
tsx: true,
},
transform: {
react: {
runtime: 'automatic',
},
},
},
},
};
};

// Confusing: function that creates a function that creates actual js loaders
// This is done on purpose because the js loader factory is a public API
// It is injected in configureWebpack plugin lifecycle for plugin authors
Expand All @@ -27,11 +47,22 @@ export async function createJsLoaderFactory({
}: {
siteConfig: {
webpack?: DocusaurusConfig['webpack'];
future?: {
future: {
experimental_faster: DocusaurusConfig['future']['experimental_faster'];
};
};
}): Promise<ConfigureWebpackUtils['getJSLoader']> {
const currentBundler = await getCurrentBundler({siteConfig});
const isSWCLoader = siteConfig.future.experimental_faster.swcJsLoader;

if (currentBundler.name === 'rspack') {
if (!isSWCLoader) {
throw new Error(
'When using Rspack bundler, it is required to enable swcJsLoader too',
);
}
return RspackJsLoaderFactory;
}
const jsLoader = siteConfig.webpack?.jsLoader ?? 'babel';
if (
jsLoader instanceof Function &&
Expand Down
20 changes: 18 additions & 2 deletions packages/docusaurus-bundler/src/minification.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import CssMinimizerPlugin from 'css-minimizer-webpack-plugin';
import {
importSwcJsMinimizerOptions,
importLightningCssMinimizerOptions,
importBrowserslistQueries,
} from './importFaster';
import type {CustomOptions, CssNanoOptions} from 'css-minimizer-webpack-plugin';
import type {WebpackPluginInstance} from 'webpack';
Expand Down Expand Up @@ -137,8 +138,23 @@ async function getWebpackMinimizers(
async function getRspackMinimizers({
currentBundler,
}: MinimizersConfig): Promise<WebpackPluginInstance[]> {
console.log('currentBundler', currentBundler.name);
throw new Error('TODO Rspack minimizers not implemented yet');
const browserslistQueries = await importBrowserslistQueries();
return [
// @ts-expect-error: Rspack has this built-in
new currentBundler.instance.SwcJsMinimizerRspackPlugin(),
// @ts-expect-error: Rspack has this built-in
new currentBundler.instance.LightningCssMinimizerRspackPlugin({
minimizerOptions: {
...(await importLightningCssMinimizerOptions()),
// Not sure why but Rspack takes browserslist queries directly
// While LightningCSS targets are normally not browserslist queries
// We have to override the option to avoid errors
// See https://rspack.dev/plugins/rspack/lightning-css-minimizer-rspack-plugin#minimizeroptions
// See https://lightningcss.dev/transpilation.html
targets: browserslistQueries,
},
}),
];
}

export async function getMinimizers(
Expand Down
1 change: 1 addition & 0 deletions packages/docusaurus-faster/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
},
"license": "MIT",
"dependencies": {
"@rspack/core": "^1.0.8",
"@swc/core": "^1.7.28",
"@swc/html": "^1.7.28",
"browserslist": "^4.24.0",
Expand Down
14 changes: 14 additions & 0 deletions packages/docusaurus-faster/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,17 @@
* LICENSE file in the root directory of this source tree.
*/

import Rspack from '@rspack/core';
import * as lightningcss from 'lightningcss';
import browserslist from 'browserslist';
import {minify as swcHtmlMinifier} from '@swc/html';
import type {RuleSetRule} from 'webpack';
import type {JsMinifyOptions} from '@swc/core';

export function getRspack(): typeof Rspack {
return Rspack;
}

export function getSwcHtmlMinifier(): typeof swcHtmlMinifier {
return swcHtmlMinifier;
}
Expand Down Expand Up @@ -63,6 +68,15 @@ export function getSwcJsMinimizerOptions(): JsMinifyOptions {
};
}

// We need this because of Rspack built-in LightningCSS integration
// See https://github.com/orgs/browserslist/discussions/846
export function getBrowserslistQueries(): string[] {
const queries = browserslist.loadConfig({path: process.cwd()}) ?? [
...browserslist.defaults,
];
return queries;
}

// LightningCSS doesn't expose any type for css-minimizer-webpack-plugin setup
// So we derive it ourselves
// see https://lightningcss.dev/docs.html#with-webpack
Expand Down
1 change: 1 addition & 0 deletions packages/docusaurus-theme-classic/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
},
"dependencies": {
"@docusaurus/core": "3.5.2",
"@docusaurus/logger": "3.5.2",
"@docusaurus/mdx-loader": "3.5.2",
"@docusaurus/module-type-aliases": "3.5.2",
"@docusaurus/plugin-content-blog": "3.5.2",
Expand Down
9 changes: 7 additions & 2 deletions packages/docusaurus/src/client/docusaurus.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ const loaded = new Set<string>();

declare global {
// eslint-disable-next-line camelcase, no-underscore-dangle
const __webpack_require__: {gca: (name: string) => string};
const __webpack_require__: {gca?: (name: string) => string};
interface Navigator {
connection?: {effectiveType: string; saveData: boolean};
}
Expand Down Expand Up @@ -65,17 +65,22 @@ const prefetch: Docusaurus['prefetch'] = (
// Prefetch all webpack chunk assets file needed.
return Promise.all(
chunkNamesNeeded.map((chunkName) => {
// TODO
if (!__webpack_require__.gca) {
console.warn('Rspack bug to fix: unable to prefetch');
return Promise.resolve();
}
// "__webpack_require__.gca" is injected by ChunkAssetPlugin. Pass it
// the name of the chunk you want to load and it will return its URL.
// eslint-disable-next-line camelcase
const chunkAsset = __webpack_require__.gca(chunkName);

// In some cases, webpack might decide to optimize further, leading to
// the chunk assets being merged to another chunk. In this case, we can
// safely filter it out and don't need to load it.
if (chunkAsset && !chunkAsset.includes('undefined')) {
return prefetchHelper(chunkAsset);
}

return Promise.resolve();
}),
);
Expand Down
19 changes: 8 additions & 11 deletions packages/docusaurus/src/commands/build.ts
Original file line number Diff line number Diff line change
Expand Up @@ -189,17 +189,14 @@ async function buildLocale({
);

// Run webpack to build JS bundle (client) and static html files (server).
await PerfLogger.async(
`Bundling with ${configureWebpackUtils.currentBundler.name}`,
() => {
return compile({
configs:
// For hash router we don't do SSG and can skip the server bundle
router === 'hash' ? [clientConfig] : [clientConfig, serverConfig],
currentBundler: configureWebpackUtils.currentBundler,
});
},
);
await PerfLogger.async(`Bundling with ${props.currentBundler.name}`, () => {
return compile({
configs:
// For hash router we don't do SSG and can skip the server bundle
router === 'hash' ? [clientConfig] : [clientConfig, serverConfig],
currentBundler: configureWebpackUtils.currentBundler,
});
});

const {collectedData} = await PerfLogger.async('SSG', () =>
executeSSG({
Expand Down
3 changes: 1 addition & 2 deletions packages/docusaurus/src/commands/start/webpack.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@

import path from 'path';
import merge from 'webpack-merge';
import webpack from 'webpack';
import {formatStatsErrorMessage, printStatsWarnings} from '@docusaurus/bundler';
import logger from '@docusaurus/logger';
import WebpackDevServer from 'webpack-dev-server';
Expand Down Expand Up @@ -168,7 +167,7 @@ export async function createWebpackDevServer({
configureWebpackUtils,
});

const compiler = webpack(config);
const compiler = props.currentBundler.instance(config);
registerWebpackE2ETestHook(compiler);

const defaultDevServerConfig = await createDevServerConfig({
Expand Down
21 changes: 15 additions & 6 deletions packages/docusaurus/src/webpack/base.ts
Original file line number Diff line number Diff line change
Expand Up @@ -98,10 +98,14 @@ export async function createBaseConfig({
currentBundler: props.currentBundler,
});

return {
mode,
name,
cache: {
function getCache(): Configuration['cache'] {
if (props.currentBundler.name === 'rspack') {
// TODO Rspack only supports memory cache (as of Sept 2024)
// TODO re-enable file persistent cache one Rspack supports it
// See also https://rspack.dev/config/cache#cache
return undefined;
}
return {
type: 'filesystem',
// Can we share the same cache across locales?
// Exploring that question at https://github.com/webpack/webpack/issues/13034
Expand All @@ -127,7 +131,13 @@ export async function createBaseConfig({
siteConfigPath,
],
},
},
};
}

return {
mode,
name,
cache: getCache(),
output: {
pathinfo: false,
path: outDir,
Expand All @@ -145,7 +155,6 @@ export async function createBaseConfig({
},
devtool: isProd ? undefined : 'eval-cheap-module-source-map',
resolve: {
unsafeCache: false, // Not enabled, does not seem to improve perf much
extensions: ['.wasm', '.mjs', '.js', '.jsx', '.ts', '.tsx', '.json'],
symlinks: true, // See https://github.com/facebook/docusaurus/issues/3272
roots: [
Expand Down
2 changes: 1 addition & 1 deletion packages/docusaurus/src/webpack/client.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ async function createBaseClientConfig({
});

const ProgressBarPlugin = await getProgressBarPlugin({
currentBundler: configureWebpackUtils.currentBundler,
currentBundler: props.currentBundler,
});

return merge(baseConfig, {
Expand Down
2 changes: 2 additions & 0 deletions packages/docusaurus/src/webpack/plugins/ChunkAssetPlugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ export default class ChunkAssetPlugin {
}

// Inspired by https://github.com/webpack/webpack/blob/v5.94.0/lib/runtime/CompatRuntimeModule.js
// See also https://rspack.dev/api/javascript-api/compilation#addruntimemodule
// See also https://rspack.dev/api/plugin-api/compilation-hooks#additionaltreeruntimerequirements
class ChunkAssetRuntimeModule extends webpack.RuntimeModule {
constructor() {
super('ChunkAssetRuntimeModule', webpack.RuntimeModule.STAGE_ATTACH);
Expand Down
1 change: 1 addition & 0 deletions website/docusaurus.config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import ConfigLocalized from './docusaurus.config.localized.json';

import PrismLight from './src/utils/prismLight';
import PrismDark from './src/utils/prismDark';

import type {Config, DocusaurusConfig} from '@docusaurus/types';

import type * as Preset from '@docusaurus/preset-classic';
Expand Down
Loading
Loading