Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(rollup.config): improve tree-shaking using preserveModules: true #1133

Merged

Conversation

ryoppippi
Copy link
Contributor

Related to #1128
I fixed the import path of randexp

Copy link

socket-security bot commented Jun 27, 2024

👍 Dependency issues cleared. Learn more about Socket for GitHub ↗︎

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

View full report↗︎

@ryoppippi ryoppippi marked this pull request as draft June 27, 2024 08:53
@ryoppippi ryoppippi force-pushed the feature/improve-tree-shaking-build branch 3 times, most recently from b162db6 to aec9902 Compare June 27, 2024 09:03
@ryoppippi ryoppippi marked this pull request as ready for review June 27, 2024 09:12
@ryoppippi
Copy link
Contributor Author

After merging this pr #1134 , you can easily check the build result

@ryoppippi ryoppippi force-pushed the feature/improve-tree-shaking-build branch from aec9902 to d4acfec Compare June 27, 2024 17:36
@ryoppippi ryoppippi changed the title feat(rollup.config): change entryFileNames format feat(rollup.config): improve tree-shaking using preserveModules: true Jun 27, 2024
@ryoppippi
Copy link
Contributor Author

@samchon
Any update on this?

@samchon
Copy link
Owner

samchon commented Jul 1, 2024

Will check this at Wednesday.

Copy link
Owner

@samchon samchon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checked your PR, but it still makes invalid import path.

import RandExp from '../../_external/.pnpm/randexp@0.5.3/node_modules/randexp/lib/randexp.mjs';

const ALPHABETS = "abcdefghijklmnopqrstuvwxyz";
/* -----------------------------------------------------------
  REGULAR
----------------------------------------------------------- */
const boolean = () => Math.random() < 0.5;
const integer = (min, max) => {
    min ??= 0;
    max ??= 100;
    return Math.floor(Math.random() * (max - min + 1)) + min;
};
const bigint = (min, max) => BigInt(integer(Number(min ?? BigInt(0)), Number(max ?? BigInt(100))));
const number = (min, max) => {
    min ??= 0;
    max ??= 100;
    return Math.random() * (max - min) + min;
};
const string = (length) => new Array(length ?? integer(5, 10))
    .fill(0)
    .map(() => ALPHABETS[integer(0, ALPHABETS.length - 1)])
    .join("");
const array = (closure, count) => new Array(count ?? length()).fill(0).map((_e, index) => closure(index));
const pick = (array) => array[integer(0, array.length - 1)];
const length = () => integer(0, 3);
const pattern = (regex) => {
    const r = new RandExp(regex);
    for (let i = 0; i < 10; ++i) {
        const str = r.gen();
        if (regex.test(str))
            return str;
    }
    return r.gen();
};
/* -----------------------------------------------------------
  SECIAL FORMATS
----------------------------------------------------------- */
// SPECIAL CHARACTERS
const byte = () => "vt7ekz4lIoNTTS9sDQYdWKharxIFAR54+z/umIxSgUM=";
const password = () => string(integer(4, 16));
const regex = () => "/^(?:(?:25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)\\.){3}(?:25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)$/";
const uuid = () => "xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx".replace(/[xy]/g, (c) => {
    const r = (Math.random() * 16) | 0;
    const v = c === "x" ? r : (r & 0x3) | 0x8;
    return v.toString(16);
});
// ADDRESSES
const email = () => `${string(10)}@${string(10)}.${string(3)}`;
const hostname = () => `${string(10)}.${string(3)}`;
const idnEmail = () => email();
const idnHostname = () => hostname();
const iri = () => url();
const iriReference = () => url();
const ipv4 = () => array(() => integer(0, 255), 4).join(".");
const ipv6 = () => array(() => integer(0, 65535).toString(16), 8).join(":");
const uri = () => url();
const uriReference = () => url();
const uriTemplate = () => url();
const url = () => `https://${string(10)}.${string(3)}`;
// TIMESTAMPS
const datetime = (min, max) => new Date(number(min ?? Date.now() - 30 * DAY, max ?? Date.now() + 7 * DAY)).toISOString();
const date = (min, max) => new Date(number(min ?? 0, max ?? Date.now() * 2))
    .toISOString()
    .substring(0, 10);
const time = () => new Date(number(0, DAY)).toISOString().substring(11);
const duration = () => {
    const period = durate([
        ["Y", integer(0, 100)],
        ["M", integer(0, 12)],
        ["D", integer(0, 31)],
    ]);
    const time = durate([
        ["H", integer(0, 24)],
        ["M", integer(0, 60)],
        ["S", integer(0, 60)],
    ]);
    if (period.length + time.length === 0)
        return "PT0S";
    return `P${period}${time.length ? "T" : ""}${time}`;
};
// POINTERS
const jsonPointer = () => `/components/schemas/${string(10)}`;
const relativeJsonPointer = () => `${integer(0, 10)}#`;
const DAY = 86400000;
const durate = (elements) => elements
    .filter(([_unit, value]) => value !== 0)
    .map(([unit, value]) => `${value}${unit}`)
    .join("");

export { array, bigint, boolean, byte, date, datetime, duration, email, hostname, idnEmail, idnHostname, integer, ipv4, ipv6, iri, iriReference, jsonPointer, length, number, password, pattern, pick, regex, relativeJsonPointer, string, time, uri, uriReference, uriTemplate, url, uuid };
//# sourceMappingURL=RandomGenerator.mjs.map

@ryoppippi
Copy link
Contributor Author

Oh, really? I'll check it

@samchon samchon added enhancement New feature or request good first issue Good for newcomers invalid This doesn't seem right labels Jul 2, 2024
@ryoppippi ryoppippi force-pushed the feature/improve-tree-shaking-build branch from d4acfec to aeda795 Compare July 2, 2024 11:08
@ryoppippi
Copy link
Contributor Author

I'll check after this PR solves #1142

ryoppippi added 5 commits July 2, 2024 12:31
The entryFileNames property in the rollup.config.js file has been
modified. Now, it checks if the chunk name includes 'node_modules'. If
it does, it replaces 'node_modules' with 'external' and appends '.js'.
Otherwise, it returns the original name with '.js' appended.
The output directory in rollup.config.js has been changed from a
variable to a relative path. This makes the configuration more
readable and less prone to errors due to variable mismanagement.
The 'preserveModules' and 'preserveModulesRoot' options have been moved
down in the output configuration. This change does not affect the
functionality but improves the readability of the code.
The terser plugin was removed from the rollup.config.js file. This
change simplifies the build process as the terser plugin was not
necessary for the current project setup.
The TypeScript configuration in rollup.config.js has been updated. The
module and target settings have been changed from ES2020 to ESNext. This
allows the use of the latest ECMAScript features in the codebase.
@ryoppippi ryoppippi force-pushed the feature/improve-tree-shaking-build branch from aeda795 to 54779c5 Compare July 2, 2024 11:31
Copy link

pkg-pr-new bot commented Jul 2, 2024

Review PR in StackBlitz Codeflow Run & review this pull request in StackBlitz Codeflow.

commit: c0a9c0d

typia

npm i https://pkg.pr.new/typia@1133


templates

@ryoppippi
Copy link
Contributor Author

ryoppippi commented Jul 2, 2024

@samchon
so, you can install this PR from this link #1133 (comment) !

I installed this and checked it on my local environment.
As you can see, the RandExp path is correct

import RandExp from '../../_external/randexp/lib/randexp.mjs';

idk why it is not working in your enviroment.
Maybe because of pnpm??

@ryoppippi ryoppippi requested a review from samchon July 2, 2024 11:36
@samchon
Copy link
Owner

samchon commented Jul 2, 2024

Understood, it was not a bug in npm, but only the problem of pnpm.

By the way, previous typia had generated only one index.mjs file, and your new PR contribution makes multiple mjs files for every TS files.

As far as I know, number of files does not affetct to esm tree shaking. Therefore, I am not sure if the actual bundling size is correct as the number of files increases.

@ryoppippi
Copy link
Contributor Author

ryoppippi commented Jul 2, 2024

Ohhhh I got it
so, when using npm or bun, the package is installed in flat structure,
so the external path is like this

node_modules/randexp/lib/randexp
node_modules/ret/lib/index
node_modules/drange/lib/index
node_modules/ret/lib/util
node_modules/ret/lib/types
node_modules/ret/lib/sets
node_modules/ret/lib/positions

However, pnpm is different structure, and the path looks like this

node_modules/.pnpm/randexp@0.5.3/node_modules/randexp/lib/randexp
node_modules/.pnpm/ret@0.2.2/node_modules/ret/lib/index
node_modules/.pnpm/drange@1.1.1/node_modules/drange/lib/index
node_modules/.pnpm/ret@0.2.2/node_modules/ret/lib/util
node_modules/.pnpm/ret@0.2.2/node_modules/ret/lib/types
node_modules/.pnpm/ret@0.2.2/node_modules/ret/lib/sets
node_modules/.pnpm/ret@0.2.2/node_modules/ret/lib/positions

Anyway I'll add support for pnpm

@samchon
Copy link
Owner

samchon commented Jul 2, 2024

Then no way to solve the problem in the pnpm through rollup configuration?

@ryoppippi
Copy link
Contributor Author

@samchon
I'm implementing configuration.

Also, I investigated that this improves tree-shaking.
I'll show you an example after fix this

@ryoppippi ryoppippi force-pushed the feature/improve-tree-shaking-build branch from 95a2bff to dacbac7 Compare July 2, 2024 12:09
@ryoppippi
Copy link
Contributor Author

ryoppippi commented Jul 2, 2024

@samchon
I fixed the build config.

Also, I created an experiment enviroment.
https://stackblitz.com/edit/vitejs-vite-twjepx?file=package.json
https://stackblitz.com/edit/vitejs-vite-d7cpuz?file=package.json

Why do these differences occur?
Because re-exported namespaces cannot be tree-shaked.

evanw/esbuild#1420

If you bundle the js files into single file, the named exports are converted into objects, so that bundlers cannnot identify those methods in the objects are from real object or named export from other files.
However, when we split the js files, bundlers can identify these differences

@ryoppippi ryoppippi force-pushed the feature/improve-tree-shaking-build branch 4 times, most recently from 679f334 to a2590a3 Compare July 2, 2024 14:32
The chunk name generation logic in rollup.config.js has been improved.
Instead of simply replacing 'node_modules' with '_external', the new
logic splits the path, replaces '/' with '_', and uses the last part of
the path as the file name. This provides a more unique and descriptive
chunk name. A console.table is also added to log the before and after
names for easier debugging.
@ryoppippi ryoppippi force-pushed the feature/improve-tree-shaking-build branch from a2590a3 to c0a9c0d Compare July 2, 2024 14:33
@samchon
Copy link
Owner

samchon commented Jul 3, 2024

Great. I'll publish it to v3.5.0 at tomorrow.

Waiting for @samchon/openapi dependency update.

Also, your github sponsorship dashboard configuration is not yet?

@samchon samchon merged commit 154e155 into samchon:master Jul 3, 2024
7 checks passed
@ryoppippi
Copy link
Contributor Author

Not yet Please wait

@samchon
Copy link
Owner

samchon commented Jul 3, 2024

@ryoppippi Now, I've completed @samchon/openapi upgrade, so that ready to publish typia@6.4.0.

May I publish v6.4 now, or do you have something more to do about the tree shake bundling?

@ryoppippi
Copy link
Contributor Author

For now it is fine!
Thanks!!
Please publish it!

@samchon
Copy link
Owner

samchon commented Jul 3, 2024

It still makes invalid path in pnpm.

import RandExp from '../../_external/node_modules_.pnpm_randexp@0.5.3_node_modules_randexp_lib/randexp.mjs';

const ALPHABETS = "abcdefghijklmnopqrstuvwxyz";
/* -----------------------------------------------------------
  REGULAR
----------------------------------------------------------- */
const boolean = () => Math.random() < 0.5;
const integer = (min, max) => {
    min ??= 0;
    max ??= 100;
    return Math.floor(Math.random() * (max - min + 1)) + min;
};
const bigint = (min, max) => BigInt(integer(Number(min ?? BigInt(0)), Number(max ?? BigInt(100))));
const number = (min, max) => {
    min ??= 0;
    max ??= 100;
    return Math.random() * (max - min) + min;
};
const string = (length) => new Array(length ?? integer(5, 10))
    .fill(0)
    .map(() => ALPHABETS[integer(0, ALPHABETS.length - 1)])
    .join("");
const array = (closure, count, unique) => {
    count ??= length();
    unique ??= false;
    if (unique === false)
        return new Array(count ?? length())
            .fill(0)
            .map((_e, index) => closure(index));
    else {
        const set = new Set();
        while (set.size < count)
            set.add(closure(set.size));
        return Array.from(set);
    }
};
const pick = (array) => array[integer(0, array.length - 1)];
const length = () => integer(0, 3);
const pattern = (regex) => {
    const r = new RandExp(regex);
    for (let i = 0; i < 10; ++i) {
        const str = r.gen();
        if (regex.test(str))
            return str;
    }
    return r.gen();
};
/* -----------------------------------------------------------
  SECIAL FORMATS
----------------------------------------------------------- */
// SPECIAL CHARACTERS
const byte = () => "vt7ekz4lIoNTTS9sDQYdWKharxIFAR54+z/umIxSgUM=";
const password = () => string(integer(4, 16));
const regex = () => "/^(?:(?:25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)\\.){3}(?:25[0-5]|2[0-4]\\d|1\\d\\d|[1-9]?\\d)$/";
const uuid = () => "xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx".replace(/[xy]/g, (c) => {
    const r = (Math.random() * 16) | 0;
    const v = c === "x" ? r : (r & 0x3) | 0x8;
    return v.toString(16);
});
// ADDRESSES
const email = () => `${string(10)}@${string(10)}.${string(3)}`;
const hostname = () => `${string(10)}.${string(3)}`;
const idnEmail = () => email();
const idnHostname = () => hostname();
const iri = () => url();
const iriReference = () => url();
const ipv4 = () => array(() => integer(0, 255), 4).join(".");
const ipv6 = () => array(() => integer(0, 65535).toString(16), 8).join(":");
const uri = () => url();
const uriReference = () => url();
const uriTemplate = () => url();
const url = () => `https://${string(10)}.${string(3)}`;
// TIMESTAMPS
const datetime = (min, max) => new Date(number(min ?? Date.now() - 30 * DAY, max ?? Date.now() + 7 * DAY)).toISOString();
const date = (min, max) => new Date(number(min ?? 0, max ?? Date.now() * 2))
    .toISOString()
    .substring(0, 10);
const time = () => new Date(number(0, DAY)).toISOString().substring(11);
const duration = () => {
    const period = durate([
        ["Y", integer(0, 100)],
        ["M", integer(0, 12)],
        ["D", integer(0, 31)],
    ]);
    const time = durate([
        ["H", integer(0, 24)],
        ["M", integer(0, 60)],
        ["S", integer(0, 60)],
    ]);
    if (period.length + time.length === 0)
        return "PT0S";
    return `P${period}${time.length ? "T" : ""}${time}`;
};
// POINTERS
const jsonPointer = () => `/components/schemas/${string(10)}`;
const relativeJsonPointer = () => `${integer(0, 10)}#`;
const DAY = 86400000;
const durate = (elements) => elements
    .filter(([_unit, value]) => value !== 0)
    .map(([unit, value]) => `${value}${unit}`)
    .join("");

export { array, bigint, boolean, byte, date, datetime, duration, email, hostname, idnEmail, idnHostname, integer, ipv4, ipv6, iri, iriReference, jsonPointer, length, number, password, pattern, pick, regex, relativeJsonPointer, string, time, uri, uriReference, uriTemplate, url, uuid };
//# sourceMappingURL=RandomGenerator.mjs.map

@samchon
Copy link
Owner

samchon commented Jul 3, 2024

Oops, I'm sorry, it was not a bug.

@ryoppippi ryoppippi deleted the feature/improve-tree-shaking-build branch July 3, 2024 16:16
@ryoppippi
Copy link
Contributor Author

Yeah I managed to do it

@ryoppippi
Copy link
Contributor Author

ryoppippi commented Jul 3, 2024

@samchon
BTW, I enabled GH sponsorship rn !!!

https://github.com/ryoppippi

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers invalid This doesn't seem right
Projects
No open projects
Status: Done
Development

Successfully merging this pull request may close these issues.

2 participants