Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(create-waku): --example option #581

Merged
merged 16 commits into from
Mar 12, 2024
Merged
4 changes: 3 additions & 1 deletion packages/create-waku/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,16 +25,18 @@
"start": "node ./dist/index.js",
"dev": "ncc build ./src/index.ts -w -o ./dist/",
"compile": "rm -rf template dist *.tsbuildinfo && pnpm run template && pnpm run build",
"template": "cp -r ../../examples template/ && rm -rf template/*/dist && rm -rf template/*/node_modules && mv template/01_template/.gitignore template/01_template/gitignore",
"template": "mkdir template && cp -r ../../examples/01_template template/01_template && rm -rf template/*/dist && rm -rf template/*/node_modules && mv template/01_template/.gitignore template/01_template/gitignore",
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can just cp -r ../../examples/01_template template. You can consider it in the follow-up PRs.

Copy link
Contributor Author

@ojj1123 ojj1123 Mar 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i've tried to do that. but i've got it like below:

/template
  /src
  package.json
  ...

but i want to do this below:

/template
  /01_template

because i consider that it will be added the extra templates 👍

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, i thought the former is fine (single template). We don't know the future, but for now we have only one template.

"build": "ncc build ./src/index.ts -o ./dist/ --minify --no-cache --no-source-map-register"
},
"devDependencies": {
"@types/fs-extra": "^11.0.4",
"@types/prompts": "^2.4.9",
"@types/tar": "^6.1.11",
"@vercel/ncc": "^0.38.1",
"fs-extra": "^11.2.0",
"kolorist": "^1.8.0",
"prompts": "^2.4.2",
"tar": "^6.2.0",
"update-check": "^1.5.4"
}
}
130 changes: 130 additions & 0 deletions packages/create-waku/src/helpers/example-option.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,130 @@
import * as tar from 'tar';
import { Readable } from 'node:stream';
import { pipeline } from 'node:stream/promises';
import type { ReadableStream } from 'stream/web';

export type RepoInfo = {
username: string | undefined;
name: string | undefined;
branch: string | undefined;
filePath: string | undefined;
};

export async function isUrlOk(url: string): Promise<boolean> {
try {
const res = await fetch(url, { method: 'HEAD' });
return res.status === 200;
} catch {
return false;
}
}

/**
* this is a part of the response type for github "Get a repository" API
* @see {@link https://docs.github.com/en/rest/repos/repos?apiVersion=2022-11-28#get-a-repository|GitHub REST API}
*/
interface GetRepoInfo {
/** A default branch of the repository */
default_branch: string;
}
export async function getRepoInfo(url: URL): Promise<RepoInfo | undefined> {
const [, username, name, t, _branch, ...file] = url.pathname.split('/');
const filePath = file.join('/');

if (
// Support repos whose entire purpose is to be a waku example, e.g.
// https://github.com/:username/:my-cool-waku-example-repo-name.
t === undefined ||
// Support GitHub URL that ends with a trailing slash, e.g.
// https://github.com/:username/:my-cool-waku-example-repo-name/
// In this case "t" will be an empty string while the next part "_branch" will be undefined
(t === '' && _branch === undefined)
) {
try {
const infoResponse = await fetch(
`https://api.github.com/repos/${username}/${name}`,
);
if (infoResponse.status !== 200) {
return;
}

const info = (await infoResponse.json()) as GetRepoInfo;
return { username, name, branch: info['default_branch'], filePath };
} catch {
return;
}
}

if (username && name && _branch && t === 'tree') {
return { username, name, branch: _branch, filePath };
}
}

export function hasRepo({
username,
name,
branch,
filePath,
}: RepoInfo): Promise<boolean> {
const contentsUrl = `https://api.github.com/repos/${username}/${name}/contents`;
const packagePath = `${filePath ? `/${filePath}` : ''}/package.json`;

return isUrlOk(contentsUrl + packagePath + `?ref=${branch}`);
}

export function existsInRepo(nameOrUrl: string): Promise<boolean> {
try {
const url = new URL(nameOrUrl);
return isUrlOk(url.href);
} catch {
return isUrlOk(
`https://api.github.com/repos/dai-shi/waku/contents/examples/${encodeURIComponent(
nameOrUrl,
)}`,
);
}
}

async function downloadTarStream(url: string) {
const res = await fetch(url);

if (!res.body) {
throw new Error(`Failed to download: ${url}`);
}

return Readable.fromWeb(res.body as ReadableStream);
}

export async function downloadAndExtractRepo(
root: string,
{ username, name, branch, filePath }: RepoInfo,
) {
await pipeline(
await downloadTarStream(
`https://codeload.github.com/${username}/${name}/tar.gz/${branch}`,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this downloading the whole waku repo for users?

Copy link
Contributor Author

@ojj1123 ojj1123 Mar 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the only way to get examples of waku is to download the whole waku repository.
Do you have any concerns or suggestions?
if you do, i would like to consider it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, it might be too much size to download for the user. If there's no other way then let's go with this. but it's better to find a way.

),
tar.x({
cwd: root,
strip: filePath ? filePath.split('/').length + 1 : 1,
filter: (p) =>
p.startsWith(
`${name}-${branch?.replace(/\//g, '-')}${
filePath ? `/${filePath}/` : '/'
}`,
),
}),
);
}

export async function downloadAndExtractExample(root: string, name: string) {
await pipeline(
await downloadTarStream(
'https://codeload.github.com/dai-shi/waku/tar.gz/main',
),
tar.x({
cwd: root,
strip: 2 + name.split('/').length,
filter: (p) => p.includes(`waku-main/examples/${name}/`),
}),
);
}
Loading
Loading