Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add e2e tests to CI #3344

Merged
merged 3 commits into from
Jun 14, 2023
Merged

Add e2e tests to CI #3344

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e49a53f
Add e2e tests to CI
penalosa Jun 12, 2023
e8e864c
PR comments
penalosa Jun 13, 2023
e5af057
Vite config isn't being read, so use CLI args
penalosa Jun 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
87 changes: 87 additions & 0 deletions .github/workflows/e2e.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
name: E2E tests

on:
push:
branches:
- main
- changeset-release/main
pull_request:
types: [synchronize, opened, reopened, labeled, unlabeled]
repository_dispatch:

jobs:
e2e-test:
if: github.repository_owner == 'cloudflare' && (github.event_name != 'pull_request' || (github.event_name == 'pull_request' && contains(github.event.*.labels.*.name, 'e2e' )))
name: "E2E Test"
strategy:
matrix:
os: [
# macos-11,
# macos-12,
macos-13,
windows-2019,
windows-2022,
# ubuntu-20.04,
ubuntu-22.04,
]
node: ["16", "18"]
runs-on: ${{ matrix.os }}
steps:
- name: Checkout Repo
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Use Node.js ${{ matrix.node }}
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node }}
cache: "npm"

- name: Install workerd Dependencies
if: ${{ runner.os == 'Linux' }}
run: |
export DEBIAN_FRONTEND=noninteractive
sudo apt-get update
sudo apt-get install -y libc++1

# Attempt to cache all the node_modules directories based on the OS and package lock.
- name: Cache node_modules
id: npm-cache
uses: actions/cache@v3
env:
cache-name: cache-node-modules
with:
key: ${{ runner.os }}-${{ matrix.node }}-${{ env.cache-name }}-${{ hashFiles('**/package-lock.json') }}
path: "**/node_modules"

# If the cache missed then install using `npm ci` to follow package lock exactly
- if: ${{ steps.npm-cache.outputs.cache-hit != 'true'}}
name: Install NPM Dependencies
run: npm ci

- name: Run builds
run: npm run -w wrangler build
env:
NODE_ENV: "production"

- name: Build Wrangler package for npm
run: cd packages/wrangler && npm pack
env:
NODE_ENV: "production"

- name: Move Wrangler package to tmp directory
shell: bash
id: "move-wrangler"
run: |
cp packages/wrangler/wrangler-*.tgz $HOME
echo "dir=$(ls $HOME/wrangler-*.tgz)" >> $GITHUB_OUTPUT;
env:
NODE_ENV: "production"

- name: Run tests
run: npm run -w wrangler test:e2e
env:
CLOUDFLARE_API_TOKEN: ${{ secrets.TEST_CLOUDFLARE_API_TOKEN }}
CLOUDFLARE_ACCOUNT_ID: ${{ secrets.TEST_CLOUDFLARE_ACCOUNT_ID }}
WRANGLER: npm install ${{ steps.move-wrangler.outputs.dir}} && npx --prefer-offline wrangler
NODE_OPTIONS: "--max_old_space_size=8192"
16 changes: 8 additions & 8 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

33 changes: 16 additions & 17 deletions packages/wrangler/e2e/deploy.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import crypto from "node:crypto";
import path from "node:path";
import { setTimeout } from "node:timers/promises";
import { fetch } from "undici";
import { describe, expect, it } from "vitest";
import { retry } from "./helpers/retry";
import { RUN, runIn } from "./helpers/run";
import { dedent, makeRoot, seed } from "./helpers/setup";

Expand Down Expand Up @@ -33,15 +33,11 @@ describe("deploy", async () => {
✨ Created smoke-test-worker/src/index.ts
Your project will use Vitest to run your tests.
✨ Created smoke-test-worker/src/index.test.ts

added (N) packages, and audited (N) packages in (TIMINGS)

(N) packages are looking for funding
run \`npm fund\` for details

found 0 vulnerabilities
✨ Installed @cloudflare/workers-types, typescript, and vitest into devDependencies

To start developing your Worker, run \`cd smoke-test-worker && npm start\`
To start testing your Worker, run \`npm test\`
To publish your Worker to the Internet, run \`npm run deploy\`"
Expand All @@ -65,10 +61,11 @@ describe("deploy", async () => {
expect(stderr).toMatchInlineSnapshot('""');
workersDev = matchWorkersDev(rawStdout);

await setTimeout(5_000);
await expect(
fetch(`https://${workerName}.${workersDev}`).then((r) => r.text())
).resolves.toMatchInlineSnapshot('"Hello World!"');
await retry(() =>
expect(
fetch(`https://${workerName}.${workersDev}`).then((r) => r.text())
).resolves.toMatchInlineSnapshot('"Hello World!"')
);
});
it("modify & deploy worker", async () => {
await seed(workerPath, {
Expand Down Expand Up @@ -96,10 +93,11 @@ describe("deploy", async () => {
expect(stderr).toMatchInlineSnapshot('""');
workersDev = matchWorkersDev(rawStdout);

await setTimeout(10_000);
await expect(
fetch(`https://${workerName}.${workersDev}`).then((r) => r.text())
).resolves.toMatchInlineSnapshot('"Updated Worker!"');
await retry(() =>
expect(
fetch(`https://${workerName}.${workersDev}`).then((r) => r.text())
).resolves.toMatchInlineSnapshot('"Updated Worker!"')
);
});

it("delete worker", async () => {
Expand All @@ -114,9 +112,10 @@ describe("deploy", async () => {
Successfully deleted smoke-test-worker"
`);
expect(stderr).toMatchInlineSnapshot('""');
await setTimeout(10_000);
await expect(
fetch(`https://${workerName}.${workersDev}`).then((r) => r.status)
).resolves.toBe(404);
await retry(() =>
expect(
fetch(`https://${workerName}.${workersDev}`).then((r) => r.status)
).resolves.toBe(404)
);
});
});
44 changes: 16 additions & 28 deletions packages/wrangler/e2e/deployments.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import crypto from "node:crypto";
import path from "node:path";
import { setTimeout } from "node:timers/promises";
import { fetch } from "undici";
import { describe, expect, it } from "vitest";
import { retry } from "./helpers/retry";
import { RUN, runIn } from "./helpers/run";
import { dedent, makeRoot, seed } from "./helpers/setup";

Expand Down Expand Up @@ -46,15 +46,11 @@ describe("deployments", async () => {
✨ Created smoke-test-worker/src/index.ts
Your project will use Vitest to run your tests.
✨ Created smoke-test-worker/src/index.test.ts

added (N) packages, and audited (N) packages in (TIMINGS)

(N) packages are looking for funding
run \`npm fund\` for details

found 0 vulnerabilities
✨ Installed @cloudflare/workers-types, typescript, and vitest into devDependencies

To start developing your Worker, run \`cd smoke-test-worker && npm start\`
To start testing your Worker, run \`npm test\`
To publish your Worker to the Internet, run \`npm run deploy\`"
Expand All @@ -78,10 +74,11 @@ describe("deployments", async () => {
expect(stderr).toMatchInlineSnapshot('""');
workersDev = matchWorkersDev(rawStdout);

await setTimeout(2_000);
await expect(
fetch(`https://${workerName}.${workersDev}`).then((r) => r.text())
).resolves.toMatchInlineSnapshot('"Hello World!"');
await retry(() =>
expect(
fetch(`https://${workerName}.${workersDev}`).then((r) => r.text())
).resolves.toMatchInlineSnapshot('"Hello World!"')
);
});

it("list 1 deployment", async () => {
Expand All @@ -93,8 +90,6 @@ describe("deployments", async () => {
`;
expect(stdout).toMatchInlineSnapshot(`
"🚧\`wrangler deployments\` is a beta command. Please report any issues to https://github.com/cloudflare/workers-sdk/issues/new/choose


Deployment ID: 00000000-0000-0000-0000-000000000000
Created on: TIMESTAMP
Author: person@example.com
Expand Down Expand Up @@ -130,10 +125,11 @@ describe("deployments", async () => {
expect(stderr).toMatchInlineSnapshot('""');
workersDev = matchWorkersDev(rawStdout);

await setTimeout(10_000);
await expect(
fetch(`https://${workerName}.${workersDev}`).then((r) => r.text())
).resolves.toMatchInlineSnapshot('"Updated Worker!"');
await retry(() =>
expect(
fetch(`https://${workerName}.${workersDev}`).then((r) => r.text())
).resolves.toMatchInlineSnapshot('"Updated Worker!"')
);
});

it("list 2 deployments", async () => {
Expand All @@ -145,13 +141,10 @@ describe("deployments", async () => {
`;
expect(stdout).toMatchInlineSnapshot(`
"🚧\`wrangler deployments\` is a beta command. Please report any issues to https://github.com/cloudflare/workers-sdk/issues/new/choose


Deployment ID: 00000000-0000-0000-0000-000000000000
Created on: TIMESTAMP
Author: person@example.com
Source: Upload from Wrangler 🤠

Deployment ID: 00000000-0000-0000-0000-000000000000
Created on: TIMESTAMP
Author: person@example.com
Expand All @@ -170,8 +163,6 @@ describe("deployments", async () => {
`;
expect(stdout).toMatchInlineSnapshot(`
"🚧\`wrangler rollback\` is a beta command. Please report any issues to https://github.com/cloudflare/workers-sdk/issues/new/choose


Successfully rolled back to Deployment ID: 00000000-0000-0000-0000-000000000000
Current Deployment ID: 00000000-0000-0000-0000-000000000000"
`);
Expand All @@ -187,18 +178,14 @@ describe("deployments", async () => {
`;
expect(stdout).toMatchInlineSnapshot(`
"🚧\`wrangler deployments\` is a beta command. Please report any issues to https://github.com/cloudflare/workers-sdk/issues/new/choose


Deployment ID: 00000000-0000-0000-0000-000000000000
Created on: TIMESTAMP
Author: person@example.com
Source: Upload from Wrangler 🤠

Deployment ID: 00000000-0000-0000-0000-000000000000
Created on: TIMESTAMP
Author: person@example.com
Source: Upload from Wrangler 🤠

Deployment ID: 00000000-0000-0000-0000-000000000000
Created on: TIMESTAMP
Author: person@example.com
Expand All @@ -222,9 +209,10 @@ describe("deployments", async () => {
Successfully deleted smoke-test-worker"
`);
expect(stderr).toMatchInlineSnapshot('""');
await setTimeout(10_000);
await expect(
fetch(`https://${workerName}.${workersDev}`).then((r) => r.status)
).resolves.toBe(404);
await retry(() =>
expect(
fetch(`https://${workerName}.${workersDev}`).then((r) => r.status)
).resolves.toBe(404)
);
});
});
Loading