Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: async workflows #306

Merged
merged 30 commits into from
Mar 9, 2023
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
2e66902
promise hook and tests
thantos Feb 10, 2023
f22dd6c
feat: async workflow executor
thantos Mar 3, 2023
057b074
fix...
thantos Mar 3, 2023
8484f38
tests pass
thantos Mar 3, 2023
4f35d15
feedback
thantos Mar 5, 2023
a00fd19
simplify test env, fix bugs, remove async transformer
thantos Mar 5, 2023
ce4a7d1
more test refactoring
thantos Mar 5, 2023
8da98d8
fix replay bug
thantos Mar 5, 2023
c4ea0ea
signal handler access to workflow hook
thantos Mar 5, 2023
95116f2
move commands to core runtime
thantos Mar 5, 2023
aec39b7
simplify await timer calls
thantos Mar 6, 2023
44bcb65
refactor executor
thantos Mar 6, 2023
a0f7b94
old spelling error
thantos Mar 6, 2023
1ac48bd
test continue
thantos Mar 6, 2023
94ef5f7
fix some tests
thantos Mar 6, 2023
724e7ac
fix tests
thantos Mar 6, 2023
31fd5fa
cleanup
thantos Mar 7, 2023
fe4538d
date hook with async local store
thantos Mar 7, 2023
58f34bc
docs
thantos Mar 7, 2023
3141700
support events after workflow success
thantos Mar 7, 2023
8755120
clean up
thantos Mar 7, 2023
169e295
fix date bug
thantos Mar 7, 2023
cf3069d
revert als
thantos Mar 8, 2023
7ef962f
Merge branch 'sussman/async_workflows' of https://github.com/function…
thantos Mar 8, 2023
9f81bf7
Merge branch 'main' into sussman/async_workflows
thantos Mar 8, 2023
1bce98b
workflow failures
thantos Mar 8, 2023
b262d57
reenable the chaos ext
thantos Mar 8, 2023
fb25b0f
speed and clean up
thantos Mar 8, 2023
944dd90
testing then, catch, finally
thantos Mar 9, 2023
bb6a982
feedback
thantos Mar 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions apps/tests/aws-runtime/scripts/test-cli
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,22 @@ npx eventual list services

npx eventual list workflows

npx eventual list executions
npx eventual start workflow sleepy

npx eventual start workflow sleepy -f

npx eventual list executions

npx eventual list executions --workflow parallel --json > .eventual/out.json
npx eventual list executions --workflow sleepy --json > .eventual/out.json
Comment on lines +11 to +17
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test was not using an execution that was started by the same script, which was confusing and could be faulty.


execution_id=$(node -p 'JSON.parse(require("fs").readFileSync(".eventual/out.json").toString("utf8")).executions[0].id')

npx eventual get history -e ${execution_id}

npx eventual get logs --all

npx eventual get logs --workflow parallel
npx eventual get logs --workflow sleepy

npx eventual get logs --execution ${execution_id}

npx eventual start workflow sleepy

npx eventual start workflow sleepy -f

npx eventual replay execution ${execution_id} --entry 'test/test-service.ts'
18 changes: 11 additions & 7 deletions packages/@eventual/cli/src/commands/replay.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,19 @@ import {
isSucceededExecution,
} from "@eventual/core";
import {
isFailed,
isResolved,
normalizeFailedResult,
parseWorkflowName,
processEvents,
progressWorkflow,
resultToString,
} from "@eventual/core-runtime";
import {
encodeExecutionId,
isFailed,
isResolved,
normalizeFailedResult,
Result,
resultToString,
ServiceType,
serviceTypeScopeSync,
serviceTypeScope,
workflows,
} from "@eventual/core/internal";
import path from "path";
Expand Down Expand Up @@ -62,7 +62,7 @@ export const replay = (yargs: Argv) =>
}
spinner.start("Running program");

serviceTypeScopeSync(ServiceType.OrchestratorWorker, () => {
await serviceTypeScope(ServiceType.OrchestratorWorker, async () => {
const processedEvents = processEvents(
events,
[],
Expand All @@ -74,7 +74,11 @@ export const replay = (yargs: Argv) =>
)
);

const res = progressWorkflow(execution, workflow, processedEvents);
const res = await progressWorkflow(
execution,
workflow,
processedEvents
);

assertExpectedResult(executionObj, res.result);

Expand Down
73 changes: 3 additions & 70 deletions packages/@eventual/compiler/src/ast-util.ts
Original file line number Diff line number Diff line change
@@ -1,17 +1,11 @@
import {
Argument,
ArrowFunctionExpression,
CallExpression,
ComputedPropName,
Expression,
FunctionExpression,
Span,
Node,
StringLiteral,
FunctionDeclaration,
Statement,
HasSpan,
Identifier,
ComputedPropName,
Node,
Span,
} from "@swc/core";

/**
Expand Down Expand Up @@ -101,67 +95,6 @@ function isId<Value extends string>(
);
}

/**
* A heuristic for identifying a {@link CallExpression} that is a call
* to the eventual.workflow utility:
*
* 1. must be a function call with exactly 2 arguments
* 2. first argument is a string literal
* 3. second argument is a FunctionExpression or ArrowFunctionExpression
* 4. callee is an identifier `"workflow"` or `<identifier>.workflow`
*/
export function isWorkflowCall(call: CallExpression): call is CallExpression & {
arguments: [
Argument & { expression: StringLiteral },
Argument & { expression: FunctionExpression | ArrowFunctionExpression }
];
} {
return (
isWorkflowCallee(call.callee) &&
call.arguments[0]?.expression.type === "StringLiteral" &&
((call.arguments.length === 2 &&
isNonGeneratorFunction(call.arguments[1]?.expression)) ||
(call.arguments.length === 3 &&
isNonGeneratorFunction(call.arguments[2]?.expression)))
);
}

export function isNonGeneratorFunction(
expr?: Expression
): expr is ArrowFunctionExpression | FunctionExpression {
return (
(expr?.type === "ArrowFunctionExpression" ||
expr?.type === "FunctionExpression") &&
!expr.generator
);
}

export function isActivityCallee(callee: CallExpression["callee"]) {
return isCallee("activity", callee);
}

export function isWorkflowCallee(callee: CallExpression["callee"]) {
return isCallee("workflow", callee);
}

export function isCallee(
type: "activity" | "workflow",
callee: CallExpression["callee"]
) {
return (
(callee.type === "Identifier" && callee.value === type) ||
(callee.type === "MemberExpression" &&
callee.property.type === "Identifier" &&
callee.property.value === type)
);
}

export function isAsyncFunctionDecl(
stmt: Statement
): stmt is FunctionDeclaration & { async: true } {
return stmt.type === "FunctionDeclaration" && stmt.async;
}

export function hasSpan(expr: Node): expr is Node & HasSpan {
return "span" in expr;
}
Expand Down
30 changes: 0 additions & 30 deletions packages/@eventual/compiler/src/esbuild-plugin.ts

This file was deleted.

3 changes: 0 additions & 3 deletions packages/@eventual/compiler/src/eventual-bundle.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@ import { aliasPath } from "esbuild-plugin-alias-path";
import fs from "fs/promises";
import path from "path";
import { prepareOutDir } from "./build.js";
import { eventualESPlugin } from "./esbuild-plugin.js";

export async function bundleSources(
outDir: string,
Expand Down Expand Up @@ -65,7 +64,6 @@ export async function build({
injectedServiceSpec,
name,
entry,
eventualTransform = false,
sourcemap,
serviceType,
external,
Expand Down Expand Up @@ -102,7 +100,6 @@ export async function build({
}),
]
: []),
...(eventualTransform ? [eventualESPlugin] : []),
],
conditions: ["module", "import", "require"],
// external: ["@aws-sdk"],
Expand Down
2 changes: 0 additions & 2 deletions packages/@eventual/compiler/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
export * from "./eventual-bundle.js";
export * from "./eventual-infer.js";
export * from "./esbuild-plugin.js";
export * from "./build.js";
export * from "./workflow-visitor.js";
Loading