Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(idempotency): Add function wrapper and decorator #1262

Merged
merged 73 commits into from
Feb 6, 2023
Merged
Show file tree
Hide file tree
Changes from 72 commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
51104f6
feat: initial idempotency classes
vgphoenixcampos Sep 2, 2022
d54e22e
feat: refactor persistence layer classes into their own folder
vgphoenixcampos Sep 8, 2022
6ea753e
feat: rename idempotency config to differentiate from idempotency opt…
vgphoenixcampos Sep 8, 2022
c5f2144
feat: added type for a generic function
vgphoenixcampos Sep 8, 2022
ab254c6
feat: remove idempotency configuration for this FR
vgphoenixcampos Sep 8, 2022
8677376
feat: refactored type of function to accept any combo of parameters
vgphoenixcampos Sep 8, 2022
b955ac4
feat: adding PersistenceLayer
jeffrey-baker-vg Oct 6, 2022
c8776d2
feat: PersistenceLayer unit tests for saveInProgress
jeffrey-baker-vg Oct 14, 2022
f258d55
feat: added saveSuccess
jeffrey-baker-vg Oct 14, 2022
edc9b33
feat: added getRecord
jeffrey-baker-vg Oct 17, 2022
dfc72c0
feat: added delete record
jeffrey-baker-vg Oct 18, 2022
4d63218
feat: branch coverage and cleaning up imports
jeffrey-baker-vg Oct 18, 2022
d92efb1
feat: added more tests
jeffrey-baker-vg Oct 18, 2022
5258d10
feat: deleted unused methods
jeffrey-baker-vg Oct 18, 2022
5d82215
feat: added comments
jeffrey-baker-vg Oct 18, 2022
95004c2
feat: implement get command for dynamo persistence layer
KevenFuentes9 Sep 14, 2022
c5a9335
feat: implement get command for dynamo persistence layer
KevenFuentes9 Sep 14, 2022
3d1f52c
feat: allow for data attr to be passed and return in persistence laye…
KevenFuentes9 Sep 14, 2022
0a6e839
feat: added implementation for delete, update, put
vgphoenixcampos Sep 23, 2022
b9ae754
feat: create condition on put for not in progress status
vgphoenixcampos Sep 23, 2022
2cd4390
feat: use inprogress enum for status
vgphoenixcampos Sep 23, 2022
01bcc99
feat: added error when unable to get record for idempotency key
vgphoenixcampos Sep 23, 2022
c1a6c60
feat: added error for conditional write of an existing record
vgphoenixcampos Sep 23, 2022
1011002
feat: tests added for put record on dynamo persistence layer
vgphoenixcampos Sep 23, 2022
adcfab3
feat: implemented the idempotency record functions for status, expiry…
KevenFuentes9 Sep 23, 2022
9f0535e
test: check if the status is expired
KevenFuentes9 Sep 23, 2022
9bf7885
test: idempotency record is not expired and status maintained
KevenFuentes9 Sep 23, 2022
ffb2c22
feat: added tests for get record
vgphoenixcampos Sep 23, 2022
ec31dbb
feat: add aws-sdk-client-mock jest assertion library
vgphoenixcampos Sep 30, 2022
48dfdec
feat: add unit tests for update record and delete record
vgphoenixcampos Sep 30, 2022
f666dfa
feat: remove optional chaining from item made unnecessary with error …
vgphoenixcampos Sep 30, 2022
0945801
feat: remove unused block
vgphoenixcampos Sep 30, 2022
e161521
feat: refactored mock child class to be shared amongst dynamo persist…
vgphoenixcampos Sep 30, 2022
9efeae0
test: add path to get the response data from the data record
KevenFuentes9 Sep 30, 2022
0fea9f3
feat: added branch to handle conditional check failure
vgphoenixcampos Oct 7, 2022
b4165bd
feat: add configuration option to dynamo client creation to remove un…
vgphoenixcampos Oct 7, 2022
d1dad17
feat: change how time is measured to seconds
vgphoenixcampos Oct 26, 2022
b9afc2a
feat: change type of the response/result to a record
vgphoenixcampos Oct 26, 2022
0ec8301
feat:updated imports
jeffrey-baker-vg Oct 28, 2022
1366614
feat: added save in progress to handle already existing records in dy…
KevenFuentes9 Nov 1, 2022
38d7a67
feat: add log message for the already in progress error
KevenFuentes9 Nov 1, 2022
dd442e2
feat: change the anyfunction type definition to also include a sync f…
KevenFuentes9 Nov 1, 2022
8e97a61
refactor: create constructor object for dynamo persistence layer
vgphoenixcampos Nov 4, 2022
afb9523
fix: remove temp eslint disable
vgphoenixcampos Nov 4, 2022
8a031ca
fix: adjust verbiage on test blocks
vgphoenixcampos Nov 4, 2022
f476e21
style: put constructor parameters onto one line for readability
vgphoenixcampos Nov 4, 2022
a044fa6
fix: update dynamo persistence layer tests to use new construtor opt…
vgphoenixcampos Nov 4, 2022
cefae2d
Merge branch 'main' of github.com:jeffrey-baker-vg/aws-lambda-powerto…
vgphoenixcampos Nov 4, 2022
dd979a8
fix: remove unneeded eslint ignore from persistence layer
vgphoenixcampos Nov 4, 2022
b290d19
style: put parameters for dynamo client command object onto one line …
vgphoenixcampos Nov 4, 2022
f76c720
fix: move lib-dynamo dep under the correct package
vgphoenixcampos Nov 4, 2022
b4931bf
refactor: change idempotency record to use options object in contructor
vgphoenixcampos Nov 4, 2022
9b4787a
feat: add consistent read to dynamo persistence layer
vgphoenixcampos Nov 4, 2022
08598f9
fix: revert changes to layer-publisher package-lock
vgphoenixcampos Nov 4, 2022
b3fe0be
feat added the call to the function in the idempotency handler and ad…
KevenFuentes9 Dec 5, 2022
0af6a5e
feat: add logic to invoke function if it has not already been called;…
KevenFuentes9 Dec 9, 2022
68ce9a2
chore: move and enhance comment on question
KevenFuentes9 Dec 9, 2022
19e01eb
chore: update comments
KevenFuentes9 Dec 9, 2022
5afab50
feat: use new record formatting for idempotent function wrapper
KevenFuentes9 Jan 20, 2023
78f1a89
test: add test case for issues even getting to save the record in per…
KevenFuentes9 Jan 20, 2023
58e1bda
chore: refactoring test suite for idempotent wrapper
KevenFuentes9 Jan 20, 2023
8fd2436
chore: clean up comments
KevenFuentes9 Jan 25, 2023
2e43938
feat: added decorator for idempotency
KevenFuentes9 Jan 25, 2023
9e369f7
chore: get rid of extra private member on the idempotency handler class
KevenFuentes9 Jan 25, 2023
cdae838
chore: merging
KevenFuentes9 Jan 25, 2023
f5e3b6b
chore: refactor to use class for options
KevenFuentes9 Jan 25, 2023
927654f
chore: bring in old version of package-locks
KevenFuentes9 Jan 25, 2023
d0d0574
Merge pull request #5 from jeffrey-baker-vg/origin/idempotency-core
KevenFuentes9 Jan 26, 2023
1ec1c80
Merge remote-tracking branch 'upstream/main'
KevenFuentes9 Jan 27, 2023
e3eb775
chore: update paths for interface
KevenFuentes9 Jan 27, 2023
bd2388c
chore: remove env files
KevenFuentes9 Jan 27, 2023
eadb129
chore: rename file
KevenFuentes9 Jan 27, 2023
780a8ef
chore: renaming test names and function names for idempotency decorat…
KevenFuentes9 Feb 4, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 16 additions & 13 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -86,4 +86,4 @@
"dependencies": {
"hosted-git-info": "^6.1.1"
}
}
}
17 changes: 16 additions & 1 deletion packages/idempotency/src/Exceptions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,23 @@ class IdempotencyInvalidStatusError extends Error {

}

class IdempotencyInconsistentStateError extends Error {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: adding a meaningful and easy to understand error message for each of these Errors would improve the DX and help troubleshooting errors

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a message to this when the error is created since this error can happen in more than one type of scenario. That way we can tailor the message by the scenario

}

class IdempotencyAlreadyInProgressError extends Error {

}

class IdempotencyPersistenceLayerError extends Error {

}

export {
IdempotencyItemNotFoundError,
IdempotencyItemAlreadyExistsError,
IdempotencyInvalidStatusError
IdempotencyInvalidStatusError,
IdempotencyInconsistentStateError,
IdempotencyAlreadyInProgressError,
IdempotencyPersistenceLayerError
};
39 changes: 39 additions & 0 deletions packages/idempotency/src/IdempotencyHandler.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
/* eslint-disable @typescript-eslint/no-explicit-any */

import { AnyFunctionWithRecord, IdempotencyRecordStatus } from './types';
import { IdempotencyOptions } from './types/IdempotencyOptions';
import { IdempotencyRecord } from 'persistence';
import { IdempotencyInconsistentStateError, IdempotencyItemAlreadyExistsError, IdempotencyAlreadyInProgressError, IdempotencyPersistenceLayerError } from './Exceptions';

export class IdempotencyHandler<U> {

public constructor(private functiontoMakeIdempotent: AnyFunctionWithRecord<U>, private functionPayloadToBeHashed: unknown,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion for this constructor signature:
Rename functiontoMakeIdempotent to functionToMakeIdempotent

Question: why the explicit access modifiers in the parameters? (private)

private idempotencyOptions: IdempotencyOptions, private fullFunctionPayload: Record<string, any>) {}

public determineResultFromIdempotencyRecord(idempotencyRecord: IdempotencyRecord): Promise<U> | U{
if (idempotencyRecord.getStatus() === IdempotencyRecordStatus.EXPIRED) {
throw new IdempotencyInconsistentStateError();
} else if (idempotencyRecord.getStatus() === IdempotencyRecordStatus.INPROGRESS){
throw new IdempotencyAlreadyInProgressError(`There is already an execution in progress with idempotency key: ${idempotencyRecord.idempotencyKey}`);
} else {
// Currently recalling the method as this fulfills FR1. FR3 will address using the previously stored value https://github.com/awslabs/aws-lambda-powertools-typescript/issues/447
return this.functiontoMakeIdempotent(this.fullFunctionPayload);
}
}

public async process_idempotency(): Promise<U> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use camelCase for consistency?

try {
await this.idempotencyOptions.persistenceStore.saveInProgress(this.functionPayloadToBeHashed);
} catch (e) {
if (e instanceof IdempotencyItemAlreadyExistsError) {
const idempotencyRecord: IdempotencyRecord = await this.idempotencyOptions.persistenceStore.getRecord(this.functionPayloadToBeHashed);

return this.determineResultFromIdempotencyRecord(idempotencyRecord);
} else {
throw new IdempotencyPersistenceLayerError();
}
}

return this.functiontoMakeIdempotent(this.fullFunctionPayload);
}
}
19 changes: 19 additions & 0 deletions packages/idempotency/src/idempotentDecorator.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
/* eslint-disable @typescript-eslint/no-explicit-any */
import { IdempotencyOptions } from './types/IdempotencyOptions';
import { IdempotencyHandler } from './IdempotencyHandler';

const idempotent = function (options: IdempotencyOptions) {
return function (_target: any, _propertyKey: string, descriptor: PropertyDescriptor) {
const childFunction = descriptor.value;
descriptor.value = function(record: Record<string, any>){
const idempotencyHandler: IdempotencyHandler<unknown> = new IdempotencyHandler<unknown>(childFunction, record[options.dataKeywordArgument], options, record);

return idempotencyHandler.process_idempotency();
};

return descriptor;
};
};

export { idempotent };

26 changes: 17 additions & 9 deletions packages/idempotency/src/makeFunctionIdempotent.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,19 @@
import type { AnyFunction } from './types/AnyFunction';
import type { IdempotencyOptions } from './types/IdempotencyOptions';

const makeFunctionIdempotent = <U>(
fn: AnyFunction<U>,
_options: IdempotencyOptions
// TODO: revisit this with a more specific type if possible
/* eslint-disable @typescript-eslint/no-explicit-any */
): (...args: Array<any>) => Promise<U | void> => (...args) => fn(...args);
/* eslint-disable @typescript-eslint/no-explicit-any */
import { AnyFunctionWithRecord, AnyIdempotentFunction } from './types/AnyFunction';
import { IdempotencyOptions } from './types/IdempotencyOptions';
import { IdempotencyHandler } from './IdempotencyHandler';

const makeFunctionIdempotent = function <U>(
fn: AnyFunctionWithRecord<U>,
saragerion marked this conversation as resolved.
Show resolved Hide resolved
options: IdempotencyOptions
): AnyIdempotentFunction<U> {
const wrappedFunction: AnyIdempotentFunction<U> = function (record: Record<string, any>): Promise<U> {
const idempotencyHandler: IdempotencyHandler<U> = new IdempotencyHandler<U>(fn, record[options.dataKeywordArgument], options, record);

return idempotencyHandler.process_idempotency();
};

return wrappedFunction;
};

export { makeFunctionIdempotent };
9 changes: 7 additions & 2 deletions packages/idempotency/src/types/AnyFunction.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
// eslint-disable-next-line @typescript-eslint/no-explicit-any
type AnyFunction<U> = (...args: Array<any>) => Promise<U>;
type AnyFunctionWithRecord<U> = (record: Record<string,any>) => Promise<U> | U;

// eslint-disable-next-line @typescript-eslint/no-explicit-any
type AnyIdempotentFunction<U> = (record: Record<string,any>) => Promise<U>;

export {
AnyFunction
// AnyFunction,
AnyFunctionWithRecord,
AnyIdempotentFunction
};
3 changes: 2 additions & 1 deletion packages/idempotency/src/types/index.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
export * from './AnyFunction';
export * from './IdempotencyRecordStatus';
export * from './PersistenceLayer';
export * from './IdempotencyRecordOptions';
export * from './PersistenceLayer';
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,4 @@ process.env.AWS_LAMBDA_FUNCTION_MEMORY_SIZE = '128';
if (process.env.AWS_REGION === undefined && process.env.CDK_DEFAULT_REGION === undefined) {
process.env.AWS_REGION = 'eu-west-1';
}
process.env._HANDLER = 'index.handler';
process.env._HANDLER = 'index.handler';
Loading