feat(clients): compose transfer handler with middleware & retry middleware #11188

AllanZhengYP · 2023-04-05T00:05:37Z

Description of changes

Propose a function composing HTTP handler(fetch transfer handler) and middleware into a transfer handler.
Minor update to the middleware interface
Propse a general retry middleware implementation for client side. Removing overly complexed retry token mechanism from AWS SDK.

This change is intended to replace the current Core/Util/Retry.ts and standardize the retry behaviors across the board.

Issue #, if available

Description of how you validated changes

Unit test

Checklist

PR description included
yarn test passes
Tests are changed or added
Relevant documentation is changed or added (and PR referenced)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

jimblanc

Looks good! It appears that api-rest bundle size went up with this change, any idea why?

jimblanc · 2023-04-05T14:24:49Z

packages/core/package.json

+		{
+			"name": "Custom clients (retry middleware)",
+			"path": "./lib-esm/clients/middleware/retry.js",
+			"import": "{ retry }",
+			"limit": "1.2 kB"
+		},
+		{
+			"name": "Custom clients (fetch handler)",
+			"path": "./lib-esm/clients/fetch.js",
+			"import": "{ fetchTransferHandler }",
+			"limit": "1.73 kB"


Thank you! 🙏

jimblanc · 2023-04-05T14:51:53Z

packages/core/src/clients/middleware/retry.ts

+					}
+					context[CONTEXT_KEY_RETRY_COUNT] = attemptsCount;
+					continue;
+				} else if (response) {


Just to check, the reason the retryDecider check comes before checking the response is to give us an opportunity to retry requests that don't outright reject if needed?

Yes, we can decide to retry based on a HTTP response, not only error.

svidgen

I'm at my timebox for review on this. I have questions enumerated in my comments, but this is a lot of unfamiliar code to grok. I'm not at all sure you should block on my review — some of these middleware patterns might be inspired by prior art that I haven't really studied. If that's the case, please just tell me!

svidgen · 2023-04-05T20:06:46Z

packages/core/src/clients/middleware/retry.ts

+	backOffStrategy: {
+		/**
+		 * Function to compute the delay in milliseconds before the next retry based
+		 * on the number of attempts.
+		 * @param attempt Current number of attempts, including the first attempt.
+		 * @returns Delay in milliseconds.
+		 */
+		computeDelay: (attempt: number) => number;
+	};
+	/**
+	 * Maximum number of retry attempts, starting from 1. Defaults to 3.
+	 */
+	maxAttempts?: number;


Food for thought: This is surprising nesting and delineation to me. I would assume we opted for a backOffStrategy container because we expect to eventually have more than a single child property here. What else might be part of the backoff strategy in the future? With the container as-is, I'm not sure I could articulate why maxAttempts isn't part of the backoff strategy.

I'm not sure I could articulate why maxAttempts isn't part of the backoff strategy.

This is a very good question. I referred to the interface from AWS SDK, where maxAttempts lives outside of backOffStrategy by standard -- AWS SDK doc.

What else might be part of the backoff strategy in the future?

I'm thinking of the retryBookkeeping() and hasRetryQuota() in the above linked doc. But there isn't any evidence indicating we will need them now. Maybe I should use only 1 layer?

If we're basing this on those docs, backoff strategy (or "mode") should really just be an enum! 😅

If we're basing this on those docs, backoff strategy (or "mode") should really just be an enum! 😅

That would require the code to import all strategies and dynamically resolving the backoff strategy. It would harm the bundle size.

I just removed backOffStrategy parameter and replaced with a computeDelay function. It would support a variety customizations like maxDelay

svidgen · 2023-04-05T21:10:26Z

packages/core/src/clients/middleware/retry.ts

+		 * @param attempt Current number of attempts, including the first attempt.
+		 * @returns Delay in milliseconds.
+		 */
+		computeDelay: (attempt: number) => number;


Are there any cases where the computed delay should ideally be based on a return value or error? Or cases where the backoff should "reset"?

We only need to decide if we need retry with given error or return, instead of how long to wait with given error or return. Same as shown in AWS SDK's shared retry behavior. I'd prefer to avoid prematual optimization.

From those docs, I'm thinking along the lines of the adaptive strategy.

packages/core/src/clients/middleware/retry.ts

packages/core/src/clients/internal/composeTransferHandler.ts

svidgen · 2023-04-05T21:43:33Z

packages/core/__tests__/clients/retry-middleware-test.ts

+			expect(clearTimeout).toBeCalledTimes(1); // cancel 2nd attempt
+		}
+	});
+	test('track attempts count in context across 2 retry middleware', async () => {


This test seems to indicate that including retry in the chain twice is actually just a mistake. If a repeated handler is a mistake, can it throw an exception instead?

On the contrary, this test case is to make sure duplicated retry middleware will not affect the retry functionality. I can have 2 retry middleware, they can use the same retryDecider and back-off the delays coherently.

svidgen · 2023-04-05T21:46:31Z

packages/core/__tests__/clients/retry-middleware-test.ts

+		const doubleRetryableHandler = composeTransferHandler<
+			any,
+			any,
+			any,
+			[RetryOptions, {}, RetryOptions]
+		>(coreHandler, [retry, betweenRetryMiddleware, retry]);
+
+		const retryDecider = jest
+			.fn()
+			.mockImplementation((response, error: Error) => {
+				if (error && error.message.endsWith('RetryableError')) return true;
+				return false;
+			});
+		const backOffStrategy = {
+			computeDelay: jest.fn().mockReturnValue(0),
+		};
+		const response = await doubleRetryableHandler(defaultRequest, {
+			...defaultRetryOptions,
+			retryDecider,
+			backOffStrategy,
+		});


The consequences of some of these composition/interface decisions didn't hit me until reading this. We can't really have a middleware included twice accepts two different distinct sets of parameters.

Am I understanding that right? Is there any middleware we can think of for which that might be a problem?

You are right, we can not have a middleware included twice accepts 2 different sets of parameters.

I can not think of an existing use case where we are limited by this constraint.

packages/core/src/clients/internal/composeTransferHandler.ts

* chore(core): bump to ts 5.0 (#11077) * feat(clients): basic types and fetch handler (#11120) * feat(clients): compose transfer handler with middleware & retry middleware (#11188) * feat(clients): middleware interface and retry middleware * test(clients): retry middleare unit test * fix(clients): middleware type to include options * feat(clients): add retry middleware unit test and update interface * chore(clients): update bundle size limit for fetch and retry * chore(clients): add retry docs; fix format * fix(clients): address feedbacks * fix(retry): add metadata to returns from retry middldeware (#11212) * fix(retry): add metadata to returns from retry middldeware; fix minor bugs * feat(clients): address retry middleware feedbacks Co-authored-by: Chris F <5827964+cshfang@users.noreply.github.com> * chore(core): update size limit * feat(clients): cognito identity client (#11213) * feat(clients): implement service API composer * chore(clients): rename middleware interface * feat(clients): middleware handler interface can be non request/response interface * chore(clients): move fetch handler to handlers folder * feat(clients): implement user agent middleware * feat(clients): implement unauth aws transfer handler * feat(clients): implement api handler composer * feat(clients): implement cognito-identity client * chore(clients): update bundle size limit * feat(clients): add serde utils; remove retry in api handler composer * feat(clients): integrate cognito-identity client to Credentials class * fix(clients): make retryDecider interface async * chore: move cognito client to dev dep * chore(clients): use Pascale path name * fix(clients): handle fetch response.body undefined in RN * chore: publish under v5-custom-clients dist-tag * fix(clients): read body once for errors * chore(clients): enable tagged release of custom clients (#11267) * test(clients): add functional test to cognito identity client (#11266) * test(clients): add cognito-identity functional test * chore(clients): prefer destructuring parameters at top * feat(clients): add useragent to cognito identity (#11269) * chore(clients): add license header to files (#11292) * feat(clients): Add custom signature v4 signer (#11273) * feat(clients): Add custom signature v4 signer * Updated docstrings * Extracted common code * Use == null instead of isNil * Add unit tests * Add missing licensing headers * Use test case options in presign tests * Fixed comment * Add test for data hashing with SourceData keys * Remove buffer dependency * Remove internal sdk dependency * chore(clients): Make signing functions synchronous (#11307) * chore(clients): Replace existing Signer implementation (#11310) * chore(clients): Add @internal annotation (#11320) * feat(clients): Add signing middleware (#11323) * feat(clients): Add signing middleware * Use closure for clock offset * Add bundle size test entry * Address comments * feat(clients): support CN partition by adding DNS suffix resolver (#11311) * feat(clients): support CN partition by adding DNS suffix resolver * chore(clients): update bundle size test limit * fix(clients): address feedbacks * chore: update bundle size limit * chore(clients): Add context to some regex (#11334) * feat(clients) Add updateEndpoint API (#11330) * feat(clients) Add updateEndpoint API * Rename handler * chore(clients): Use DNS suffix util in Pinpoint client (#11340) * chore(clients): Annotate custom client APIs with @internal (#11347) * feat(clients) Add putEvents API (#11342) * feat(clients) Add putEvents API * Add additional verification for expected date format * Mark API as internal * feat(clients) Add getInAppMessages API (#11348) * feat(clients) Add getInAppMessages API * Update unit test * chore(clients): Replace SDK Pinpoint Client (#11359) Co-authored-by: Allan Zheng <zheallan@amazon.com> * chore: update size limit * chore(clients): export pinpoint client from internal subpath (#11369) * feat(clients): allow fetch handler to read body multiple times * chore: update size limit * fix(clients): add react-native entrypoint for internal modules * chore: address feedbacks * feat(clients): vendor TS types from pinpoint and cognito-identity clients (#11393) * chore: dts bundler script for AWS SDK types * feat(clients): vendor AWS SDK client types * chore: workaround the tslint errors for generated rollup types * docs: add readme to build clients script * feat: update SDK types * chore(storage): update size limit (#11406) * fix(clients): middleware chain revert after every invocation (#11432) * chore: prepare for release; split out data packages diffs * fix: address feedbacks * fix(analytics): apply useragent enhancement to pinpoint client * chore: update size limit * fix: Auth bundle size test * fix: Update bundle size for pubsub * fix: api-graphql bundle size update * fix: Update api bundle sizes * fix: DataStore bundle size test --------- Co-authored-by: Chris F <5827964+cshfang@users.noreply.github.com> Co-authored-by: Aaron S <94858815+stocaaro@users.noreply.github.com>

AllanZhengYP added 5 commits March 27, 2023 17:43

feat(clients): middleware interface and retry middleware

4699323

test(clients): retry middleare unit test

947b9ca

fix(clients): middleware type to include options

927d9b7

feat(clients): add retry middleware unit test and update interface

be00c88

chore(clients): update bundle size limit for fetch and retry

b3b0f88

AllanZhengYP marked this pull request as ready for review April 5, 2023 00:05

AllanZhengYP requested review from a team as code owners April 5, 2023 00:05

AllanZhengYP requested review from svidgen and haverchuck April 5, 2023 00:06

AllanZhengYP force-pushed the v5/custom-clients-middleware branch from 35c549f to ac0813f Compare April 5, 2023 00:08

chore(clients): add retry docs; fix format

d4d0a7b

AllanZhengYP force-pushed the v5/custom-clients-middleware branch from ac0813f to d4d0a7b Compare April 5, 2023 00:16

jimblanc reviewed Apr 5, 2023

View reviewed changes

AllanZhengYP requested a review from jimblanc April 5, 2023 17:24

jimblanc approved these changes Apr 5, 2023

View reviewed changes

svidgen reviewed Apr 5, 2023

View reviewed changes

fix(clients): address feedbacks

a0f256a

AllanZhengYP requested review from svidgen and iartemiev April 6, 2023 00:57

AllanZhengYP merged commit 85bc134 into v5/custom-clients Apr 7, 2023

AllanZhengYP deleted the v5/custom-clients-middleware branch May 30, 2023 00:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(clients): compose transfer handler with middleware & retry middleware #11188

feat(clients): compose transfer handler with middleware & retry middleware #11188

AllanZhengYP commented Apr 5, 2023 •

edited

Loading

jimblanc left a comment

jimblanc Apr 5, 2023

jimblanc Apr 5, 2023

AllanZhengYP Apr 5, 2023

svidgen left a comment

svidgen Apr 5, 2023

AllanZhengYP Apr 5, 2023 •

edited

Loading

svidgen Apr 6, 2023 •

edited

Loading

AllanZhengYP Apr 6, 2023 •

edited

Loading

svidgen Apr 5, 2023

AllanZhengYP Apr 5, 2023

svidgen Apr 6, 2023

svidgen Apr 5, 2023

AllanZhengYP Apr 5, 2023

svidgen Apr 5, 2023

AllanZhengYP Apr 5, 2023

feat(clients): compose transfer handler with middleware & retry middleware #11188

feat(clients): compose transfer handler with middleware & retry middleware #11188

Conversation

AllanZhengYP commented Apr 5, 2023 • edited Loading

Description of changes

Issue #, if available

Description of how you validated changes

Checklist

jimblanc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

svidgen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AllanZhengYP Apr 5, 2023 • edited Loading

Choose a reason for hiding this comment

svidgen Apr 6, 2023 • edited Loading

Choose a reason for hiding this comment

AllanZhengYP Apr 6, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AllanZhengYP commented Apr 5, 2023 •

edited

Loading

AllanZhengYP Apr 5, 2023 •

edited

Loading

svidgen Apr 6, 2023 •

edited

Loading

AllanZhengYP Apr 6, 2023 •

edited

Loading