Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(endpoints)!: integrate endpoints 2.0 #607

Merged
merged 23 commits into from
Oct 27, 2022
Merged

Conversation

ganeshnj
Copy link
Contributor

@ganeshnj ganeshnj commented Aug 17, 2022

Issue #

#624

Description of changes

Counterpart: smithy-lang/smithy-swift#433

Endpoint struct

Endpoint struct now supports attributes and headers properties.

public struct Endpoint {
    let url: String
    let attributes: [String: Any]
    let headers: [String: [String]]
}

EndpointResolver

By default, EndpointResolver is a wrapper on top of Common Runtime (CRT) component, but could be overridden by customers using service client configuration that require some variance in how an endpoint is resolved. Both EndpointResolver and EndpointParameter are service specific and codegen to AWS<service name> package.

public protocol EndpointResolver {
    func resolve(parameters: EndpointParameters) throws -> Endpoint
}

public struct EndpointParameters {
    let region: String?
    let useFIPSEndpoint: Bool
    let useDualStackEndpoint: Bool
    let endpointId: String?
}

Client & Configuration

Service client initializer accepts service specific configuration <service name>ClientConfigurationProtocol instead base AWSClientRuntime.AWSClientConfiguration. This is breaking change to existing service clients which take a generic client configuration. This change is required to allow service specific configuration parameters and endpoint resolver.

public class S3Client {
    let config: S3ClientConfigurationProtocol

    public init(config: S3ClientConfigurationProtocol) {
    }
}
public protocol S3ClientConfigurationProtocol : AWSClientRuntime.AWSClientConfiguration {
    var endpointResolver: EndpointResolver { get }
}
public class S3ClientConfiguration: S3ClientConfigurationProtocol {
    // omitted for brevity
    public var endpointResolver: EndpointResolver
    // omitted for brevity
}

Client configuration initiazlier now updated with alphatic order of the params which is also a breaking change because previously region was always the first parameter and runtimeConfig last. This creates confusing and inconsistent experience comparing with other structs we have in the SDK where params follows alphabatic order make it easier to introduce more params without breaking code.

Middleware

Middleware is going to be service specific to handle customization for endpoint parameters creation. EndpointMiddleware is codegen in the service package. Endpoint Parameters are service specific which are also codegen.

For an hypothetical PutEvents operation in an example service

Request and response

public struct PutEventsInput: Swift.Equatable {
    let endpointId: String?
}

public struct PutEventsOutputResponse: Swift.Equatable {
}

EndpointResolverMiddleware

EndpointResolverMiddleware is service specific which takes dependency on service specific EndpointResolver & EndpointParams.

public struct EndpointResolverMiddleware<OperationStackOutput: ClientRuntime.HttpResponseBinding, OperationStackError: ClientRuntime.HttpResponseBinding>: ClientRuntime.Middleware {
    public let id: Swift.String = "EndpointResolverMiddleware"

    let endpointResolver: EndpointResolver

    let endpointParams: EndpointParams

    public init(endpointResolver: EndpointResolver, endpointParams: EndpointParams) {
        self.endpointResolver = endpointResolver
        self.endpointParams = endpointParams
    }

    public func handle<H>(context: Context,
                  input: ClientRuntime.SdkHttpRequestBuilder,
                  next: H) async throws -> ClientRuntime.OperationOutput<OperationStackOutput>
    where H: Handler,
    Self.MInput == H.Input,
    Self.MOutput == H.Output,
    Self.Context == H.Context
    {
        var endpoint: Endpoint
        do {
            endpoint = try endpointResolver.resolve(params: endpointParams)
        } catch {
            throw ClientRuntime.SdkError<OperationStackError>.client(ClientError.unknownError(("Unable to resolve endpoint.")))
        }

        guard let authScheme = endpoint.authScheme(name: "v4") else {
            throw ClientRuntime.SdkError<OperationStackError>.client(ClientError.unknownError(("Unable to resolve endpoint. Missing auth scheme.")))
        }

        let awsEndpoint = AWSEndpoint(endpoint: endpoint, signingName: authScheme["signingName"] as? String, signingRegion: authScheme["signingRegion"] as? String)

        var host = ""
        if let hostOverride = context.getHost() {
            host = hostOverride
        } else {
            host = "\(context.getHostPrefix() ?? "")\(awsEndpoint.endpoint.host)"
        }

        if let protocolType = awsEndpoint.endpoint.protocolType {
            input.withProtocol(protocolType)
        }

        var updatedContext = context
        if let signingRegion = awsEndpoint.signingRegion {
            updatedContext.attributes.set(key: HttpContext.signingRegion, value: signingRegion)
        }
        if let signingName = awsEndpoint.signingName {
            updatedContext.attributes.set(key: HttpContext.signingName, value: signingName)
        }

        input.withMethod(context.getMethod())
            .withHost(host)
            .withPort(awsEndpoint.endpoint.port)
            .withPath(context.getPath())
            .withHeader(name: "Host", value: host)

        return try await next.handle(context: updatedContext, input: input)
    }

    public typealias MInput = ClientRuntime.SdkHttpRequestBuilder
    public typealias MOutput = ClientRuntime.OperationOutput<OperationStackOutput>
    public typealias Context = ClientRuntime.HttpContext
}

EndpointMiddleware usage

Similar to other middleware construction, EndpointResolverMiddlware is injected before the buildStep with extra codegen endpointParams instance that goes to the init.

let endpointParams = EndpointParams(accelerate: config.accelerate ?? false,
    bucket: input.bucket,
    endpoint: config.endpoint,
    forcePathStyle: config.forcePathStyle,
    region: config.region,
    useArnRegion: config.useArnRegion,
    useDualStack: config.useDualStack ?? false,
    useFips: config.useFips ?? false)
operation.buildStep.intercept(position: .before, middleware: EndpointResolverMiddleware<AbortMultipartUploadOutputResponse, AbortMultipartUploadOutputError>(endpointResolver: config.endpointResolver, endpointParams: endpointParams))

User experience

Case 1: client init with region

This is simplest way to instantiate a service client by just providing the region as a param.

let client = try S3Client(region: "us-west-2")
let input = GetObjectInput(bucket: "jangirg-testing", key: "helloworld")
let output = try await client.getObject(input: input)

Case 2: client init with config

Customers can instantiate the config with their endpoint specific configurations such as useDualStack, useFips or any client level configuration and pass down to the service client init.

let config = try S3Client.S3ClientConfiguration(region: "us-west-2", useDualStack: true)
let client = S3Client(config: config)
let input = GetObjectInput(bucket: "jangirg-testing", key: "helloworld")
let output = try await client.getObject(input: input)

Case 3: customer endpoint resolver

This is advanced use case where customers can override the EndpointResolver with their own implementation. First thing first, EndpointResolver protocol must be confirmed by their own implementation.

struct CustomEndpointResolver : EndpointResolver {
    func resolve(params: EndpointParams) throws -> Endpoint {
        let properties: [String: AnyHashable] = [
            "authSchemes": [
                ["name": "sigv4",
                 "signingName": "s3",
                 "signingRegion": "us-west-2"]
            ],
        ]
        return try Endpoint(urlString: "http://s3.us-west-2.amazonaws.com", headers: nil, properties: properties)
    }
}

Followed by creating the client config by passing confirming struct/class

let customResolver = DefaultEndpointResolver()
let config = try S3Client.S3ClientConfiguration(endpointResolver: customResolver, region: "us-west-2")
let client = S3Client(config: config)
let input = GetObjectInput(bucket: "jangirg-testing", key: "helloworld")
let output = try await client.getObject(input: input)

New/existing dependencies impact assessment, if applicable

Conventional Commits

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@ganeshnj ganeshnj changed the title endpoints refactor feat: endpoints refactor Aug 17, 2022
@ganeshnj ganeshnj changed the title feat: endpoints refactor refactor: endpoints Aug 24, 2022
@ganeshnj ganeshnj force-pushed the jangirg/feat/endpoints-2 branch 2 times, most recently from f9aaf38 to 8fea690 Compare August 30, 2022 00:49
@ganeshnj ganeshnj changed the title refactor: endpoints feat(endpoints)!: integrate endpoints 2.0 Aug 30, 2022
@ganeshnj ganeshnj marked this pull request as ready for review August 30, 2022 01:01
Comment on lines 28 to +37
/**
The service name that should be used for signing requests to this endpoint.
This overrides the default signing name used by an SDK client.
*/
let signingName: String?
public let signingName: String?
/**
The region that should be used for signing requests to this endpoint.
This overrides the default signing region used by an SDK client.
*/
let signingRegion: String?
public let signingRegion: String?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: Aren't these superseded by similar entries in Endpoint.properties?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, they have now become convenience properties. Just removing them will result in a bigger refactor given their references all over.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'm missing something. Are these pre-existing fields now backed by the properties bag somehow? How will existing code that utilizes them get valid values?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is what you are looking for in a test case https://github.com/awslabs/aws-sdk-swift/blob/jangirg/feat/endpoints-2/codegen/smithy-aws-swift-codegen/src/test/kotlin/software/amazon/smithy/aws/swift/codegen/EndpointResolverMiddlewareTests.kt#L50-L54

I take back the word convenience rather authScheme is determined first and then passed to AWSEndpoint init.

guard let authScheme = endpoint.authScheme(name: "sigv4") else {
    throw ClientRuntime.SdkError<OperationStackError>.client(ClientError.unknownError(("Unable to resolve endpoint. Unsupported auth scheme.")))
}

let awsEndpoint = AWSEndpoint(endpoint: endpoint, signingName: authScheme["signingName"] as? String, signingRegion: authScheme["signingRegion"] as? String)

codegen/smithy-aws-swift-codegen/build.gradle.kts Outdated Show resolved Hide resolved
@jbelkins jbelkins self-requested a review September 2, 2022 16:33
Copy link
Contributor

@jbelkins jbelkins left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed more for training purposes than for correctness or architecture. Interesting to see how types and their tests are generated.

@ChristopheBougere
Copy link

@ganeshnj so this PR should enable using S3 Transfer Acceleration and fix #610, right?

@ganeshnj
Copy link
Contributor Author

@ganeshnj so this PR should enable using S3 Transfer Acceleration and fix #610, right?

A new config params will be added that will allow virtual host styling required for S3 Transfer Acceleration.

@ganeshnj ganeshnj merged commit 2c0d598 into main Oct 27, 2022
@ganeshnj ganeshnj deleted the jangirg/feat/endpoints-2 branch October 27, 2022 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants