Add support for AWS_ENDPOINT_URL environment variable

[Walther]

Describe the bug

AWS_ENDPOINT_URL in the environment gets ignored by the SDK.

Expected Behavior

The correct endpoint URL will be loaded from the environment variables and used for the SDK client(s).

Current Behavior

The endpoint URL gets ignored, and requests will be made against the AWS production endpoints.

Reproduction Steps

main.rs:

use anyhow::Result;
use dotenv::dotenv;

#[tokio::main]
async fn main() -> Result<()> {
    // load configs
    dotenv().ok();
    let region = dotenv::var("AWS_DEFAULT_REGION")?;
    println!("{region}");
    let endpoint = dotenv::var("AWS_ENDPOINT_URL")?;
    println!("{endpoint}");

    // load aws config
    let awsconfig = aws_config::load_from_env().await;
    let s3client = aws_sdk_s3::Client::new(&awsconfig);
    println!("{:?}", s3client);

    Ok(())
}

.env:

AWS_ACCESS_KEY_ID=test
AWS_SECRET_ACCESS_KEY=test
AWS_DEFAULT_REGION=eu-north-1
AWS_ENDPOINT_URL=http://localstack:4566
AWS_ENDPOINT_URL_S3=http://localstack:4566
ENDPOINT_URL=http://localstack:4566

(Note: I've included a couple of additional possible spellings of the endpoint URL in order to eliminate the possibility of the error originating from a slight naming difference)

Running this code and searching the output for EndpointUrl shows that the endpoint URL has not been configured correctly.

Possible Solution

Workaround: manually setting the endpoint override works, but this is not ideal:

use anyhow::Result;
use dotenv::dotenv;

#[tokio::main]
async fn main() -> Result<()> {
    // load configs
    dotenv().ok();
    let region = dotenv::var("AWS_DEFAULT_REGION")?;
    println!("{region}");
    let endpoint = dotenv::var("AWS_ENDPOINT_URL")?;
    println!("{endpoint}");

    // load aws config
    let awsconfig = aws_config::from_env().endpoint_url(endpoint).load().await;
    let s3client = aws_sdk_s3::Client::new(&awsconfig);
    println!("{:?}", s3client);

    Ok(())
}

---

I wonder if this error is originating from here:

aws-sdk-rust/sdk/aws-config/src/lib.rs

Lines 670 to 677 in 464d782

	builder.set_app_name(app_name);
	builder.set_credentials_cache(credentials_cache);
	builder.set_credentials_provider(credentials_provider);
	builder.set_sleep_impl(sleep_impl);
	builder.set_endpoint_url(self.endpoint_url);
	builder.set_use_fips(use_fips);
	builder.set_use_dual_stack(use_dual_stack);
	builder.build()

Additional Information/Context

Being able to set the endpoint URL would be really useful for local development. Consider for example the following snippet for Docker Compose, utilizing LocalStack:

services:
  localstack:
    container_name: "${LOCALSTACK_DOCKER_NAME-localstack_main}"
    image: localstack/localstack:2.3
    ports:
      - "4566:4566" # LocalStack Gateway
    environment:
      - SERVICES=s3:4566
      - DEBUG=${DEBUG-}
    volumes:
      - "${LOCALSTACK_VOLUME_DIR:-./volume}:/var/lib/localstack"

  createbucket:
    image: amazon/aws-cli:2.13.29
    depends_on:
      - localstack
    volumes:
      - bucket:/aws
    environment:
      - AWS_ACCESS_KEY_ID=test
      - AWS_SECRET_ACCESS_KEY=test
      - AWS_DEFAULT_REGION=eu-north-1
    entrypoint: /bin/sh -c
    command: >
      "
        aws --endpoint-url=http://localstack:4566 s3api create-bucket --bucket testbucket --region eu-north-1 --create-bucket-configuration LocationConstraint=eu-north-1
      "

  # other services, including the client that uses the Rust SDK

volumes:
  bucket:

and the client container having a .env file with the appropriate AWS_ENDPOINT_URL and other env vars.

Version

aws-config = "0.56.1"
aws-sdk-s3 = "0.34.0"

Environment details (OS name and version, etc.)

macOS Sonoma 14.0

Logs

No response

[rcoh]

Yeah that's correct. Not yet supported by the SDK.

[NishitHingu]

Hi, I am new to aws-sdk-rust, and I have started to work on this issue. I think we need to set the endpoint_url from the environment variable at

aws-sdk-rust/sdk/aws-config/src/lib.rs

Lines 592 to 602 in 464d782

	let region = if let Some(provider) = self.region {
	provider.region().await
	} else {
	region::Builder::default()
	.configure(&conf)
	.build()
	.region()
	.await
	};
	let retry_config = if let Some(retry_config) = self.retry_config {

endpoint_url is to be taken from the environment variable only if endpoint_url is not overridden, similar to what we are doing for region in the above code snippet.

[jmj0502]

@rcoh Hey! Hope you're doing well!

Is this issue still open? I would like to give it a shot, the project is interesting to me (since I use both the language and AWS as a platform) and I would like to contribute to the development in one way or another. This issue seems like a good starting point c:

[rcoh]

yep this issue is still open

[landonxjames]

This was updated in smithy-lang/smithy-rs#3488 and should work as expected now! Updating to the latest versions should fix your issue, if it doesn't please file a new issue and we will research the bug.

[github-actions]

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.