push_project_to_s3 is not working with credentials stored in a prefect block

[uuazed]

I've created a new project using the docker-s3 template. The credentials of my AWS service account are stored in a prefect block. According to the prefect project tutorial, this is the recommended setup. This is the push section in my deployment.yaml :

push:
  - prefect_aws.projects.steps.push_project_to_s3:
        requires: "prefect-aws>=0.3.1"
        bucket: "prefect-..."
        folder: somefolder
        credentials: "{{ prefect.blocks.aws-credentials.prod }}"

When initiating a deploy via prefect deploy --name ... , the step fails with
TypeError: Session.client() got an unexpected keyword argument 'profile_name'

Expectation / Proposal

Using credentials stored in a prefect blocks works. Even better, I can use a prefect s3 block directly.

Traceback / Example

• I would like to help contribute a pull request to resolve this!

[nw-bongheelee]

I am getting the same error.
When using AWS Credentials Block, credentials in the push_project_to_s3 function is set with the following values.

{'aws_access_key_id': 'xxxXXXxxx', 'aws_secret_access_key': 'xxxXXXXxxxXXXxxx', 'aws_session_token': None, 'profile_name': None, 'region_name': 'ap-northeast-2', 'aws_client_parameters': { 'api_version': None, 'use_ssl': True, 'verify': True, 'verify_cert_path': None, 'endpoint_url': None, 'config': None}}

And when I executed the code below
client = boto3.client(
"s3", **credentials, **client_parameters, config=Config(**advanced_config)
)

I get the following error because of the arguments profile_name, aws_client_parameters, which are not required by boto3.client.
TypeError: client() got an unexpected keyword argument 'profile_name'

If you temporarily remove the unnecessary fields as shown below, the deployment will be successful.
del credentials['profile_name']
del credentials['aws_client_parameters']

But when I run the flow, I get the same error in pull_project_from_s3 :)

Additionally, it would be nice to have a way to authenticate via s3 block.

[jacobdanovitch]

Also encountering this. Temporary workaround is to create Secret blocks for each of the required values and build the dictionaries manually:

Secret(value=...).save('deploy-s3-access-key', overwrite=True)
Secret(value=...).save('deploy-s3-secret-key', overwrite=True)
Secret(value=...).save('deploy-s3-endpoint', overwrite=True)

# prefect.yaml

push: # same thing for pull
- prefect_aws.projects.steps.push_project_to_s3:
    # ...
    credentials: 
        aws_access_key_id: "{{ prefect.blocks.secret.deploy-s3-access-key }}"
        aws_secret_access_key: "{{ prefect.blocks.secret.deploy-s3-secret-key }}"
    client_parameters: 
        endpoint_url: "{{ prefect.blocks.secret.deploy-s3-endpoint }}"

[markbruning]

This is potentially fixed by PrefectHQ/prefect-aws#322