[infra] async'ify Credentials class

[thehesiod]

There are at least a couple underlying issues related to this class:

1. IAM credential refresh using requests: Blocking call when loading credentials in Session.create_client #2
2. AssumeRoleProvider uses client call: get_credentials() fails on AioSession #553 + AWS_PROFILE: TypeError: can't pickle coroutine objects #663

First one results in possible hangs of the main thread, while second one is broken because it's not await'ing the request of the client request.

The issue is that these are low-level in classes used by the Credential helper, and ultimately triggered by high level properties like secret_key, access_key and token. We need to find all the calls that will end up needing to be changed to async, and then figure a strategy of how to battle this. I'm afraid this is going to touch a LOT of botocore and greatly increase our footprint and amount of maintenance required.

[mjmdavis]

Is there a workaround for this at the moment?

[mannickutd]

@mjmdavis the work around I used was to use a synchronous botocore call using the refresh method defined in https://www.pydoc.io/pypi/botocore-1.7.0/autoapi/credentials/index.html#credentials.RefreshableCredentials and update credentials in the aiosession for the async clients.

[sbrandtb]

@mannickutd Could you elaborate and maybe provide a code snippet?

[thehesiod]

prefixing with infra as my experiment yielded a LOT of changes

[thehesiod]

btw my PR needs to be re-done to be cleaner by delaying client creation to the __aenter__ so we don't have an extra await.

[thehesiod]

just got permission to expose what our company has been using to resolve this issue internally: https://gist.github.com/thehesiod/05298c1c89c7b5a38da0abc4ccbed7b7