Update Python runtimes and add Python 3.10 runtime

[michael-k]

• Issue #, if available: Feature request: Python 3.10 support #501

• Description of changes:

  • Added Python version: 3.10.1 3.10.2 3.10.4

  • Updated Python versions:

    • 3.7.10 → 3.7.12 3.7.13
    • 3.8.10 → 3.8.12 3.8.13
    • 3.9.5 → 3.9.9 3.9.10 3.9.12 3.9.13 (final regular maintenance release)

  • Other updated versions:

    • pip: 21.1.2 → 21.3.1 22.0.4 (Changelog)
    • PyYAML: 5.4.1 → 6.0 (Changelog)
    • OpenSSL: 1.1.1k → 1.1.1m 1.1.1n 1.1.1o (Changelog)
      Note: I did not touch OpenSSL versions < 1.1.1k

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[michael-k]

Python 3.9.10 and 3.10.2 are now available, see https://pythoninsider.blogspot.com/2022/01/python-3102-3910-and-3110a4-are-now.html. I'll try to update this PR later this week. I've updated the PR.

• The next Python 3.10 maintenance release will be 3.10.3, currently scheduled for 2022-04-04.
• The next Python 3.9 maintenance release will be 3.9.11, currently scheduled for Pi Day '22 (2022-03-14).

[keithhackbarth]

@leoherran-aws @jpculp @dougnukem @xgnl - What's the process for getting this merged in?

[michael-k]

I've resolved the merge conflict by rebasing.

Python 3.7.13, 3.8.13, 3.9.12, and 3.10.4 are now available, see https://pythoninsider.blogspot.com/2022/03/python-3103-3911-3813-and-3713-are-now.html and https://pythoninsider.blogspot.com/2022/03/python-3104-and-3912-are-now-available.html. I've updated the PR.

• The next maintenance release of Python 3.10 is planned for early June.
• The next maintenance release of Python 3.9 is planned for mid May. After that the 3.9 series will enter its security-only fixes period.
  • Update: This has happened and Python 3.9 is now in its security-only fixes period.
    https://pythoninsider.blogspot.com/2022/05/python-3913-is-now-available.html

[michael-k]

Thank you AWS for 2dea17b. This was just as I expected things to do down. (╯°□°）╯︵ ┻━┻

---

From CONTRIBUTING.md:

Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that:

1. You are working against the latest source on the master branch.
2. You check existing open, and recently merged, pull requests to make sure someone else hasn't addressed the problem already.
3. You open an issue to discuss any significant work - we would hate for your time to be wasted.

Contributions via pull requests are much appreciated.

That sounds like a lie. You are ignoring pull requests from non-AWS folks in 90+% for most of your repos.

You open an issue to discuss any significant work - we would hate for your time to be wasted.

How to discuss when no one from AWS is responding?

[subinataws]

Python 3.10 is now supported in Ubuntu based images https://github.com/aws/aws-codebuild-docker-images/blob/master/ubuntu/standard/6.0/runtimes.yml. This version will get added for AL2 image in the next release. Resolving this PR, since CodeBuild builds its images from an internal repo and this GitHub repo is more for reference purposes on what CodeBuild images contain.

Thanks for your contribution.

[obataku]

@subinataws @michael-k @Dylan-AWS what about python 3.9.x in the standard:6.0 image? what is the rationale behind exposing this only in 5.0? right now 5.0 ships with a buggy version of npm by default but our lambdas (and their builds) require python 3.9 (3.10 is not yet supported)

[subinataws]

@obataku - Going forward, CodeBuild will only be providing one runtime version per language runtime in each image release. Providing multiple version of the language runtimes has led to errors and delays in our image release process. I didn't quite follow the issue you are facing with npm and python for your Lambda function. Could you highlight the bug in standard 5.0 for npm? Sounds like that's the one to fix from our end.

[obataku]

@subinataws unfortunately the bug has not been fixed in node 14/npm 6 (see npm/cli#4896) although there are a few manual workarounds (e.g. upgrading npm to 7 or 8.x; configuring git to rewrite git:// URIs; etc). we would also be served well by introducing node 16 to standard:5.0 if possible, but I figured keeping python 3.9 in standard:6.0 might be simpler. what do you think?

as for providing one runtime version per image release, does that mean e.g. node 18 will have to wait for standard:7.0? since replacing the single available runtime version would be a breaking change

[subinataws]

@obataku - we unfortunately don't have plans to add more than one version of any programming language to standard images going forward. We also are not planning to add newer version to existing images, as that makes these (already large) images bigger.

Adding more than one runtime version introduced a lot of edge cases and bugs that ultimately ended up delaying the availability of newer runtime versions for all users and resulted in slower provisioning time when a image or a layer of an image is removed from the cache.

We'd like to understand the specific dependency that you have on python and node version for your Lambda function. Are you able to create a support case with AWS to look at this further?

[obataku]

as for providing one runtime version per image release, does that mean e.g. node 18 will have to wait for standard:7.0? since replacing the single available runtime version would be a breaking change

@subinataws can you advise regarding codebuild image versioning moving forward?

---

We'd like to understand the specific dependency that you have on python and node version for your Lambda function. Are you able to create a support case with AWS to look at this further?

there are more-or-less unpleasant/tedious workarounds but ultimately our lambda build & packaging (via serverless) depends on node with npm 7+ to avoid a nasty github dependency resolution bug though our lambdas themselves must be targeted to python 3.9 (since Lambda has yet to support 3.10: aws/aws-lambda-base-images#31), which is complicated in a 3.10 build environment since binary extensions are minor-version specific. our current options for workaround look like:

1. use an older image with 3.9 and manually upgrade node/npm
2. use a newer image and manually downgrade python using pyenv
3. build a custom image with 3.9 and new node/npm

and so on

the first is undoubtedly the easiest and we can maintain our workaround for now but the best case scenario would involve

1. lambda support for 3.10 to avoid binary incompatibility issues when packaging in codebuild, or
2. codebuild providing an image with python 3.9 and node 16 ootb

[subinataws]

Node 18 support will be added in a CodeBuild image, after that version is LTS (after 10/25) in a new standard image, whichever the version happens to be at that point.