chore(ci): bump riot version for 3.7 compatibility issues [backport 2…

….14] (#11092)

Manual backport of #11085 to 2.14

With the new release of virtualenv==20.27.0, we were seeing new failures
with build_base_venv[3.7] (GitLab) and build_base_venv (CircleCI) jobs
when running:
```
riot -P -v generate --python=$PYTHON_VERSION
...
  File "/go/src/github.com/DataDog/apm-reliability/dd-trace-py/.riot/venv_py3717/lib/python3.7/site-packages/pip/_vendor/typing_extensions.py", line 1039
    def TypedDict(typename, fields=_marker, /, *, total=True, closed=False, **kwargs):
                                            ^
SyntaxError: invalid syntax
ERROR:riot.riot:Dev install failed, aborting!
```
We need to pin to virtualenv==20.26.6 to continue supporting python3.7

The upload-artifact and download-artifact bumps were required to unblock
the riot CI. It had been a while since the last update to riot, and
since then, v2 has been deprecated and v3 will be deprecated next month
(Nov 2024), so we are bumping to v4 via tag.

The above changes were implemented in riot
(DataDog/riot#232), and this PR just bumps the
riot version in dd-trace-py from 0.19.1 to 0.20.0

## Checklist
- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

## Reviewer Checklist
- [x] Reviewer has checked that all the criteria below are met 
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)

.circleci/config.templ.yml

@@ -80,7 +80,7 @@ commands:
     description: "Install riot"
     steps:
       # Make sure we install and run riot on Python 3
-      - run: pip3 install riot==0.19.1
+      - run: pip3 install riot==0.20.0
 
   setup_rust:
     description: "Install rust toolchain"

.circleci/config.yml

@@ -31,7 +31,7 @@ jobs:
           name: Generate config
           command: |
             export GIT_COMMIT_DESC=$(git log -n 1 $CIRCLE_SHA1)
-            pip3 install riot==0.19.1
+            pip3 install riot==0.20.0
             riot -P -v run --pass-env -s circleci-gen-config -- -v
       - continuation/continue:
           configuration_path: .circleci/config.gen.yml

.github/workflows/generate-package-versions.yml

@@ -13,6 +13,7 @@ jobs:
       actions: read
       contents: write
       pull-requests: write
+
     steps:
       - uses: actions/checkout@v4
 
@@ -45,7 +46,7 @@ jobs:
         uses: actions/setup-python@v5
         with:
           python-version: "3.12"
-      
+
       - name: Set up QEMU
         uses: docker/setup-qemu-action@v2
 
@@ -76,7 +77,7 @@ jobs:
           python -m pip install --upgrade pip
           pip install packaging
           pip install requests
-          pip install riot==0.19.1
+          pip install riot==0.20.0
 
       - name: Run regenerate-riot-latest
         run: scripts/regenerate-riot-latest.sh
@@ -98,4 +99,4 @@ jobs:
           base: main
           title: "chore: update ${{ env.VENV_NAME }} latest version to ${{ env.NEW_LATEST }}"
           labels: changelog/no-changelog
-          body-path: .github/PULL_REQUEST_TEMPLATE.md
+          body-path: .github/PULL_REQUEST_TEMPLATE.md

.github/workflows/requirements-locks.yml

@@ -26,7 +26,7 @@ jobs:
         run: pyenv global 3.10 3.7 3.8 3.9 3.11 3.12
 
       - name: Install Dependencies
-        run: pip install --upgrade pip && pip install riot
+        run: pip install --upgrade pip && pip install riot==0.20.0
 
       - name: Generate riot locks
         run: scripts/compile-and-prune-test-requirements

.gitlab/package.yml

@@ -8,7 +8,7 @@ build_base_venvs:
     CMAKE_BUILD_PARALLEL_LEVEL: 12
     PIP_VERBOSE: 1
   script:
-    - pip install riot~=0.19.1
+    - pip install riot==0.20.0
     - riot -P -v generate --python=$PYTHON_VERSION
   artifacts:
     name: venv_$PYTHON_VERSION

.gitlab/tests.yml

@@ -40,7 +40,7 @@ variables:
   services:
     - !reference [.services, ddagent]
   script:
-    - pip install riot~=0.19.1
+    - pip install riot==0.20.0
     - unset DD_SERVICE
     - unset DD_ENV
     - unset DD_TAGS

hatch.toml

@@ -17,7 +17,7 @@ dependencies = [
     "ddapm-test-agent>=1.2.0",
     "packaging==23.1",
     "pygments==2.16.1",
-    "riot==0.19.1",
+    "riot==0.20.0",
     "ruff==0.1.3",
     "clang-format==18.1.5",
 ]

scripts/ddtest

@@ -19,8 +19,7 @@ fi
 for i in {1..3}; do $compose_cmd pull -q testrunner && break || sleep 3; done
 
 # TODO(DEV): Install riot in the docker image
-FULL_CMD="pip install -q --disable-pip-version-check riot==0.19.1 && $CMD"
-
+FULL_CMD="pip install -q --disable-pip-version-check riot==0.20.0 && $CMD"
 
 # install and upgrade riot in case testrunner image has not been updated
 # DEV: Use `--no-TTY` and `--quiet-pull` when running in CircleCI