chore(ci): bump riot version for 3.7 compatibility issues (#11085)

With the new release of virtualenv==20.27.0, we were seeing new failures
with build_base_venv[3.7] (GitLab) and build_base_venv (CircleCI) jobs
when running:
```
riot -P -v generate --python=$PYTHON_VERSION
...
  File "/go/src/github.com/DataDog/apm-reliability/dd-trace-py/.riot/venv_py3717/lib/python3.7/site-packages/pip/_vendor/typing_extensions.py", line 1039
    def TypedDict(typename, fields=_marker, /, *, total=True, closed=False, **kwargs):
                                            ^
SyntaxError: invalid syntax
ERROR:riot.riot:Dev install failed, aborting!
```
We need to pin to virtualenv==20.26.6 to continue supporting python3.7

The upload-artifact and download-artifact bumps were required to unblock
the riot CI. It had been a while since the last update to riot, and
since then, v2 has been deprecated and v3 will be deprecated next month
(Nov 2024), so we are bumping to v4 via tag.

The above changes were implemented in riot
(DataDog/riot#232), and this PR just bumps the
riot version in dd-trace-py from 0.19.1 to 0.20.0

.circleci/config.templ.yml

@@ -80,7 +80,7 @@ commands:
     description: "Install riot"
     steps:
       # Make sure we install and run riot on Python 3
-      - run: pip3 install riot==0.19.1
+      - run: pip3 install riot==0.20.0
 
   setup_rust:
     description: "Install rust toolchain"

.circleci/config.yml

@@ -31,7 +31,7 @@ jobs:
           name: Generate config
           command: |
             export GIT_COMMIT_DESC=$(git log -n 1 $CIRCLE_SHA1)
-            pip3 install riot==0.19.1
+            pip3 install riot==0.20.0
             riot -P -v run --pass-env -s circleci-gen-config -- -v
       - continuation/continue:
           configuration_path: .circleci/config.gen.yml

.github/workflows/requirements-locks.yml

@@ -23,7 +23,7 @@ jobs:
         run: pyenv global 3.10 3.7 3.8 3.9 3.11 3.12
 
       - name: Install Dependencies
-        run: pip install --upgrade pip && pip install riot
+        run: pip install --upgrade pip && pip install riot==0.20.0
 
       - name: Generate riot locks
         run: scripts/compile-and-prune-test-requirements

hatch.toml

@@ -22,7 +22,7 @@ dependencies = [
     "ddapm-test-agent>=1.2.0",
     "packaging==23.1",
     "pygments==2.16.1",
-    "riot==0.19.1",
+    "riot==0.20.0",
     "ruff==0.1.3",
     "clang-format==18.1.5",
 ]

scripts/ddtest

@@ -12,7 +12,8 @@ fi
 # retry docker pull if fails
 for i in {1..3}; do docker-compose pull -q testrunner && break || sleep 3; done
 
-FULL_CMD="pip install -q --disable-pip-version-check riot==0.19.1 hatch && $CMD"
+# TODO(DEV): Install riot in the docker image
+FULL_CMD="pip install -q --disable-pip-version-check riot==0.20.0 && $CMD"
 
 
 # install and upgrade riot in case testrunner image has not been updated