Update document v2.12.0 (#4443)

* [🤖] Update Latest Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/13477718964 * Remove force-executed tests and scenario from system-tests workflow after release 2.11.0 * Add AppSec RestClient instrumentation for SSRF detection * Add type signatures for AppSec RestClient integration * Fix type signatures for excon and faraday AppSec integrations * Add tests for RestClient AppSec instrumentation * Extract rest-client-latest appraisal * [🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/13455470094 * [🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/13455581146 * Remove unneeded variable from RestClient SSRF integration test * [🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/13457056485 * Add rest-client gem to rake edge:update task * [🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/13496444973 * Rename RequestPatch to RequestSSRFDetectionPatch * [🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/13497053640 * Disable rubocop warning for RequestSSRFDetectionPatch * [PROF-11151] Fix test-memory-leaks flaky behavior **What does this PR do?** This PR will hopefully fix almost [all of the the flakiness](https://github.com/DataDog/dd-trace-rb/pulls?q=is%3Apr+asan) we've seen from the test-memory-leaks GitHub workflow. The previous "asan" builds we were using were built from ruby-head, which means that any instability or early breakage in ruby-head would make test-memory-leaks fail. To fix this, I've worked with upstream to create these 3.4-asan builds: these are Ruby builds that are built **from the latest 3.4 stable Ruby** with asan. Thus any breakages we see in them, should also exist in regular 3.4 builds. **Motivation:** With this change, the test-memory-leaks workflow becomes a lot more valuable, since it's now never expected to fail. Thus, any failures we see in it are worth investigating. **Additional Notes:** For context, asan (or ASan) is the "AddressSanitizer" tool, see https://github.com/google/sanitizers/wiki/AddressSanitizer for more details. **How to test the change?** Validate that the updated workflow is running on ruby-3.4.2 and that it still has the diagnostic output from asan. * DEBUG-3535 use core transport for DI to support unix domain sockets (#4426) * Composite action * Fix publish * Fix doc step with branch * Dependency inject logger (partial) (#4432) * Add 2.12.0 to CHANGELOG.md * Bump version 2.11.0 to 2.12.0 * Update lockfiles for release 2.12.0 --------- Co-authored-by: TonyCTHsu <16049123+TonyCTHsu@users.noreply.github.com> Co-authored-by: Yury Lebedev <yury.lebedev@datadoghq.com> Co-authored-by: y9v <1379701+y9v@users.noreply.github.com> Co-authored-by: Yury Lebedev <lebedev.yurii@gmail.com> Co-authored-by: Ivo Anjo <ivo.anjo@datadoghq.com> Co-authored-by: Oleg Pudeyev <156273877+p-datadog@users.noreply.github.com>
DataDog · Feb 27, 2025 · 54efa06 · 54efa06
1 parent 06f110a
commit 54efa06
Show file tree

Hide file tree

Showing 717 changed files with 4,649 additions and 1,194 deletions.
diff --git a/.github/actions/build-test/action.yml b/.github/actions/build-test/action.yml
@@ -0,0 +1,60 @@
+name: 'Build and Test'
+
+# TODO: Improve  description
+description: 'Configure test suite in batches'
+
+inputs:
+  alias:
+    description: 'Runtime alias'
+    required: true
+  container-id:
+    description: 'Container Identifier'
+    required: true
+
+runs:
+  using: "composite"
+  steps:
+  - name: Configure Git
+    run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
+    shell: bash
+
+  - run: bundle exec rake github:run_batch_build
+    shell: bash
+
+  - name: Configure RSpec
+    run: ln -s .rspec-local.example .rspec-local
+    shell: bash
+
+  - run: bundle exec rake github:run_batch_tests
+    shell: bash
+    env:
+      COVERAGE_DIR: coverage/versions/${{ inputs.alias }}/${{ inputs.container-id }}
+
+  - name: Debug with SSH connection
+    if: ${{ failure() && runner.debug == '1' }}
+    uses: mxschmitt/action-tmate@e5c7151931ca95bad1c6f4190c730ecf8c7dde48 # v3.19.0
+    with:
+      limit-access-to-actor: true
+      # This mode will wait at the end of the job for a user to connect and then to terminate the tmate session.
+      # If no user has connected within 10 minutes after the post-job step started,
+      # it will terminate the tmate session and quit gracefully.
+      detached: true
+
+  - name: Validate test agent data
+    if: ${{ !cancelled() }}
+    run: ruby .github/scripts/test_agent_check.rb
+    shell: bash
+
+  - name: Upload junit reports
+    if: ${{ !cancelled() }}
+    uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
+    with:
+      name: junit-${{ inputs.alias }}-${{ inputs.container-id }}
+      path: tmp/rspec/*.xml
+
+  - name: Upload coverage data
+    uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
+    with:
+      name: coverage-${{ inputs.alias }}-${{ inputs.container-id }}
+      path: coverage
+      include-hidden-files: true  # Coverage data generated by SimpleCov are hidden
diff --git a/.github/actions/bundle-restore/action.yml b/.github/actions/bundle-restore/action.yml
@@ -0,0 +1,30 @@
+name: 'Restore Bundle from cache'
+
+# TODO: Improve description
+description: 'Restore the bundle from cache'
+
+inputs:
+  lockfile-name:
+    description: 'Name of the lockfile artifact to download'
+    required: true
+  cache-key:
+    description: 'Cache key for bundle'
+    required: true
+
+runs:
+  using: composite
+  steps:
+  - name: Download lockfile
+    uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
+    with:
+      name: ${{ inputs.lockfile-name }}
+
+  - name: Restore cache
+    uses: actions/cache/restore@1bd1e32a3bdc45362d1e726936510720a7c30a57 # v4.2.0
+    id: restore-cache
+    with:
+      key: "${{ inputs.cache-key }}"
+      path: "/usr/local/bundle"
+
+  - run: bundle check
+    shell: bash
diff --git a/.github/forced-tests-list.json b/.github/forced-tests-list.json
@@ -1,13 +1,3 @@
 {
-  "AGENT_NOT_SUPPORTING_SPAN_EVENTS":
-  [
-    "tests/test_span_events.py"
-  ],
-  "PARAMETRIC":
-  [
-    "tests/parametric/test_span_events.py"
-  ],
-  "DEFAULT": [
-    "tests/test_graphql.py"
-  ]
+
 }
diff --git a/.github/workflows/_unit_test.yml b/.github/workflows/_unit_test.yml
@@ -27,6 +27,7 @@ jobs:
     name: batch
     outputs:
       batches: "${{ steps.set-batches.outputs.batches }}"
+      misc: "${{ steps.set-batches.outputs.misc }}"
       cache-key: "${{ steps.restore-cache.outputs.cache-primary-key }}"
       lockfile: "${{ steps.lockfile.outputs.lockfile }}"
     container:
@@ -60,18 +61,47 @@ jobs:
     - id: set-batches
       name: Distribute tasks into batches
       run: |
-        batches_json=$(bundle exec rake github:generate_batches)
-        echo "$batches_json" | ruby -rjson -e 'puts JSON.pretty_generate(JSON.parse(STDIN.read))'
-        echo "batches=$batches_json" >> $GITHUB_OUTPUT
+        data=$(bundle exec rake github:generate_batches)
+        echo "$data" | ruby -rjson -e 'puts JSON.pretty_generate(JSON.parse(STDIN.read))'
+
+        # Extract each key and set it as a separate output
+        batches_data=$(echo "$data" | ruby -rjson -e 'puts JSON.parse(STDIN.read)["batches"].to_json')
+        misc_data=$(echo "$data" | ruby -rjson -e 'puts JSON.parse(STDIN.read)["misc"].to_json')
+
+        echo "batches=$batches_data" >> $GITHUB_OUTPUT
+        echo "misc=$misc_data" >> $GITHUB_OUTPUT
     - name: Generate batch summary
       run: bundle exec rake github:generate_batch_summary
       env:
         batches_json: "${{ steps.set-batches.outputs.batches }}"
-  build-test:
+
+  # `Initialize containers` step becomes quite heavily when many services are used.
+  #
+  # The job can failed with timeout because it takes a long time pulling the image or waiting for the service to be ready).
+  #
+  # `build-test-standard` job is used to run most of our tasks and configured with the following services:
+  #   - mysql
+  #   - postgres
+  #   - redis
+  #
+  # `build-test-misc` job is extracted to run specific tasks that requires the following services:
+  #   - elasticsearch
+  #   - memcached
+  #   - mongodb
+  #   - opensearch
+  #   - presto
+  #
+  #  Benefit of this optimization:
+  #  - Unnecessary services are not started
+  #  - Reduce the overhead for pulling images
+  #  - Improve reliability for starting dependent services
+  #
+  # In the future, we could consider to extract jobs to run database dependent tasks to optimize the workflow.
+  build-test-standard:
     needs:
     - batch
     runs-on: ubuntu-24.04
-    name: build & test [${{ matrix.batch }}]
+    name: build & test (standard) [${{ matrix.batch }}]
     timeout-minutes: 30
     env:
       BATCHED_TASKS: "${{ toJSON(matrix.tasks) }}"
@@ -86,13 +116,7 @@ jobs:
         DD_REMOTE_CONFIGURATION_ENABLED: 'false'
         TEST_POSTGRES_HOST: postgres
         TEST_REDIS_HOST: redis
-        TEST_ELASTICSEARCH_HOST: elasticsearch
-        TEST_MEMCACHED_HOST: memcached
-        TEST_MONGODB_HOST: mongodb
         TEST_MYSQL_HOST: mysql
-        TEST_OPENSEARCH_HOST: opensearch
-        TEST_OPENSEARCH_PORT: '9200'
-        TEST_PRESTO_HOST: presto
         DD_AGENT_HOST: agent
         DD_TRACE_AGENT_PORT: '9126'
         DATADOG_GEM_CI: 'true'
@@ -108,6 +132,12 @@ jobs:
           DD_POOL_TRACE_CHECK_FAILURES: 'true'
           DD_DISABLE_ERROR_RESPONSES: 'true'
           ENABLED_CHECKS: trace_content_length,trace_stall,meta_tracer_version_header,trace_count_header,trace_peer_service,trace_dd_service
+      mysql:
+        image: ghcr.io/datadog/images-rb/services/mysql:8.0
+        env:
+          MYSQL_ROOT_PASSWORD: root
+          MYSQL_PASSWORD: mysql
+          MYSQL_USER: mysql
       postgres:
         image: ghcr.io/datadog/images-rb/services/postgres:9.6
         env:
@@ -116,6 +146,57 @@ jobs:
           POSTGRES_DB: postgres
       redis:
         image: ghcr.io/datadog/images-rb/services/redis:6.2
+    steps:
+    - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
+    - name: Restore bundle cache
+      uses: ./.github/actions/bundle-restore
+      with:
+        lockfile-name: ${{ needs.batch.outputs.lockfile }}
+        cache-key: ${{ needs.batch.outputs.cache-key }}
+    - name: Build & Test
+      uses: ./.github/actions/build-test
+      with:
+        alias: ${{ inputs.alias }}
+        container-id: "standard-${{ matrix.batch }}"
+
+  build-test-misc:
+    needs:
+    - batch
+    runs-on: ubuntu-24.04
+    name: build & test (misc) [${{ matrix.batch }}]
+    timeout-minutes: 30
+    env:
+      BATCHED_TASKS: "${{ toJSON(matrix.tasks) }}"
+    strategy:
+      fail-fast: false
+      matrix:
+        include: "${{ fromJson(needs.batch.outputs.misc).include }}"
+    container:
+      image: ghcr.io/datadog/images-rb/engines/${{ inputs.engine }}:${{ inputs.version }}
+      env:
+        DD_INSTRUMENTATION_TELEMETRY_ENABLED: 'false'
+        DD_REMOTE_CONFIGURATION_ENABLED: 'false'
+        TEST_ELASTICSEARCH_HOST: elasticsearch
+        TEST_MEMCACHED_HOST: memcached
+        TEST_MONGODB_HOST: mongodb
+        TEST_OPENSEARCH_HOST: opensearch
+        TEST_OPENSEARCH_PORT: '9200'
+        TEST_PRESTO_HOST: presto
+        DD_AGENT_HOST: agent
+        DD_TRACE_AGENT_PORT: '9126'
+        DATADOG_GEM_CI: 'true'
+        TEST_DATADOG_INTEGRATION: '1'
+        JRUBY_OPTS: "--dev" # Faster JVM startup: https://github.com/jruby/jruby/wiki/Improving-startup-time#use-the---dev-flag
+    services:
+      agent:
+        image: ghcr.io/datadog/dd-apm-test-agent/ddapm-test-agent:v1.18.0
+        env:
+          LOG_LEVEL: DEBUG
+          TRACE_LANGUAGE: ruby
+          PORT: '9126'
+          DD_POOL_TRACE_CHECK_FAILURES: 'true'
+          DD_DISABLE_ERROR_RESPONSES: 'true'
+          ENABLED_CHECKS: trace_content_length,trace_stall,meta_tracer_version_header,trace_count_header,trace_peer_service,trace_dd_service
       elasticsearch:
         image: ghcr.io/datadog/images-rb/services/elasticsearch:8.1.3
         env:
@@ -138,54 +219,15 @@ jobs:
           cluster.routing.allocation.disk.threshold_enabled: 'false'
       presto:
         image: ghcr.io/datadog/images-rb/services/starburstdata/presto:332-e.9
-      mysql:
-        image: ghcr.io/datadog/images-rb/services/mysql:8.0
-        env:
-          MYSQL_ROOT_PASSWORD: root
-          MYSQL_PASSWORD: mysql
-          MYSQL_USER: mysql
     steps:
     - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
-    - name: Configure Git
-      run: git config --global --add safe.directory "$GITHUB_WORKSPACE"
-    - name: Download lockfile
-      uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
+    - name: Restore bundle cache
+      uses: ./.github/actions/bundle-restore
       with:
-        name: lockfile-${{ inputs.alias }}-${{ github.run_id }}
-    - name: Restore cache
-      uses: actions/cache/restore@1bd1e32a3bdc45362d1e726936510720a7c30a57 # v4.2.0
-      id: restore-cache
-      with:
-        key: "${{ needs.batch.outputs.cache-key }}"
-        path: "/usr/local/bundle"
-    - run: bundle check || bundle install
-    - run: bundle exec rake github:run_batch_build
-    - name: Configure RSpec
-      run: ln -s .rspec-local.example .rspec-local
-    - run: bundle exec rake github:run_batch_tests
-      env:
-        COVERAGE_DIR: coverage/versions/${{ inputs.alias }}/${{ matrix.batch }}
-    - name: Debug with SSH connection
-      if: ${{ failure() && runner.debug == '1' }}
-      uses: mxschmitt/action-tmate@e5c7151931ca95bad1c6f4190c730ecf8c7dde48 # v3.19.0
-      with:
-        limit-access-to-actor: true
-        # This mode will wait at the end of the job for a user to connect and then to terminate the tmate session.
-        # If no user has connected within 10 minutes after the post-job step started,
-        # it will terminate the tmate session and quit gracefully.
-        detached: true
-    - name: Validate test agent data
-      if: ${{ !cancelled() }}
-      run: ruby .github/scripts/test_agent_check.rb
-    - name: Upload junit reports
-      if: ${{ !cancelled() }}
-      uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
-      with:
-        name: junit-${{ inputs.alias }}-${{ matrix.batch }}-${{ github.run_id }}
-        path: tmp/rspec/*.xml
-    - name: Upload coverage data
-      uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08 # v4.6.0
+        lockfile-name: ${{ needs.batch.outputs.lockfile }}
+        cache-key: ${{ needs.batch.outputs.cache-key }}
+    - name: Build & Test
+      uses: ./.github/actions/build-test
       with:
-        name: coverage-${{ inputs.alias }}-${{ matrix.batch }}-${{ github.run_id }}
-        path: coverage
-        include-hidden-files: true # Coverage data generated by SimpleCov are hidden
+        alias: ${{ inputs.alias }}
+        container-id: "misc-${{ matrix.batch }}"
diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml
@@ -126,6 +126,7 @@ jobs:
       - rubygems-release
     env:
       GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+      GH_REPO: ${{ github.repository }}
       GEM_VERSION: ${{ needs.verify-checks.outputs.version }}
     permissions:
       contents: write
@@ -153,6 +154,10 @@ jobs:
       - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
           fetch-depth: 0
+      - name: Set up Ruby
+        uses: ruby/setup-ruby@8388f20e6a9c43cd241131b678469a9f89579f37 # v1.216.0
+        with:
+          ruby-version: '3.3.7'
       - run: bundle install
       - id: next_version
         run: |
@@ -264,11 +269,16 @@ jobs:
         with:
           fetch-depth: 0
       - run: |
-          JOB_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}/jobs/${{ github.job }}"
+          JOB_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
+          TEMP_BRANCH="update-document-v${GEM_VERSION}"
+
+          # Create and checkout a new branch from the tag
+          git checkout -b "${TEMP_BRANCH}" "v${GEM_VERSION}"
+          git push origin "${TEMP_BRANCH}"
 
           gh pr create \
             --base release \
-            --head master \
+            --head "${TEMP_BRANCH}" \
             --title "Update document v${GEM_VERSION}" \
             --body "This is an auto-generated PR to update documentation from [here](${JOB_URL}). Please merge (with a merge commit) when ready." \
             --label "docs"
diff --git a/.github/workflows/test-memory-leaks.yaml b/.github/workflows/test-memory-leaks.yaml
@@ -24,9 +24,9 @@ jobs:
     runs-on: ubuntu-24.04
     steps:
       - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
-      - uses: ruby/setup-ruby@8388f20e6a9c43cd241131b678469a9f89579f37 # v1.216.0
+      - uses: ruby/setup-ruby@04c2e2b8e83fbca87e4c7ea9c16e82a1c029c074 # Adds 3.4-asan builds
         with:
-          ruby-version: asan
+          ruby-version: 3.4-asan
           bundler-cache: true # runs 'bundle install' and caches installed gems automatically
           bundler: latest
           cache-version: v2 # bump this to invalidate cache