Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Transition to Native Multi-Platform Builds for Docker Images #161

Merged
merged 5 commits into from
Mar 4, 2025
Merged

Transition to Native Multi-Platform Builds for Docker Images #161

merged 5 commits into from
Mar 4, 2025

Conversation

amansinghoriginal
Copy link
Contributor

@amansinghoriginal amansinghoriginal commented Feb 28, 2025 •
edited
Loading

Description

This pull request updates our release workflow by moving away from QEMU emulation for multi-platform Docker image builds to using native GitHub runners for linux/amd64 and linux/arm64 architectures. The goal is to enhance build stability and performance while keeping the workflow’s functionality intact.

This change should reduce the build time from 2-3 hours down to < 15 minutes.

Details

  1. Switch to Native Runners
  • Previously, multi-platform builds relied on QEMU emulation. Now, we use:
    • ubuntu-latest runners for linux/amd64.
    • ubuntu-24.04-arm runners for linux/arm64.
  • This removes emulation overhead, leading to fewer build failures and faster execution.
  1. Platform-Specific Image Tagging
  • Docker images are now built and tagged individually per architecture:
    • Example: :-amd64 for linux/amd64.
    • Example: :-arm64 for linux/arm64.
  • A new manifest creation step combines these into a single multi-platform image (<image>:<tag>)
  • Therefore, users should be able to pull images seamlessly without needing to specify the architecture.

Type of change

  • This pull request fixes a bug in Drasi and has an approved issue (issue link required).
  • This pull request adds or changes features of Drasi and has an approved issue (issue link required).
  • This pull request is a minor refactor, code cleanup, test improvement, or other maintenance task and doesn't change the functionality of Drasi (issue link optional).

@amansinghoriginal amansinghoriginal marked this pull request as ready for review February 28, 2025 07:01
@amansinghoriginal amansinghoriginal requested a review from a team as a code owner February 28, 2025 07:01
@amansinghoriginal amansinghoriginal self-assigned this Feb 28, 2025
@amansinghoriginal amansinghoriginal force-pushed the main branch 2 times, most recently from aa50d52 to 2c0f707 Compare March 1, 2025 06:39
@amansinghoriginal amansinghoriginal force-pushed the main branch 2 times, most recently from 6262edc to a09b8c0 Compare March 3, 2025 18:36
@amansinghoriginal amansinghoriginal changed the title Fix Build Issues - Use separate ARM runner Transition to Native Multi-Platform Builds for Docker Images Mar 3, 2025
ruokun-niu
ruokun-niu reviewed Mar 3, 2025
View reviewed changes
.github/workflows/draft-release.yml Outdated
uses: docker/setup-qemu-action@49b3bc8e6bdd4a60e6116a5414239cba5943d3cf # v3.2.0
with:
submodules: true
token: ${{ secrets.DRASI_CORE_PAT }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we only need this token when we are trying to build the query container

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed, thanks!

ruokun-niu
ruokun-niu reviewed Mar 3, 2025
View reviewed changes
.github/workflows/draft-release.yml
{"label": "Kubernetes Reactivator", "path": "sources/kubernetes/kubernetes-reactivator", "name": "source-kubernetes-reactivator", "platforms": "linux/amd64,linux/arm64"},
{"label": "Kubernetes Proxy", "path": "sources/kubernetes/kubernetes-proxy", "name": "source-kubernetes-proxy", "platforms": "linux/amd64,linux/arm64"}]'

REACTIONS_COMPONENTS: '[
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I forgot to add the eventbridge reaction when I merged in that PR, can you add it in this component list

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added

@amansinghoriginal amansinghoriginal merged commit d343738 into drasi-project:main Mar 4, 2025
32 checks passed
ruokun-niu added a commit to ruokun-niu/drasi-platform that referenced this pull request Mar 4, 2025
@ruokun-niu
Merge branch 'review-debug-reaction' of https://github.com/ruokun-niu…
9f48ef4 
…/drasi-platform into review-debug-reaction

* 'review-debug-reaction' of https://github.com/ruokun-niu/drasi-platform:
  Transition to Native Multi-Platform Builds for Docker Images (drasi-project#161)
  Bump jinja2 from 3.1.4 to 3.1.5 in /reactions/sdk/python (drasi-project#135)
  Bump vite in /reactions/signalr/signalr-reaction/clients/vue (drasi-project#142)
  Bump nanoid in /reactions/signalr/signalr-reaction/clients/react/example (drasi-project#131)
  Bump openssl from 0.10.66 to 0.10.71 in /query-container (drasi-project#160)
  Bump golang.org/x/crypto from 0.26.0 to 0.31.0 in /cli (drasi-project#132)
  Bump nanoid in /reactions/signalr/signalr-reaction/clients/vue/example (drasi-project#130)
  Bump openssl from 0.10.68 to 0.10.71 in /sources/sdk/rust/example/proxy (drasi-project#158)
  Bump io.undertow:undertow-core from 2.3.15.Final to 2.3.17.Final in /sources/sdk/java (drasi-project#127)
  Bump esbuild from 0.16.17 to 0.25.0 in /dev-tools/vscode/drasi (drasi-project#151)
  Bump openssl from 0.10.66 to 0.10.70 in /control-planes/mgmt_api (drasi-project#148)
  Remove Fixed Dapr Version Constraint (drasi-project#147)
  Gremlin Reaction hotfix (drasi-project#149)
ruokun-niu added a commit to ruokun-niu/drasi-platform that referenced this pull request Mar 4, 2025
@ruokun-niu
Merge branch 'debug-base-images' of https://github.com/ruokun-niu/dra…
f1223f3 
…si-platform into debug-base-images

* 'debug-base-images' of https://github.com/ruokun-niu/drasi-platform:
  Transition to Native Multi-Platform Builds for Docker Images (drasi-project#161)
  Bump jinja2 from 3.1.4 to 3.1.5 in /reactions/sdk/python (drasi-project#135)
  Bump vite in /reactions/signalr/signalr-reaction/clients/vue (drasi-project#142)
  Bump nanoid in /reactions/signalr/signalr-reaction/clients/react/example (drasi-project#131)
  Bump openssl from 0.10.66 to 0.10.71 in /query-container (drasi-project#160)
  Bump golang.org/x/crypto from 0.26.0 to 0.31.0 in /cli (drasi-project#132)
  Bump nanoid in /reactions/signalr/signalr-reaction/clients/vue/example (drasi-project#130)
  Bump openssl from 0.10.68 to 0.10.71 in /sources/sdk/rust/example/proxy (drasi-project#158)
  Bump io.undertow:undertow-core from 2.3.15.Final to 2.3.17.Final in /sources/sdk/java (drasi-project#127)
  Bump esbuild from 0.16.17 to 0.25.0 in /dev-tools/vscode/drasi (drasi-project#151)
  Bump openssl from 0.10.66 to 0.10.70 in /control-planes/mgmt_api (drasi-project#148)
  Remove Fixed Dapr Version Constraint (drasi-project#147)
  Gremlin Reaction hotfix (drasi-project#149)
  EventHub Source (drasi-project#152)
  Configured the EventGrid Reaction to use different schemas (drasi-project#138)
  AWS IRSA support and Eventbridge reaction (drasi-project#140)
  init logging (drasi-project#153)
  Dotnet source sdk (drasi-project#144)
  Refactor Relational Source and add MySQL Support. (drasi-project#146)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@danielgerlag danielgerlag danielgerlag approved these changes

@ruokun-niu ruokun-niu ruokun-niu approved these changes

Assignees

@amansinghoriginal amansinghoriginal

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@amansinghoriginal @danielgerlag @ruokun-niu