Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[tests] - optimize trace testing #1659

Merged
merged 5 commits into from
Jul 8, 2024
Merged

[tests] - optimize trace testing #1659

merged 5 commits into from
Jul 8, 2024

Conversation

puckpuck
Copy link
Contributor

@puckpuck puckpuck commented Jul 4, 2024 •
edited
Loading

Changes

Significantly optimizes trace based testing going from 10 minutes to about 1 minute to run all tests. This was done by doing 3 main things:

  1. Run all tests concurrently instead of sequentially. This was the largest contributor to the improvements
  2. Remove Jaeger from testing and test directly from the OpenTelemetry collector. When I was working on this, I saw that the tests would fail periodically when Jaeger was used as a backend, particularly for the service. Moving to OpenTelemetry collector only for the backend resolved this
  3. Shorten the retry interval from 10s to 5s in the tracetest server provisioning.

This should help speed up going from an Approved PR to getting it merged, as waiting for a 10+ minute check to complete can lead to context switching.

Merge Requirements

For new features contributions please make sure you have completed the following
essential items:

  • CHANGELOG.md updated to document new feature additions
    * [ ] Appropriate documentation updates in the docs
    * [ ] Appropriate Helm chart updates in the helm-charts

Maintainers will not merge until the above have been completed. If you're unsure
which docs need to be changed ping the
@open-telemetry/demo-approvers.

@puckpuck puckpuck requested a review from a team July 4, 2024 12:57
@github-actions github-actions bot added the helm-update-required Requires an update to the Helm chart when released label Jul 4, 2024
@puckpuck puckpuck removed the helm-update-required Requires an update to the Helm chart when released label Jul 4, 2024
julianocosta89
julianocosta89 approved these changes Jul 8, 2024
View reviewed changes
Copy link
Member

@julianocosta89 julianocosta89 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great improvement! 🚀

@github-actions github-actions bot added the helm-update-required Requires an update to the Helm chart when released label Jul 8, 2024
rogercoll
rogercoll approved these changes Jul 8, 2024
View reviewed changes
Copy link
Contributor

@rogercoll rogercoll left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for decoupling the tests from the main compose file too

@puckpuck puckpuck merged commit 62d1d9b into open-telemetry:main Jul 8, 2024
30 checks passed
@puckpuck puckpuck deleted the tests.optimize-tracetest branch July 8, 2024 17:06
eliasmueller pushed a commit to UST-DeMAF/opentelemetry-demo that referenced this pull request Jul 14, 2024
@puckpuck @julianocosta89 @eliasmueller
[tests] - optimize trace testing (open-telemetry#1659)
c85f722 
* optimize trace testing

* run trace based tests concurrently

* fix lint failures

---------

Co-authored-by: Juliano Costa <julianocosta89@outlook.com>
ahealy-newr pushed a commit to ahealy-newr/opentelemetry-demo-ahealy that referenced this pull request Jul 24, 2024
@puckpuck @julianocosta89 @ahealy-newr
[tests] - optimize trace testing (open-telemetry#1659)
1dfe3d1 
* optimize trace testing

* run trace based tests concurrently

* fix lint failures

---------

Co-authored-by: Juliano Costa <julianocosta89@outlook.com>
CharlieTLe added a commit to CharlieTLe/opentelemetry.io that referenced this pull request Sep 8, 2024
@CharlieTLe
Update docker-deployment.md
a7755bc 
Optimizations on trace-based testing in open-telemetry/opentelemetry-demo#1659 changed how tracetest is enabled. This change updates the documentation to reflect on enabling trace based tests should be done.
@CharlieTLe CharlieTLe mentioned this pull request Sep 8, 2024
Closed
eliasmueller added a commit to UST-DeMAF/opentelemetry-demo that referenced this pull request Oct 3, 2024
@eliasmueller @heidrifx
@dependabot @mviitane @rogercoll @puckpuck @SvenKirschbaum @flands @RassK @julianocosta89 @beeme1mr @Arikuma97 @schurpl @365Bit
feat(EnPRO): extend OTel-shop with MongoDB, add Ansible, Terraform an…
a93fbb9 
…d EDMM models (#9)

* [productcatalogservice] Added MongoDB

This adds a MongoDB to the demo instead of parsing the JSON before usage. The implementation is not yet perfect but it works for now. Thus far, only docker compose is supported.

Things that still need more work are:
- The Go application still uses static MongoDB username and password and the hostname and port are also static. This should be changed to use environmental variables instead.
- I wasn't able to check whether the healthcheck of the MongoDB works or not.
- Is the resource limit too generous?
- Should we fix the port inside the MongoDB container to its default value (i.e. change using a secret here to 27017)?

* [productcatalogservice] Changed to use env vars

This changes the productcatalogservice to use environmental variables for the MongoDB username, password, hostname and port.

* [K8s] Added MongoDB

*Disclaimer:* this is not working yet!

* build(deps): bump docker/build-push-action from 6.0.1 to 6.1.0 (open-telemetry#1620)

Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.0.1 to 6.1.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@v6.0.1...v6.1.0)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mikko Viitanen <74129181+mviitane@users.noreply.github.com>

* fix: emailservice docker compose image (open-telemetry#1630)

Co-authored-by: Pierre Tessier <pierre@pierretessier.com>

* add collector configs as environment variable (open-telemetry#1632)

Co-authored-by: Mikko Viitanen <74129181+mviitane@users.noreply.github.com>

* fix(frontend): Use port 443 as default to connect to flagd if https is in use (open-telemetry#1609)

* fix(frontend): Use port 443 as default to connect to flagd if https is in use

* chore: Update CHANGELOG.md

---------

Co-authored-by: Pierre Tessier <pierre@pierretessier.com>

* Update recommendation flag to match flagd configuration (open-telemetry#1634)

Address open-telemetry#1626

* Change AccountService from Go to DotNet (auto) (open-telemetry#1538)

* Change AccountService from go to dotnet (auto)

* fix path

* fix folder name

* dockerfile and other fixes

* add copyright

* fix encoding and cleanup

* Cleanup dockerfile

* Update OTel Auto

* fix kafka processing issues and otel export

* remove eof

* Update changelog

* Use default CancellationDelayMaxMs

* update packages

* fix merge failure

* Fix tracetest 'accountingservice' is not part of the trace anymore

---------

Co-authored-by: Juliano Costa <juliano.costa@datadoghq.com>
Co-authored-by: Juliano Costa <julianocosta89@outlook.com>

* build(deps): bump docker/build-push-action from 6.1.0 to 6.2.0 (open-telemetry#1636)

Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.1.0 to 6.2.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@v6.1.0...v6.2.0)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Pierre Tessier <pierre@pierretessier.com>

* [chore] clarify complete release process (open-telemetry#1638)

* clarify complete release process

* clarify complete release process

* clarify complete release process

* clarify complete release process

* clarify complete release process

* update to 1.11.0 release (open-telemetry#1639)

* update to 1.11.0 release (open-telemetry#1640)

* [chore] update demo role memberships (open-telemetry#1649)

* update demo role membership

* update demo role membership

* [otel-col] Add docker stats receiver (open-telemetry#1650)

* Add docker stats receiver

* changelog

* Nit: prevent write access to the Docker socket

Co-authored-by: Roger Coll <roger.coll@elastic.co>

* Add compose minimal

---------

Co-authored-by: Roger Coll <roger.coll@elastic.co>

* add Kafka Dockerfile env variable (open-telemetry#1652)

Co-authored-by: Pierre Tessier <pierre@pierretessier.com>

* build(deps): bump docker/build-push-action from 6.2.0 to 6.3.0 (open-telemetry#1656)

Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6.2.0 to 6.3.0.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](docker/build-push-action@v6.2.0...v6.3.0)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Simplify error scenario logic (open-telemetry#1657)

* remove fractional config from rule definition

Signed-off-by: Michael Beemer <beeme1mr@users.noreply.github.com>

* add random failure logic to ad service

Signed-off-by: Michael Beemer <beeme1mr@users.noreply.github.com>

* add random failure logic to cart service

Signed-off-by: Michael Beemer <beeme1mr@users.noreply.github.com>

---------

Signed-off-by: Michael Beemer <beeme1mr@users.noreply.github.com>

* [tests] - optimize trace testing (open-telemetry#1659)

* optimize trace testing

* run trace based tests concurrently

* fix lint failures

---------

Co-authored-by: Juliano Costa <julianocosta89@outlook.com>

* [chore] fix build-images workflow (open-telemetry#1661)

* fix build-images workflow

* use consistent naming for workflows

* Set OTLP receiver endpoint (open-telemetry#1662)

* Set OTelCol receiver endpoint

* changelog

* Apply suggestions from code review

Co-authored-by: Roger Coll <roger.coll@elastic.co>

* Add env vars to collector container

---------

Co-authored-by: Roger Coll <roger.coll@elastic.co>

* feat: [K8s] Added MongoDB

*Disclaimer:* this is not working yet!

* feat: ES-9 add EDMM Model (K8s) of otel-shop

* feat: ES-39 add Terrfaform Model of otel-shop

* feat: ES-44 add Ansible Model of otel-shop

* fixed MongoDB not working and removed duplicate MongoDB service

* feat: ES-9 add some refinement and connects_to

* Update otel_store_k8s_translated.yaml

YAML style adjusted to a uniform format.

* Update otel_store_k8s_translated.yaml

YAML style adjusted to a uniform format. Adjusted, of relation_types and components_types (underscore was missing) --> Inconsistency

* Terraform model: Deletetd testing services and fixed image declaration of the email service

* feat: ES-32 minor fix and todo

* Adds missing component types and fixes relations

* fix: ES-56 fix some missing stuff in EDMM

* fix: ES-56 fix some missing stuff in EDMM

* fix: ES-56 fix moore

* fix: ES-56 fix more

* fix: ES-56 lint

* fix: ES-56 add opensearch hosted on

* fix: ES-56 remove duplicate

* fix: ES-56 fix naming

* Added health checks where applicable

some health checks are rather dirty but they work

* fix: auto-formatted K8s file

this enables us to finally parse this example with our plugin!

* chore(ES-80): move and cleanup main.tf (#8)

* chore: ES-80 move .env to k8s folder and cleanup main.tf

* chore: ES-80 remove .env

* ES-59 Fixed main.tf and added linux support

* fix: fix merge issues

* chore(ES-86): keep models up-to-date and add target models for all technologies (#10)

* chore: update versions and memory

* chore: some more adaptions

* chore: add target edmm models

* ci: test build job for productcatalogservice

* ci: try this tag

* ci: now try this tag

* ci: try enhance tagging

* chore: make use of pc-service build, readme, switch to concrete versioning in terraform

* chore: remove branch build

* docs: finalize readme

* chore: mr adaptions

* fix: fix typos from generator

* fix: remove submodules

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Michael Beemer <beeme1mr@users.noreply.github.com>
Co-authored-by: Felix <st157773@stud.uni-stuttgart.de>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Mikko Viitanen <74129181+mviitane@users.noreply.github.com>
Co-authored-by: Roger Coll <rogercoll@protonmail.com>
Co-authored-by: Pierre Tessier <pierre@pierretessier.com>
Co-authored-by: Sven Kirschbaum <sven@kirschbaum.me>
Co-authored-by: Steve Flanders <sflanders@splunk.com>
Co-authored-by: Rasmus Kuusmann <rkuusmann@splunk.com>
Co-authored-by: Juliano Costa <juliano.costa@datadoghq.com>
Co-authored-by: Juliano Costa <julianocosta89@outlook.com>
Co-authored-by: Roger Coll <roger.coll@elastic.co>
Co-authored-by: Michael Beemer <beeme1mr@users.noreply.github.com>
Co-authored-by: Arikuma97 <44363150+Arikuma97@users.noreply.github.com>
Co-authored-by: Pascal Schur <pascal@uli-schur.de>
Co-authored-by: Rene Tischler <20995752+365Bit@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cedricziel cedricziel cedricziel approved these changes

@julianocosta89 julianocosta89 julianocosta89 approved these changes

@rogercoll rogercoll rogercoll approved these changes

Assignees
No one assigned
Labels
helm-update-required Requires an update to the Helm chart when released
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@puckpuck @cedricziel @julianocosta89 @rogercoll