Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use smaller instances for AWS E2E tests #2415

Merged
merged 3 commits into from
Oct 24, 2022
Merged

Use smaller instances for AWS E2E tests #2415

merged 3 commits into from
Oct 24, 2022

Conversation

xmudrii
Copy link
Member

@xmudrii xmudrii commented Oct 21, 2022 •
edited
Loading

What this PR does / why we need it:

This PR introduces the following changes to the AWS E2E tests:

  • Run E2E tests in the Ireland region (eu-west-1)
    • This is the cheapest Europen region available
  • Use t3a.small for the control plane and worker nodes
    • This instance size provides 2 vCPUs and 2 GB RAM which is enough for running E2E tests
    • t3a instances are slightly slower than t3 instances, but they're also cheaper. This doesn't seem to affect the speed of E2E tests at all
  • Use t3a.micro instances for worker nodes in the upgrade tests
    • We don't run any conformance tests when running upgrade tests, so we don't need big machines
  • Use spot instances for the worker nodes
  • Use 25 GB disks (50 GB for RHEL) instead of 100 GB for the control plane and 50 GB for worker nodes
  • Use t3a.nano (t3.micro for RHEL) instead of t3.nano for the bastion instance

Old setup:

  • Paris (eu-west-3)
  • t3.medium and t3.nano instances
  • Total per run/hour assumes running 4 AWS E2E tests for 1 hour

Disclaimer: the following table doesn't take into account other cloud resources.

Instance size Price Amount Total per job Total per run/hour
t3.medium $0.0472 6 $0.2832 $1.1328
t3.nano $0.0059 1 $0.0059 $0.0236
$0.2891 $1.1564

New setup:

  • Ireland (eu-west-1)
  • t3a.small and t3a.nano instances
  • Total per run/hour assumes running 4 AWS E2E tests for 1 hour

Disclaimer: the following table doesn't take into account other cloud resources and using spot instances for the worker nodes.

Instance size Price Amount Total per job Total per run/hour
t3a.small $0.0204 6 $0.1224 $0.4896
t3a.nano $0.0051 1 $0.0051 $0.0204
$0.1275 $0.51

Savings per job/hour: $0,1616
Savings per run/hour: $0,6464

Considered:

  • Using spot instances for the control plane nodes
    • Requires significant changes to the Terraform configs
  • Using even smaller instances for worker nodes
    • Tested t3.micro, but it only has 1 GB RAM which is not enough for running the conformance tests. All tests were failing.

What type of PR is this?

/kind chore

Does this PR introduce a user-facing change? Then add your Release Note here:

Make volume size for worker nodes configurable in Terraform configs for AWS (50 GB by default)

Documentation:

NONE

@kubermatic-bot kubermatic-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. docs/none Denotes a PR that doesn't need documentation (changes). kind/chore Updating grunt tasks etc; no production code changes. dco-signoff: yes Denotes that all commits in the pull request have the valid DCO signoff message. labels Oct 21, 2022
@xmudrii xmudrii changed the title Use smaller instances for AWS E2E tests WIP: Use smaller instances for AWS E2E tests Oct 21, 2022
@kubermatic-bot kubermatic-bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 21, 2022
@xmudrii
Copy link
Member Author

xmudrii commented Oct 21, 2022

/retest

@xmudrii
Make worker volume size configurable in AWS configs
870cc47 
Signed-off-by: Marko Mudrinić <mudrinic.mare@gmail.com>
@xmudrii
Copy link
Member Author

xmudrii commented Oct 22, 2022

/test pull-kubeone-e2e-aws-default-stable-upgrade-containerd-from-v1.23.13-to-v1.24.7

@xmudrii
Use smaller instances for AWS E2E tests
14c0fd2 
Signed-off-by: Marko Mudrinić <mudrinic.mare@gmail.com>
@xmudrii
Copy link
Member Author

xmudrii commented Oct 22, 2022

/test pull-kubeone-e2e-aws-default-stable-upgrade-containerd-from-v1.23.13-to-v1.24.7

1 similar comment
@xmudrii
Copy link
Member Author

xmudrii commented Oct 22, 2022

/test pull-kubeone-e2e-aws-default-stable-upgrade-containerd-from-v1.23.13-to-v1.24.7

@xmudrii
Copy link
Member Author

xmudrii commented Oct 22, 2022

/retest

1 similar comment
@xmudrii
Copy link
Member Author

xmudrii commented Oct 22, 2022

/retest

@xmudrii xmudrii changed the title WIP: Use smaller instances for AWS E2E tests Use smaller instances for AWS E2E tests Oct 22, 2022
@kubermatic-bot kubermatic-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 22, 2022
@xmudrii xmudrii requested a review from kron4eg October 22, 2022 15:38
@xmudrii xmudrii requested a review from kron4eg October 23, 2022 10:23
@xmudrii
Use varFile for AWS infra definitions
ee48d5d 
Signed-off-by: Marko Mudrinić <mudrinic.mare@gmail.com>
@xmudrii xmudrii requested a review from kron4eg October 23, 2022 15:06
ahmedwaleedmalik
ahmedwaleedmalik approved these changes Oct 24, 2022
View reviewed changes
Copy link
Member

@ahmedwaleedmalik ahmedwaleedmalik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@kubermatic-bot kubermatic-bot added the lgtm Indicates that a PR is ready to be merged. label Oct 24, 2022
@kubermatic-bot
Copy link
Contributor

LGTM label has been added.

Git tree hash: b7409c9b46029ad1468f0d2940ef4f724ebeb1b7

@kubermatic-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahmedwaleedmalik

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubermatic-bot kubermatic-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 24, 2022
@xmudrii
Copy link
Member Author

xmudrii commented Oct 24, 2022

/retest

1 similar comment
@xmudrii
Copy link
Member Author

xmudrii commented Oct 24, 2022

/retest

@xmudrii xmudrii added the backport-needed Denotes a PR or issue that has not been fully backported. label Oct 24, 2022
@kubermatic-bot kubermatic-bot merged commit f2e7b64 into main Oct 24, 2022
@kubermatic-bot kubermatic-bot added this to the KubeOne 1.6 milestone Oct 24, 2022
@kubermatic-bot kubermatic-bot deleted the e2e-aws-instance-size branch October 24, 2022 11:12
@xmudrii xmudrii mentioned this pull request Oct 26, 2022
Merged
@xmudrii xmudrii mentioned this pull request Nov 7, 2022
Merged
@xmudrii xmudrii added backport-complete Denotes a PR or issue which has been fully backported to all required release branches. and removed backport-needed Denotes a PR or issue that has not been fully backported. labels Nov 7, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ahmedwaleedmalik ahmedwaleedmalik ahmedwaleedmalik approved these changes

@kron4eg kron4eg Awaiting requested review from kron4eg

Assignees

@ahmedwaleedmalik ahmedwaleedmalik

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. backport-complete Denotes a PR or issue which has been fully backported to all required release branches. dco-signoff: yes Denotes that all commits in the pull request have the valid DCO signoff message. docs/none Denotes a PR that doesn't need documentation (changes). kind/chore Updating grunt tasks etc; no production code changes. lgtm Indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
KubeOne 1.6
Development

Successfully merging this pull request may close these issues.
4 participants
@xmudrii @kubermatic-bot @kron4eg @ahmedwaleedmalik