Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: Latest helm-loki release (6.7.2) not installable. #13633

Closed
Tommyf opened this issue Jul 23, 2024 · 15 comments
Closed

Bug: Latest helm-loki release (6.7.2) not installable. #13633

Tommyf opened this issue Jul 23, 2024 · 15 comments

Comments

@Tommyf
Copy link

Tommyf commented Jul 23, 2024

The latest release of the loki helm chart (helm-loki-6.7.2) is not usable as it is too large.

As this is a 'patch' level version bump, many environments will automatically pick up this new version (as it's meant to be a non-breaking change) and then failure will occur.

Steps to reproduce the behavior
Use Helm to install or update Loki using version 6.7.2 of the Helm Chart:
helm install releasename -f myvalues.yaml --version 6.7.2 grafana/loki
or
helm update releasename -f myvalues.yaml --version 6.7.2 grafana/loki

The above results in the following error as the Helm Chart is too big to fit inside a Kubernetes Secret:
Error: INSTALLATION FAILED: create: failed to create: Request entity too large: limit is 3145728

Expected behavior
The command should complete without error.

Cause of behaviour
Upon investigation, I see that a large binary file has been added to the Helm Chart in commit bd1d6a7a.
This looks like it was causing tests to fail and was mentioned in #13461.
It seems that the changes were then merged despite the failing test.

This large file will be the reason for the tests failing in that PR.
I see no indication in the commit as to why the file was added, so maybe it was a mistake? Removing it should fix the failing tests and also make the Helm Chart usable again :)

buroa added a commit to buroa/k8s-gitops that referenced this issue Jul 24, 2024
budimanjojo added a commit to budimanjojo/home-cluster that referenced this issue Jul 24, 2024
@fculpo
Copy link

fculpo commented Jul 24, 2024

Yes, same here

@jensens
Copy link

jensens commented Jul 24, 2024

I can confirm this too.

@toalba
Copy link

toalba commented Jul 24, 2024

Encountered the same issue.

@stefcho034
Copy link

Same here...

@QuentinBisson
Copy link
Contributor

@Jayclifford345 maybe you can help on this one?

@bpedersen2
Copy link

Also encountered this problem in a SuSE-rancher based installation.

Fix: cut down the values yaml (in my case: delete minio and enterprise configs) to keep the values.yml < 32kBytes.

Note: per default rancher passes the full values.yaml from the chart , but you actually would need only the things that deviate from the default.

@brainfair
Copy link

brainfair commented Jul 24, 2024

@bpedersen2 there is nothing to deal with values, the core issue is with helm-test 13mb binary inside src/helm-test directory due to which the size of the chart becomes larger than what is allowed for keeping a helm release secret

possible fixes:

  • remove helm-test binary
  • add helm-test directory or binary to helmignore

@elchenberg
Copy link

elchenberg commented Jul 24, 2024

I guess the helm-test binary was added to the repository by @Jayclifford345 and @JStickler by accident: bd1d6a7

@bpedersen2
Copy link

@bpedersen2 there is nothing to deal with values, the core issue is with helm-test 13mb binary inside src/helm-test directory due to which the size of the chart becomes larger than what is allowed for keeping a helm release secret

possible fixes:

  • remove helm-test binary
  • add helm-test directory or binary to helmignore

But why does then just trimming down the values.yml help in a successful install?

@Jayclifford345
Copy link
Contributor

Jayclifford345 commented Jul 24, 2024

Thank you for pointing this out all just working with the Loki team to sort this out. My apologies for causing this issue with my last PR. Fixing this with hotfix

@Jayclifford345
Copy link
Contributor

Just waiting for the last of the tests to pass and will get this merged: #13651

@Jayclifford345
Copy link
Contributor

That is the hotfix merged and should be now fixed. Let me know if you are still running into issues

@Tommyf
Copy link
Author

Tommyf commented Jul 25, 2024

Release 6.7.3 looks good!

Thanks for addressing this so quickly! It was not a huge blocker as one could easily pin to a known good version (as you really should be doing anyway), but many people do use unpinned latest releases so this kind of thing catches them (me!) out :)

@kosmoz
Copy link

kosmoz commented Jul 25, 2024

I can confirm that 6.7.3 works!

@JStickler
Copy link
Contributor

@Tommyf so can we close this issue?

@chaudum chaudum closed this as completed Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests