Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] running the demo start flyte app crashes due to permission error on apple's M4 #6131

Open
2 tasks done
norbit8 opened this issue Jan 1, 2025 · 4 comments
Open
2 tasks done
Assignees
Labels
bug Something isn't working waiting for reporter Used for when we need input from the bug reporter

Comments

@norbit8
Copy link

norbit8 commented Jan 1, 2025

Describe the bug

Running flytectl demo start loads up the container and then immediately crashes.

Expected behavior

The flyte's container up and running, being able to take requests.

Additional context to reproduce

First, I'm using apple's silicon M4 CPU (not sure it has anything to do with that though but just for the context)
Now, these are the steps that I've been taking that lead me to this error:

  1. Installed flytesctl
  2. Installed docker desktop
  3. launched docker desktop
  4. Ran flytectl demo start
  • note that it downloads this commit associated with a deployed tag - 8a5f54c
  1. Got this error from inside the container:
2025-01-01 17:15:06 I0101 15:15:06.341726      77 server.go:156] Version: v1.29.0+k3s1
2025-01-01 17:15:06 I0101 15:15:06.341742      77 server.go:158] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
2025-01-01 17:15:06 time="2025-01-01T15:15:06Z" level=error msg="Failed to generate kubeconfig: open /var/lib/flyte/config/kubeconfig: permission denied"
2025-01-01 17:15:06 time="2025-01-01T15:15:06Z" level=fatal msg="open /var/lib/flyte/config/kubeconfig: permission denied"

Screenshots

From my CLI:
image
From Docker desktop:
image

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes
@norbit8 norbit8 added bug Something isn't working untriaged This issues has not yet been looked at by the Maintainers labels Jan 1, 2025
Copy link

welcome bot commented Jan 1, 2025

Thank you for opening your first issue here! 🛠

@wild-endeavor wild-endeavor self-assigned this Jan 2, 2025
@wild-endeavor
Copy link
Contributor

Is there any other information you can provide on this? It doesn't repro for me though i'm on an M1. This file is supposed to end up in ~/.flyte/sandbox which is mounted into that /var/lib path you see.

Do you think you can try:

  • using something like Orbstack instead of docker desktop?
  • what happens if you run that image (the flyte-sandbox-bundled image) manually, mount in that folder, and try to touch a file there?

the only other thing i see if this

:~/.flyte [flyte-sandbox]$ l k3s
total 0
drwxr-xr-x  23 ytong  staff   736B Jan  2 12:56 ../
lrwxrwxrwx   1 ytong  staff    32B Jan  2 12:56 k3s.yaml@ -> /var/lib/flyte/config/kubeconfig

which may be causing a permissions issue. This is written to as part of the mount here by k3s/rancher itself. Keep in mind it's just a link... the file doesn't actually exist there on host.

@wild-endeavor
Copy link
Contributor

@norbit8 - if you're still facing this issue, could you try running colima as your docker runtime instead of Docker Desktop? We've had good luck with that as well in addition to orbstack.

@norbit8
Copy link
Author

norbit8 commented Jan 12, 2025

@wild-endeavor Updating that I've tried with Orbstack and got the same issue, will try again with colima and update

@eapolinario eapolinario added waiting for reporter Used for when we need input from the bug reporter and removed untriaged This issues has not yet been looked at by the Maintainers labels Jan 16, 2025
@davidmirror-ops davidmirror-ops moved this from Backlog to Assigned in Flyte Issues/PRs maintenance Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working waiting for reporter Used for when we need input from the bug reporter
Projects
Status: Assigned
Development

No branches or pull requests

3 participants