Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

live update doesn't work with jobset #6383

Open
jessicaxiejw opened this issue May 27, 2024 · 2 comments
Open

live update doesn't work with jobset #6383

jessicaxiejw opened this issue May 27, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@jessicaxiejw
Copy link

jessicaxiejw commented May 27, 2024

Expected Behavior

Say my manifest is

apiVersion: jobset.x-k8s.io/v1alpha2
kind: JobSet
metadata:
  name: jobset-test
spec:
  replicatedJobs:
    - name: head
      template:
        spec:
          containers:
            - name: head
              image: myimage
              command: ["cat", "changes.txt"]
    - name: worker
      replicas: 2
      template:
        spec:
          containers:
            - name: worker
              image: myimage
              command: ["cat", "changes.txt"]

and the myimage's dockerfile is

# syntax=docker/dockerfile:1.4
FROM busybox:latest
COPY changes.txt /changes.txt

The tiltfile is

docker_build('myimage', '/path/to/dockerfile')
helm_resource(
  'jobset-test',
  '/path/to/jobset.yaml',
  image_deps=['myimage']
  image_keys=[('image.repository', 'image.tag')],
)

If I modify changes.txt and save, I expect head, worker-0, worker-1 pods to get live synced with the new changes.

Current Behavior

Only worker-0 and worker-1 get live updated, the head still has the older copy of changes.txt.

Steps to Reproduce

See "Expected Behavior"

Context

tilt doctor Output

$ tilt doctor
Tilt: v0.33.13, built 2024-04-26
System: darwin-arm64
---
Docker
- Host: unix:///Users/jessicaxie/.colima/docker.sock
- Server Version: 24.0.7
- API Version: 1.43
- Builder: 2
- Compose Version: v2.20.3
---
Kubernetes
- Env: unknown
- Context: <my context>
- Cluster Name: <my cluster>
- Namespace: default
- Container Runtime: cri-o
- Version: v1.27.2
- Cluster Local Registry: none
---
Thanks for seeing the Tilt Doctor!
Please send the info above when filing bug reports. 💗

The info below helps us understand how you're using Tilt so we can improve,
but is not required to ask for help.
---
Analytics Settings
--> (These results reflect your personal opt in/out status and may be overridden by an `analytics_settings` call in your Tiltfile)
- User Mode: opt-out
- Machine: 0cf4cdb1dda5938d522ff39f8c4d5974
- Repo: M2XaMpEc+34CeHQQUKDYew==

About Your Use Case

I am trying to use jobset with tilt.

@jessicaxiejw jessicaxiejw added the bug Something isn't working label May 27, 2024
@jessicaxiejw
Copy link
Author

jessicaxiejw commented May 27, 2024

I think the problem is that tilt calls client.Get in tilt/internal/controllers/core/liveupdate/reconciler.go. Instead, it should call client.List when it could match multiple 🤔

The filtering is controlled by &v1alpha1.LiveUpdate{}, it returns several pods even though it only calls client.Get(). I am confused by how the filtering work though...

@jessicaxiejw
Copy link
Author

The problem seems to come from

if hasValidOwner(pod) && newestOwner != nil && pod.Owner.Name != newestOwner.Name {

For the head pod, pod.Owner.Name is set to jobset-test-head-0 but it is thinking that newestOwner.Name is jobset-test-worker-0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant