Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sometimes, logs are not shown #478

Closed
dgageot opened this issue Apr 26, 2018 · 6 comments
Closed

Sometimes, logs are not shown #478

dgageot opened this issue Apr 26, 2018 · 6 comments
Assignees
Labels
area/logging kind/bug Something isn't working

Comments

@dgageot
Copy link
Contributor

dgageot commented Apr 26, 2018

To reproduce this, I run the micro-services in dev mode, interrupt it with ctrl-c and restart it in dev mode.

Sometime all the logs are there. Sometimes, it feels like skaffold is subscribing to the containers that are terminating and doesn't subscribe to those that starting.

@aeweidne
Copy link

I am also experiencing this with the Helm deployer, and I also had the guess it was listening to the terminating pods from the previous deploy.

@r2d4
Copy link
Contributor

r2d4 commented Apr 26, 2018

We are matching the containers by image name, but sometimes the image name is updated before the actual pod is re-rolled (i'm not sure how). We can fix this by directly referencing the image checksum. I tried to describe the issue here

#191

@aeweidne
Copy link

aeweidne commented May 3, 2018

will #191 be reopened any time in the near future? having to trail the logs in a separate terminal ~50% of the time is one of the biggest things stopping skaffold dev from being a great experience for me

@dgageot
Copy link
Contributor Author

dgageot commented May 3, 2018

@shimmerjs The thing is I don't think #191 would fix your issue. Can you describe your issue? Do you lose all the logs or only the first second?

@dgageot
Copy link
Contributor Author

dgageot commented May 3, 2018

@shimmerjs Could you test #513 and see if it gets better?

@aeweidne
Copy link

aeweidne commented May 3, 2018

@dgageot what I'm seeing is subsequent iterations of skaffold dev (whether interrupted by me or triggered by changes in workspaces) don't get any logs streamed about 50% of the time. The error rate seems to be tied to how fast K8s is tearing down the pods (i.e. this problem is lessened on my skaffold projects where the build and deploy time is longer) and occasionally I will get streamed the logs from the last few seconds of the terminating pod, which led me to this issue and thinking it was caused by the same root problem.

I will pull #513 and test it later today and get back to you, thank you!

@dgageot dgageot added kind/bug Something isn't working area/logging labels May 9, 2018
@dgageot dgageot self-assigned this May 10, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/logging kind/bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants