Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Correctly capture failures from the Subshell #103

Merged
merged 1 commit into from
Mar 6, 2018

Conversation

Sutto
Copy link
Contributor

@Sutto Sutto commented Mar 6, 2018

Currently the -e option is turned off from inside the subshell - This fails though, as it remains on for the parent shell.

Thus, to capture (and handle) failures, we need to turn the option off in the top level shell correctly, rather than in a child shell, and restore this after our block of code has been run.

This manifests itself in the form of failed commands only invoking cleanup, and none of the rest of the code. The check on exit code is completely skipped and, for instance, docker logs from linked containers are never uploaded (which is far from ideal).

Currently the -e option is turned off from inside the subshell - This fails though, as it remains on for the parent shell.

Thus, to capture (and handle) failures, we need to turn the option off in the top level shell correctly, rather than in a child shell, and restore this *after* our block of code has been run.
@lox
Copy link
Contributor

lox commented Mar 6, 2018

Thanks for this! I noticed that this was happening the other day and was trying to write some tests to prove what was going on. I'm going to try and commit those tests first to prove the breakage.

@lox
Copy link
Contributor

lox commented Mar 6, 2018

I'm going to merge this and add tests later. Thanks again @Sutto.

@lox lox merged commit bf152ff into buildkite-plugins:master Mar 6, 2018
@Sutto
Copy link
Contributor Author

Sutto commented Mar 6, 2018

No problem! Thanks for looking at it so quickly. I also noticed that it's possible for the docker logs to fail in some cases - so it might be worth wrapping that with the same trick / ignoring their exit code (or else cleanup will be invoked early).

@lox
Copy link
Contributor

lox commented Mar 7, 2018

No problem! Thanks for looking at it so quickly. I also noticed that it's possible for the docker logs to fail in some cases - so it might be worth wrapping that with the same trick / ignoring their exit code (or else cleanup will be invoked early).

Do you mean check_linked_containers_and_save_logs?

@Sutto
Copy link
Contributor Author

Sutto commented Mar 9, 2018

Sorry, I missed your reply - yep. It's possible if a container had no output that it will exit with a non-zero error code - and likewise, it double prints the output (since it runs and prints to stdout, then uploads the artifact) - for the main container run.

@lox
Copy link
Contributor

lox commented Mar 9, 2018

Ah yup, I noticed that too. I'll look into it. Cheers @Sutto.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants