add retry to S3 uploads for test results #16579
Conversation
|
Good idea. Will this rethrow the original exception if the retry limit is reached? |
|
Nice idea. But do we know if it will actually upload the xml within 3 attempts? Coz I have seen S3 upload fail many times when some test fails. S3upload is independent of test failure. But just to be sure. |
|
@marcoabreu I don't know. Looks like people are using it assuming it handle exceptions: https://gist.github.com/daipresents/c7f982ffce9c1bda73b4387d6b1ae3da @ChaiBapchya ok, what would you suggest instead? |
|
Seems like it's handling in exactly that fashion. |
|
Could you also add some diagnostics to debug the error? The execution in the mentioned PR said success but then had the return code 1. Seems odd |
No my point was have you by any chance tested that if PR fails on a particular build, does it still upload on S3 or does it fail everytime build fails? For eg for this PR - http://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/mxnet-validation%2Funix-cpu/detail/PR-16660/2/pipeline It doesn't upload to S3 |
|
I wouldn't think it uploads to s3 if the PR fails. |
See title.
#16512