timestamp gitlab ci job outputs #13605
Conversation
altaua
added
A3-in_progress
|
I don't like an optional timestamp functionality. This adds complexity. I'd rather find the way to fix the issues with some pipelines failing with this script. |
Aye, I'd like them to be enabled everywhere as well; I'm planning to investigate the failures once I've confirmed everything else works so far at least. |
altaua
force-pushed
the
mira/gitlab-timestamp
branch
2 times, most recently
from
3ee5fc4
to
9516fa1
Compare
|
Turns out the linux-stable-int test was just a hardcoded timeout inside the test being hit because the naive timestamping script we'd been using slows output-heavy tests down a lot. This was further compounded by the old script being bugged and discarding all remaining buffered output when the job script finishes. |
altaua
force-pushed
the
mira/gitlab-timestamp
branch
3 times, most recently
from
2f606ea
to
2cb5f65
Compare
Some of our jobs don't check out the substrate repo.
altaua
force-pushed
the
mira/gitlab-timestamp
branch
from
2cb5f65
to
1499706
Compare
Still not including it in the zombienet jobs, they have their own timestamping anyway.
altaua
force-pushed
the
mira/gitlab-timestamp
branch
from
1499706
to
7a9b587
Compare
altaua
changed the title
altaua
added
A0-please_review
scripts/ci/gitlab/timestamp.yml
Outdated
| exec > >(s_datestamp) 2>&1 | ||
|
|
||
| TIMESTAMP_PID=$! | ||
| trap cleanup EXIT |
There was a problem hiding this comment.
Interesting. I don't understand what is it for) Could you please explain?
There was a problem hiding this comment.
Oh, that's to fix a bug in the original script: The main job script might finish before the s_datestamp process substitution has had a chance to process all output; if we don't wait for it, bash will just exit immediately, killing s_datestamp and thus discarding the last chunk of output.
There was a problem hiding this comment.
(Incidentally, that's what made us not see the timeout error message in test-linux-stable-int for #13047)
|
It might be useful to add this script to other projects. Do you know is there any shared library for scripts in the gitlab? |
Other repos could include the .timestamp spec with an include:project directive; we just need to discuss if we're fine with including it from substrate or would like to start a dedicated pipeline library repo oslt (or do we perchance have one and I just haven't discovered it yet?). |
I've created this project. Feel free to add the |
The dedicated library sounds just right. This would help us to maintain shared code properly and use DRY approach. |
altaua
force-pushed
the
mira/gitlab-timestamp
branch
from
2171f72
to
8299e22
Compare
|
.timestamp will have to remain disabled for macos jobs; For reference, the missing bash features are
|
|
bot merge |
|
Waiting for commit status. |
* timestamp gitlab ci job outputs Based on previous work by @alvicsam in paritytech#13047. * inline timestamp script Some of our jobs don't check out the substrate repo. * include .timestamp in pipelines overriding the default before_script Still not including it in the zombienet jobs, they have their own timestamping anyway. * move timestamp.yml to shared pipeline repo https://gitlab.parity.io/parity/infrastructure/ci_cd/shared
* timestamp gitlab ci job outputs Based on previous work by @alvicsam in paritytech#13047. * inline timestamp script Some of our jobs don't check out the substrate repo. * include .timestamp in pipelines overriding the default before_script Still not including it in the zombienet jobs, they have their own timestamping anyway. * move timestamp.yml to shared pipeline repo https://gitlab.parity.io/parity/infrastructure/ci_cd/shared
cf https://github.com/paritytech/ci_cd/issues/715
based on previous work by @alvicsam in #13047