Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RFC] Add checkpoint support for streamlookup #9299

Merged
merged 2 commits into from
Sep 18, 2024
Merged

[RFC] Add checkpoint support for streamlookup #9299

merged 2 commits into from
Sep 18, 2024

Conversation

yumkam
Copy link
Collaborator

@yumkam yumkam commented Sep 16, 2024

Changelog entry

...

Changelog category

  • Not for changelog (changelog entry is not required)

Additional information

Requires #9072 (cherry-pick included to avoid testcase failures/etc, should be applied after).

@yumkam
Fix inputTransform freezing on checkpoint
a660d63 
InputTransforms reads data from channels, so it should not be suspended during
checkpoints.
@yumkam yumkam requested a review from resetius September 16, 2024 11:20
@yumkam yumkam changed the title Add checkpoint support for streamlookup [WIP] Add checkpoint support for streamlookup Sep 16, 2024
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 16, 2024 •
edited
Loading

⚪ 2024-09-16 11:21:36 UTC Pre-commit check linux-x86_64-release-clang14 for c9c2f72 has started.
⚪ 2024-09-16 11:21:46 UTC Artifacts will be uploaded here
⚪ 2024-09-16 11:24:37 UTC ya make is running...
⚫ 2024-09-16 11:24:55 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 16, 2024 •
edited
Loading

⚪ 2024-09-16 11:21:50 UTC Pre-commit check linux-x86_64-relwithdebinfo for c9c2f72 has started.
⚪ 2024-09-16 11:21:54 UTC Artifacts will be uploaded here
⚫ 2024-09-16 11:24:48 UTC Check cancelled

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 16, 2024 •
edited
Loading

⚪ 2024-09-16 11:22:29 UTC Pre-commit check linux-x86_64-release-asan for c9c2f72 has started.
⚪ 2024-09-16 11:24:32 UTC Artifacts will be uploaded here
⚫ 2024-09-16 11:24:52 UTC Check cancelled

@yumkam yumkam force-pushed the add-checkpoint-support-for-streamlookup branch from 7e2bae5 to 45360c8 Compare September 16, 2024 11:24
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 16, 2024 •
edited
Loading

⚪ 2024-09-16 11:27:14 UTC Pre-commit check linux-x86_64-release-clang14 for da1ada1 has started.
⚪ 2024-09-16 11:27:23 UTC Artifacts will be uploaded here
⚪ 2024-09-16 11:30:14 UTC ya make is running...
🔴 2024-09-16 11:34:44 UTC Build failed, see the logs.

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 16, 2024 •
edited
Loading

⚪ 2024-09-16 11:27:19 UTC Pre-commit check linux-x86_64-release-asan for da1ada1 has started.
⚪ 2024-09-16 11:27:23 UTC Artifacts will be uploaded here
⚪ 2024-09-16 11:30:19 UTC ya make is running...
🔴 2024-09-16 11:36:24 UTC Build failed, see the logs.

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 16, 2024 •
edited
Loading

⚪ 2024-09-16 11:27:24 UTC Pre-commit check linux-x86_64-relwithdebinfo for da1ada1 has started.
⚪ 2024-09-16 11:27:28 UTC Artifacts will be uploaded here
⚪ 2024-09-16 11:30:18 UTC ya make is running...
🔴 2024-09-16 11:35:59 UTC Build failed, see the logs.

@yumkam yumkam force-pushed the add-checkpoint-support-for-streamlookup branch from 45360c8 to 100a772 Compare September 16, 2024 11:44
@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 16, 2024 •
edited
Loading

⚪ 2024-09-16 11:54:23 UTC Pre-commit check linux-x86_64-release-clang14 for b6c12c2 has started.
⚪ 2024-09-16 11:54:50 UTC Artifacts will be uploaded here
⚪ 2024-09-16 11:57:48 UTC ya make is running...
🟢 2024-09-16 12:18:01 UTC Build successful.

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 16, 2024 •
edited
Loading

⚪ 2024-09-16 11:54:52 UTC Pre-commit check linux-x86_64-relwithdebinfo for b6c12c2 has started.
⚪ 2024-09-16 11:55:24 UTC Artifacts will be uploaded here
⚪ 2024-09-16 11:58:55 UTC ya make is running...
🟡 2024-09-16 13:21:50 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
74484 60426 0 2 14024 32

⚪ 2024-09-16 13:29:09 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-09-16 13:36:20 UTC Tests successful.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
271 (only retried tests) 178 0 0 8 85

🟢 2024-09-16 13:36:27 UTC Build successful.
🟢 2024-09-16 13:37:09 UTC ydbd size 8.4 GiB changed* by +18.2 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 946e6a2 merge: b6c12c2 diff diff %
ydbd size 9 031 084 480 Bytes 9 031 103 160 Bytes +18.2 KiB +0.000%
ydbd stripped size 488 804 968 Bytes 488 807 080 Bytes +2.1 KiB +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@github-actions GitHub Actions
Copy link

github-actions bot commented Sep 16, 2024 •
edited
Loading

⚪ 2024-09-16 12:10:41 UTC Pre-commit check linux-x86_64-release-asan for b6c12c2 has started.
⚪ 2024-09-16 12:10:49 UTC Artifacts will be uploaded here
⚪ 2024-09-16 12:13:47 UTC ya make is running...
🔴 2024-09-16 14:01:40 UTC Some tests failed, follow the links below.

Test history | Ya make output

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
12114 11937 0 31 43 103

🟢 2024-09-16 14:02:41 UTC Build successful.
🟢 2024-09-16 14:03:13 UTC ydbd size 5.6 GiB changed* by +45.9 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 946e6a2 merge: b6c12c2 diff diff %
ydbd size 6 045 727 016 Bytes 6 045 773 984 Bytes +45.9 KiB +0.001%
ydbd stripped size 1 513 221 360 Bytes 1 513 235 184 Bytes +13.5 KiB +0.001%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@yumkam yumkam marked this pull request as ready for review September 16, 2024 14:00
@yumkam yumkam requested review from a team as code owners September 16, 2024 14:00
@yumkam yumkam changed the title [WIP] Add checkpoint support for streamlookup [RFC] Add checkpoint support for streamlookup Sep 16, 2024
@kardymonds kardymonds enabled auto-merge (squash) September 17, 2024 20:09
@kardymonds kardymonds merged commit c5d3c1a into ydb-platform:main Sep 18, 2024
11 of 13 checks passed
@GrigoriyPA GrigoriyPA mentioned this pull request Sep 19, 2024
Merged
@shnikd shnikd mentioned this pull request Sep 19, 2024
Merged
@niksaveliev niksaveliev mentioned this pull request Sep 24, 2024
Merged
yumkam added a commit to yumkam/ydb that referenced this pull request Sep 24, 2024
@yumkam
[RFC] Add checkpoint support for streamlookup (ydb-platform#9299)
39c1baa 
(cherry picked from commit c5d3c1a)
uzhastik pushed a commit that referenced this pull request Sep 25, 2024
@yumkam
Add checkpoint support for streamlookup (backport #9299) (#9719)
67e35be
This was referenced Oct 3, 2024
Merged
Merged
This was referenced Oct 21, 2024
Merged
Merged
@GrigoriyPA GrigoriyPA mentioned this pull request Oct 29, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@uzhastik uzhastik uzhastik approved these changes

@kardymonds kardymonds kardymonds approved these changes

@aakulaga-ydb aakulaga-ydb aakulaga-ydb approved these changes

@resetius resetius Awaiting requested review from resetius

@zverevgeny zverevgeny Awaiting requested review from zverevgeny

Assignees
No one assigned
Labels
not-for-changelog
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@yumkam @uzhastik @kardymonds @aakulaga-ydb