Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix compilation of SerializeToBinaryJson on macos #10783

Merged
merged 5 commits into from
Oct 30, 2024

Conversation

swalrus1
Copy link
Collaborator

@swalrus1 swalrus1 commented Oct 23, 2024

  • fix build
  • add info to error messages

Copy link

github-actions bot commented Oct 23, 2024

2024-10-23 13:27:46 UTC Pre-commit check linux-x86_64-release-asan for d9612a7 has started.
2024-10-23 13:27:54 UTC Artifacts will be uploaded here
2024-10-23 13:30:50 UTC ya make is running...
🟡 2024-10-23 14:57:09 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
10252 10068 0 43 41 100

🟢 2024-10-23 14:58:06 UTC Build successful.
🔴 2024-10-23 14:58:40 UTC ydbd size 5.7 GiB changed* by +27.0 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: f1d756b merge: d9612a7 diff diff %
ydbd size 6 100 510 128 Bytes 6 128 858 360 Bytes +27.0 MiB +0.465%
ydbd stripped size 1 522 674 704 Bytes 1 530 649 840 Bytes +7.6 MiB +0.524%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 23, 2024

2024-10-23 13:28:22 UTC Pre-commit check linux-x86_64-relwithdebinfo for d9612a7 has started.
2024-10-23 13:28:33 UTC Artifacts will be uploaded here
2024-10-23 13:31:41 UTC ya make is running...
🟢 2024-10-23 14:31:52 UTC Tests successful.

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15254 13764 0 0 1391 99

🟢 2024-10-23 14:33:16 UTC Build successful.
🔴 2024-10-23 14:33:34 UTC ydbd size 2.8 GiB changed* by +16.0 MiB, which is >= 2.0 MiB vs main: Alert

ydbd size dash main: f1d756b merge: d9612a7 diff diff %
ydbd size 3 012 732 080 Bytes 3 029 532 360 Bytes +16.0 MiB +0.558%
ydbd stripped size 478 250 424 Bytes 480 714 168 Bytes +2.3 MiB +0.515%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@swalrus1 swalrus1 changed the title add description to json parsing errors fix compilation of SerializeToBinaryJson on macos Oct 28, 2024
Copy link

github-actions bot commented Oct 28, 2024

2024-10-28 08:37:30 UTC Pre-commit check linux-x86_64-release-asan for 7f65713 has started.
2024-10-28 08:37:41 UTC Artifacts will be uploaded here
2024-10-28 08:40:46 UTC ya make is running...
🟡 2024-10-28 10:07:05 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15953 15758 0 45 54 96

🟢 2024-10-28 10:08:30 UTC Build successful.
🟢 2024-10-28 10:09:02 UTC ydbd size 5.7 GiB changed* by +13.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4a9f860 merge: 7f65713 diff diff %
ydbd size 6 138 348 688 Bytes 6 138 362 688 Bytes +13.7 KiB +0.000%
ydbd stripped size 1 531 765 808 Bytes 1 531 765 936 Bytes +128 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 28, 2024

2024-10-28 08:37:39 UTC Pre-commit check linux-x86_64-relwithdebinfo for 7f65713 has started.
2024-10-28 08:37:50 UTC Artifacts will be uploaded here
2024-10-28 08:41:08 UTC ya make is running...
🟡 2024-10-28 09:46:13 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
79239 65185 0 3 13952 99

2024-10-28 09:53:47 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-10-28 10:06:48 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
100 (only retried tests) 7 0 0 0 93

🟢 2024-10-28 10:06:56 UTC Build successful.
🟢 2024-10-28 10:07:17 UTC ydbd size 2.8 GiB changed* by +8.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 4a9f860 merge: 7f65713 diff diff %
ydbd size 3 033 099 368 Bytes 3 033 107 688 Bytes +8.1 KiB +0.000%
ydbd stripped size 480 544 472 Bytes 480 544 600 Bytes +128 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@swalrus1 swalrus1 marked this pull request as ready for review October 28, 2024 16:14
@swalrus1 swalrus1 requested a review from a team as a code owner October 28, 2024 16:14
Copy link

github-actions bot commented Oct 29, 2024

2024-10-29 08:54:58 UTC Pre-commit check linux-x86_64-release-asan for 5e24741 has started.
2024-10-29 08:55:33 UTC Artifacts will be uploaded here
2024-10-29 08:59:13 UTC ya make is running...
🟡 2024-10-29 10:10:53 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15307 15114 0 60 39 94

🟢 2024-10-29 10:12:14 UTC Build successful.
🟢 2024-10-29 10:12:46 UTC ydbd size 5.7 GiB changed* by +13.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 903c6b6 merge: 5e24741 diff diff %
ydbd size 6 139 714 808 Bytes 6 139 728 816 Bytes +13.7 KiB +0.000%
ydbd stripped size 1 532 099 056 Bytes 1 532 099 184 Bytes +128 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 29, 2024

2024-10-29 08:56:09 UTC Pre-commit check linux-x86_64-relwithdebinfo for 5e24741 has started.
2024-10-29 08:56:20 UTC Artifacts will be uploaded here
2024-10-29 08:59:36 UTC ya make is running...
🟡 2024-10-29 10:06:14 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
79246 65192 0 2 13952 100

2024-10-29 10:13:47 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-10-29 10:25:52 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
101 (only retried tests) 6 0 0 0 95

🟢 2024-10-29 10:25:59 UTC Build successful.
🟢 2024-10-29 10:26:20 UTC ydbd size 2.8 GiB changed* by +8.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 903c6b6 merge: 5e24741 diff diff %
ydbd size 3 033 922 776 Bytes 3 033 931 096 Bytes +8.1 KiB +0.000%
ydbd stripped size 480 682 264 Bytes 480 682 392 Bytes +128 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 29, 2024

2024-10-29 10:41:17 UTC Pre-commit check linux-x86_64-release-asan for c673e6c has started.
2024-10-29 10:41:28 UTC Artifacts will be uploaded here
2024-10-29 10:44:34 UTC ya make is running...
🟡 2024-10-29 12:00:15 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15070 14934 0 45 61 30

🟢 2024-10-29 12:01:36 UTC Build successful.
🟢 2024-10-29 12:02:09 UTC ydbd size 5.7 GiB changed* by +13.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 8c63dbf merge: c673e6c diff diff %
ydbd size 6 139 835 928 Bytes 6 139 849 928 Bytes +13.7 KiB +0.000%
ydbd stripped size 1 532 105 648 Bytes 1 532 105 776 Bytes +128 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 29, 2024

2024-10-29 10:43:27 UTC Pre-commit check linux-x86_64-relwithdebinfo for c673e6c has started.
2024-10-29 10:43:37 UTC Artifacts will be uploaded here
2024-10-29 10:46:43 UTC ya make is running...
🟡 2024-10-29 11:50:52 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
79248 65182 0 4 13965 97

2024-10-29 11:58:14 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-10-29 12:09:47 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
112 (only retried tests) 20 0 0 0 92

🟢 2024-10-29 12:09:54 UTC Build successful.
🟢 2024-10-29 12:10:15 UTC ydbd size 2.8 GiB changed* by +8.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 8c63dbf merge: c673e6c diff diff %
ydbd size 3 033 991 864 Bytes 3 034 000 168 Bytes +8.1 KiB +0.000%
ydbd stripped size 480 683 032 Bytes 480 683 160 Bytes +128 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@@ -22,6 +22,10 @@ SRCS(

GENERATE_ENUM_SERIALIZATION(format.h)

CFLAGS(
Copy link
Collaborator Author

@swalrus1 swalrus1 Oct 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Игнорирование warning'а, чтобы сборка хедеров simdjson не падала на windows

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Правильно ли понял, что здесь это были варнинги внешней библиотеки simdjson?

Copy link
Collaborator Author

@swalrus1 swalrus1 Oct 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Да, вот они в логах сборки: https://nda.ya.ru/t/HW0alCdi79HxbP

RETURN_IF_NOT_SUCCESS(value.get(v));
callbacks.OnInteger(v);
break;
}
case simdjson::dom::element_type::UINT64: {
ui64 v;
uint64_t v;
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ui64 -- unsigned long на windows, из-за чего компиляция value.get(v) падает

RETURN_IF_NOT_SUCCESS(value.get(v));
callbacks.OnInteger(v);
break;
}
case simdjson::fallback::number_type::unsigned_integer: {
ui64 v;
case simdjson::builtin::number_type::unsigned_integer: {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fallback -- конкретная имплементация парсера

builtin -- имплементация парсера, выбранная при компиляции

@swalrus1 swalrus1 enabled auto-merge (squash) October 30, 2024 07:27
return arrow::Status::SerializationError(
"Cannot serialize json (", binaryJson.GetErrorMessage(), "): ", valueBuf.SubStr(0, Min(valueBuf.Size(), 1024ul)));
return arrow::Status::SerializationError("Cannot serialize json (", binaryJson.GetErrorMessage(),
"): ", valueBuf.SubStr(0, Min(valueBuf.Size(), static_cast<size_t>(1024))));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Наверное в static_cast литерала ничего плохого нет, но я бы написал size_t{1024} здесь

@@ -22,6 +22,10 @@ SRCS(

GENERATE_ENUM_SERIALIZATION(format.h)

CFLAGS(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Правильно ли понял, что здесь это были варнинги внешней библиотеки simdjson?

Copy link

github-actions bot commented Oct 30, 2024

2024-10-30 09:52:32 UTC Pre-commit check linux-x86_64-relwithdebinfo for 4656b05 has started.
2024-10-30 09:52:42 UTC Artifacts will be uploaded here
2024-10-30 09:55:57 UTC ya make is running...
🟡 2024-10-30 11:00:57 UTC Some tests failed, follow the links below. Going to retry failed tests...

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
79259 65207 0 1 13952 99

2024-10-30 11:08:27 UTC ya make is running... (failed tests rerun, try 2)
🟢 2024-10-30 11:20:20 UTC Tests successful.

Test history | Ya make output | Test bloat | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
99 (only retried tests) 5 0 0 0 94

🟢 2024-10-30 11:20:27 UTC Build successful.
🟢 2024-10-30 11:20:48 UTC ydbd size 2.8 GiB changed* by +8.1 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 0b573b9 merge: 4656b05 diff diff %
ydbd size 3 038 282 872 Bytes 3 038 291 192 Bytes +8.1 KiB +0.000%
ydbd stripped size 481 266 424 Bytes 481 266 552 Bytes +128 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

Copy link

github-actions bot commented Oct 30, 2024

2024-10-30 09:52:41 UTC Pre-commit check linux-x86_64-release-asan for 4656b05 has started.
2024-10-30 09:52:52 UTC Artifacts will be uploaded here
2024-10-30 09:56:01 UTC ya make is running...
🟡 2024-10-30 11:16:40 UTC Some tests failed, follow the links below. This fail is not in blocking policy yet

Test history | Ya make output | Test bloat

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
15075 14990 0 35 18 32

🟢 2024-10-30 11:17:53 UTC Build successful.
🟢 2024-10-30 11:18:24 UTC ydbd size 5.7 GiB changed* by +13.7 KiB, which is < 100.0 KiB vs main: OK

ydbd size dash main: 0b573b9 merge: 4656b05 diff diff %
ydbd size 6 147 852 648 Bytes 6 147 866 648 Bytes +13.7 KiB +0.000%
ydbd stripped size 1 534 198 032 Bytes 1 534 198 160 Bytes +128 Bytes +0.000%

*please be aware that the difference is based on comparing your commit and the last completed build from the post-commit, check comparation

@swalrus1 swalrus1 merged commit 5130589 into ydb-platform:main Oct 30, 2024
10 checks passed
This was referenced Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants