Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

YQ-2918: add processing speed statistics to logs #2720

Merged
merged 3 commits into from
Mar 18, 2024

Conversation

EgorkaZ
Copy link
Contributor

@EgorkaZ EgorkaZ commented Mar 13, 2024

Changelog entry

Changelog category

  • Not for changelog (changelog entry is not required)

Additional information

@EgorkaZ EgorkaZ requested a review from a team as a code owner March 13, 2024 15:04
@EgorkaZ EgorkaZ requested review from Hor911 and uzhastik March 13, 2024 15:04
Copy link

github-actions bot commented Mar 13, 2024

2024-03-13 15:06:31 UTC Pre-commit check for dec3126 has started.
2024-03-13 15:06:33 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-03-13 15:08:35 UTC Build successful.
2024-03-13 15:08:45 UTC Tests are running...
🔴 2024-03-13 15:17:21 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
38971 29090 0 1993 7639 249
2024-03-13 15:20:33 UTC Check cancelled

Copy link

github-actions bot commented Mar 13, 2024

2024-03-13 15:06:48 UTC Pre-commit check for dec3126 has started.
2024-03-13 15:06:49 UTC Build linux-x86_64-release-asan is running...
🟢 2024-03-13 15:08:21 UTC Build successful.
2024-03-13 15:08:34 UTC Tests are running...
🔴 2024-03-13 15:15:29 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
7860 6002 0 1591 20 247

Copy link

github-actions bot commented Mar 13, 2024

2024-03-13 15:08:05 UTC Pre-commit check for dec3126 has started.
2024-03-13 15:08:07 UTC Build linux-x86_64-release-cmake14 is running...
🟢 2024-03-13 15:09:39 UTC Build successful.

Copy link

github-actions bot commented Mar 13, 2024

2024-03-13 15:24:02 UTC Pre-commit check for 56d4da5 has started.
2024-03-13 15:24:04 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-03-13 15:25:11 UTC Build successful.
2024-03-13 15:25:25 UTC Tests are running...
🔴 2024-03-13 15:48:17 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
38971 29090 0 1993 7639 249

Copy link

github-actions bot commented Mar 13, 2024

2024-03-13 15:24:10 UTC Pre-commit check for 56d4da5 has started.
2024-03-13 15:24:12 UTC Build linux-x86_64-release-asan is running...
🟢 2024-03-13 15:25:16 UTC Build successful.
2024-03-13 15:25:28 UTC Tests are running...
🔴 2024-03-13 15:37:10 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
7854 6018 0 1580 9 247

Copy link

github-actions bot commented Mar 13, 2024

2024-03-13 15:24:15 UTC Pre-commit check for 56d4da5 has started.
2024-03-13 15:24:17 UTC Build linux-x86_64-release-cmake14 is running...
🟢 2024-03-13 15:29:56 UTC Build successful.

first = false;
}
PrintSpeeds(builder, statistics.Stats);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Нужно две чиселки печатать. Одна скорость по статистике, а другая скорость на все выполнение query. Полное время выполнения query можно из query.meta достать

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Времени выполнения по статистике нет в v2, сейчас вычисляется из времени начала в job.meta(). Вторая чиселка из CpuTimeUs в статистике: считаем скорость per cpu

ydb/core/fq/libs/compute/common/utils.cpp Outdated Show resolved Hide resolved
}
// getting bytes/second = 1'000'000 * bytes/microsecond
auto speed = (value * 1000000.) / std::max(execTime.MicroSeconds(), ui64{1});
builder << ", \"" << statName << postfix << "\": " << speed;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

размерность бы к скорости дописать bytes/sec

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

У нас и так названия полей выглядят как IngressBytesPerSecond / IngressBytesPerCpuPerSecond, плюс тут печаетется жсон, проще жить, когда его не надо дополнительно парсить

uzhastik
uzhastik previously approved these changes Mar 15, 2024
Copy link

github-actions bot commented Mar 18, 2024

2024-03-18 10:37:40 UTC Pre-commit check for 6af51da has started.
2024-03-18 10:37:41 UTC Build linux-x86_64-relwithdebinfo is running...
🟢 2024-03-18 10:39:01 UTC Build successful.
2024-03-18 10:39:12 UTC Tests are running...
🔴 2024-03-18 10:48:11 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
39012 29128 0 1995 7644 245

Copy link

github-actions bot commented Mar 18, 2024

2024-03-18 10:37:42 UTC Pre-commit check for 6af51da has started.
2024-03-18 10:37:43 UTC Build linux-x86_64-release-asan is running...
🟢 2024-03-18 10:38:59 UTC Build successful.
2024-03-18 10:39:05 UTC Tests are running...
🔴 2024-03-18 10:45:57 UTC Some tests failed, follow the links below.

Test history

TESTS PASSED ERRORS FAILED SKIPPED MUTED?
7885 6014 0 1598 26 247

Copy link

github-actions bot commented Mar 18, 2024

2024-03-18 10:38:33 UTC Pre-commit check for 6af51da has started.
2024-03-18 10:38:36 UTC Build linux-x86_64-release-clang14 is running...
🔴 2024-03-18 10:40:04 UTC Build failed. see the build logs.

@EgorkaZ EgorkaZ merged commit c82b9ce into ydb-platform:main Mar 18, 2024
3 of 5 checks passed
EgorkaZ added a commit to EgorkaZ/ydb that referenced this pull request Mar 18, 2024
@shnikd shnikd mentioned this pull request Mar 26, 2024
@niksaveliev niksaveliev mentioned this pull request Mar 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants