Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Core] Enable JSON Structured Logging Format in Ray #50471

Merged
merged 15 commits into from
Feb 13, 2025
Merged

[Core] Enable JSON Structured Logging Format in Ray #50471

merged 15 commits into from
Feb 13, 2025

Conversation

MengjinYan
Copy link
Collaborator

Why are these changes needed?

  1. Added the JSON logging format option in Ray
  2. Update the API doc as well as the ray logging doc according to the change:
    • Rearranged the sections in existing ray logging document
    • Copied the content of the structured logging document from Anyscale document to ray logging document

Related issue number

Checks

  • I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed for https://docs.ray.io/en/master/.
    • I've added any new APIs to the API Reference. For example, if I added a
      method in Tune, I've added it in doc/source/tune/api/ under the
      corresponding .rst file.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
  • Testing Strategy
    • Unit tests
    • Release tests
    • This PR is not tested :(

@MengjinYan
add JSON logging format and update the open source document for the s…
7bea861 
…tructured logging feature

Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
@MengjinYan MengjinYan requested a review from a team as a code owner February 12, 2025 06:22
@MengjinYan
fix pre-build failure
3bfc455 
Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
@MengjinYan MengjinYan added the go add ONLY when ready to merge, run all tests label Feb 12, 2025
@jjyao jjyao assigned kevin85421 and unassigned dayshah Feb 12, 2025
dayshah
dayshah reviewed Feb 12, 2025
View reviewed changes
Copy link
Contributor

@dayshah dayshah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just docs review, recommend setting up vale https://docs.ray.io/en/latest/ray-contribute/docs.html#vale and looking into active vs. passive voice https://developers.google.com/style/voice, that's usually a majority of the docs comments

doc/source/ray-observability/user-guides/configure-logging.md
In both `TEXT` and `JSON` encoding formats, the logs include Ray-specific fields such as `job_id`, `worker_id`, `node_id`, `actor_id`, and `task_id`, if available.

* `log_level`: The log level for the driver process. The default is `INFO`.
Available log levels are defined in the [Python logging library](https://docs.python.org/3/library/logging.html#logging-levels).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Available log levels are defined in the [Python logging library](https://docs.python.org/3/library/logging.html#logging-levels).
Python defines available log levels in the [Python logging library](https://docs.python.org/3/library/logging.html#logging-levels).

active vs. passive

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think here it falls into the following exception scenarios of the active vs. passive:

To emphasize an object over an action.

It is to emphasize the available log levels the users can set.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we still want to prefer active voice most times, but ya i see what you're saying. Very minor though I don't think it makes much of a diff here either way

MengjinYan and others added 9 commits February 12, 2025 10:02
@MengjinYan @dayshah
Update doc/source/ray-observability/user-guides/configure-logging.md
b327de0 
Co-authored-by: Dhyey Shah <dhyey2019@gmail.com>
Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
@MengjinYan @dayshah
Update doc/source/ray-observability/user-guides/configure-logging.md
965e3aa 
Co-authored-by: Dhyey Shah <dhyey2019@gmail.com>
Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
@MengjinYan @dayshah
Update doc/source/ray-observability/user-guides/configure-logging.md
f9645e0 
Co-authored-by: Dhyey Shah <dhyey2019@gmail.com>
Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
@MengjinYan @dayshah
Update doc/source/ray-observability/user-guides/configure-logging.md
a78083e 
Co-authored-by: Dhyey Shah <dhyey2019@gmail.com>
Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
@MengjinYan @dayshah
Update doc/source/ray-observability/user-guides/configure-logging.md
60e078f 
Co-authored-by: Dhyey Shah <dhyey2019@gmail.com>
Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
@MengjinYan @dayshah
Update doc/source/ray-observability/user-guides/configure-logging.md
f6338ec 
Co-authored-by: Dhyey Shah <dhyey2019@gmail.com>
Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
@MengjinYan @dayshah
Update doc/source/ray-observability/user-guides/configure-logging.md
5771671 
Co-authored-by: Dhyey Shah <dhyey2019@gmail.com>
Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
@MengjinYan @dayshah
Update doc/source/ray-observability/user-guides/configure-logging.md
0a49e8e 
Co-authored-by: Dhyey Shah <dhyey2019@gmail.com>
Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
@MengjinYan @dayshah
Update doc/source/ray-observability/user-guides/configure-logging.md
23d9d1e 
Co-authored-by: Dhyey Shah <dhyey2019@gmail.com>
Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
edoakes
edoakes reviewed Feb 12, 2025
View reviewed changes
doc/source/ray-observability/user-guides/configure-logging.md
@@ -221,7 +221,7 @@ All Ray loggers are automatically configured in ``ray._private.ray_logging``. To
import logging
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@angelinalg can you help review this file pls?

@MengjinYan
fix doc test failure
55b2a74 
Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
@MengjinYan
Merge branch 'master' into CORE-691
cb8d748
@edoakes edoakes enabled auto-merge (squash) February 12, 2025 22:52
@github-actions github-actions bot disabled auto-merge February 12, 2025 22:52
dayshah
dayshah approved these changes Feb 12, 2025
View reviewed changes
Copy link
Contributor

@dayshah dayshah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm from docs perspective

@edoakes edoakes merged commit 02d3f3c into master Feb 13, 2025
6 checks passed
@edoakes edoakes deleted the CORE-691 branch February 13, 2025 00:09
@edoakes edoakes mentioned this pull request Feb 13, 2025
Closed
israbbani pushed a commit that referenced this pull request Feb 25, 2025
@MengjinYan @dayshah
@edoakes @israbbani
[Core] Enable JSON Structured Logging Format in Ray (#50471)
532cddd 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

1. Added the `JSON` logging format option in Ray
2. Update the API doc as well as the ray logging doc according to the
change:
    - Rearranged the sections in existing ray logging document
- Copied the content of the structured logging document from Anyscale
document to ray logging document

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

<!-- For example: "Closes #1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
Co-authored-by: Dhyey Shah <dhyey2019@gmail.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
xsuler pushed a commit to antgroup/ant-ray that referenced this pull request Mar 4, 2025
@MengjinYan @dayshah
@edoakes @xsuler
[Core] Enable JSON Structured Logging Format in Ray (ray-project#50471)
2f3356b 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

1. Added the `JSON` logging format option in Ray
2. Update the API doc as well as the ray logging doc according to the
change:
    - Rearranged the sections in existing ray logging document
- Copied the content of the structured logging document from Anyscale
document to ray logging document

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

<!-- For example: "Closes ray-project#1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
Co-authored-by: Dhyey Shah <dhyey2019@gmail.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
xsuler pushed a commit to antgroup/ant-ray that referenced this pull request Mar 4, 2025
@MengjinYan @dayshah
@edoakes @xsuler
[Core] Enable JSON Structured Logging Format in Ray (ray-project#50471)
f8d400d 
<!-- Thank you for your contribution! Please review
https://github.com/ray-project/ray/blob/master/CONTRIBUTING.rst before
opening a pull request. -->

<!-- Please add a reviewer to the assignee section when you create a PR.
If you don't have the access to it, we will shortly find a reviewer and
assign them to your PR. -->

## Why are these changes needed?

1. Added the `JSON` logging format option in Ray
2. Update the API doc as well as the ray logging doc according to the
change:
    - Rearranged the sections in existing ray logging document
- Copied the content of the structured logging document from Anyscale
document to ray logging document

<!-- Please give a short summary of the change and the problem this
solves. -->

## Related issue number

<!-- For example: "Closes ray-project#1234" -->

## Checks

- [ ] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [ ] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [ ] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

---------

Signed-off-by: Mengjin Yan <mengjinyan3@gmail.com>
Co-authored-by: Dhyey Shah <dhyey2019@gmail.com>
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jjyao jjyao jjyao approved these changes

@edoakes edoakes edoakes approved these changes

@dayshah dayshah dayshah approved these changes

@kevin85421 kevin85421 Awaiting requested review from kevin85421

Assignees

@jjyao jjyao

@kevin85421 kevin85421

Labels
go add ONLY when ready to merge, run all tests
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@MengjinYan @jjyao @edoakes @dayshah @kevin85421