[core][observability] Improve observability when the core worker fails to read data from the Raylet socket

[kevin85421]

Why are these changes needed?

If a core worker process has already established a connection with the Raylet but the Raylet process subsequently exits, the core worker process will fail to read data from the socket. However, the error code returned is ENOENT, and users will see the message "No such file or directory" which is very confusing.

This PR improves observability, allowing users to identify that the I/O error is related to the socket.

For example, comment out import logging.handlers and run a random Ray program. The runtime environment agent process will fail to import logging.handlers, causing the Raylet process to exit because it shares fate with the runtime environment agent process. See #48958 (comment) for more details.

Related issue number

Checks

1. Comment out import logging.handlers
2. Run python3 -c "import ray; ray.init();"

• Without this PR
  

• With this PR
  

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[dentiny]

Two questions:

• ENOENT indicates no such entry, or not found, is NotFound a better interpretation?

  ray/src/ray/common/status.h

  Lines 198 to 200 in 7b476ea

  	static Status NotFound(const std::string &msg) {
  	return Status(StatusCode::NotFound, msg);
  	}

• Suggest to apply early return, which is suggested by clang-tidy: https://clang.llvm.org/extra/clang-tidy/checks/readability/else-after-return.html

if (...) {
  continue;
}
if (...) {
  return ...;
}
// do something

[kevin85421]

ENOENT indicates no such entry, or not found, is NotFound a better interpretation?

If we use NotFound, the error message will appear as shown in the screenshot above. It feels a bit strange to me to see NotFound in the error message, as it indicates that the core worker process failed to connect to the raylet process.

Suggest to apply early return, which is suggested by clang-tidy: https://clang.llvm.org/extra/clang-tidy/checks/readability/else-after-return.html

Good to know!