-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
GH-35260: [C++][Python][R] Allow users to adjust S3 log level by envi…
…ronment variable (#38267) ### Rationale for this change It's useful when troubleshooting issues with Arrow's S3 filesystem implementation to raise the log level. Currently, this can only be done in C++ and Python, but not from R. In addition, the log level can only be set during S3 initialization and not directly so the user has to introduce explicit S3 initialization code to turn on logging and must make sure this code is called before S3 initialization. While discussing exposing control of log level to R, we realized that allowing the log level to be controlled by environment variable may be more intuitive and useful and would just be a good addition for C++, Python, and R. ### What changes are included in this PR? - A new environment variable `AWS_S3_LOG_LEVEL` with documentation for controlling S3 log level - Updated documentation for C++, Python, and R - A new `InitializeS3()` as a quality-of-life thing for C++ users. Feel free to ask me to remove this. No changes are needed directly for Python and R because these implementation uses the internal implicit initializer `EnsureS3Initialized` rather than the explicit form, `InitializeS3`. And it's the behavior of the `EnsureS3Initialized` routine that's changed here. ### Are these changes tested? Yes. I added a unit test for the new `GetS3LogLevelFromEnvOrDefault` and tested from Python and R manually. I didn't add a test to make sure the underlying `AwsInstance` gets set up correctly because it looked like it would require a refactor and didn't seem worth it. ### Are there any user-facing changes? Yes. A new way to turn on logging for S3 and matching docs in C++, Python, and R. * Closes: #35260 Lead-authored-by: Bryce Mecum <petridish@gmail.com> Co-authored-by: Sutou Kouhei <kou@cozmixng.org> Co-authored-by: Nic Crane <thisisnic@gmail.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>
- Loading branch information
1 parent
4011058
commit 1e9f224
Showing
9 changed files
with
139 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.