Skip to content

Conversation

iwasakims
Copy link
Member

https://issues.apache.org/jira/browse/HADOOP-18375

The hadoop-functions_test_helper.bash assumes that the value of LD_LIBRARY_PATH is blank.

It is not true in the environment like RHEL 8 with GCC 9.

$ grep LD_LIBRARY_PATH /opt/rh/gcc-toolset-9/enable
export LD_LIBRARY_PATH=/opt/rh/gcc-toolset-9/root$rpmlibdir$rpmlibdir32${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
export LD_LIBRARY_PATH=/opt/rh/gcc-toolset-9/root$rpmlibdir$rpmlibdir32:/opt/rh/gcc-toolset-9/root$rpmlibdir/dyninst$rpmlibdir32/dyninst${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Resetting the value of LD_LIBRARY_PATH in the helper script should fix this. The update of environment variable in the tests does not affect outside.

@iwasakims
Copy link
Member Author

Some tests in the hadoop_add_ldlibpath.bats fails with following error without the fix.

     [exec] not ok 3 hadoop_add_ldlibpath (simple dupecheck)
     [exec] # (from function `hadoop_add_colonpath' in file ../../main/bin/hadoop-functions.sh, line 1251,
     [exec] #  from function `hadoop_add_ldlibpath' in file ../../main/bin/hadoop-functions.sh, line 1286,
     [exec] #  in test file hadoop_add_ldlibpath.bats, line 32)
     [exec] #   `hadoop_add_ldlibpath "${TMP}"' failed
     [exec] # bindir: /home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/src/test/scripts
     [exec] # DEBUG: Append colonpath(LD_LIBRARY_PATH): /home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86220.1498
     [exec] # DEBUG: Rejected colonpath(LD_LIBRARY_PATH): /home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86220.1498
     [exec] # >/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib:/opt/rh/gcc-toolset-9/root/usr/lib64/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib:/home/rocky/srcs/hadoop/hadoop-common-project/hadoop-common/target/test-dir/bats.86220.1498<

@iwasakims
Copy link
Member Author

Test the patch on Rocky Linux 8 with gcc-toolset-9-gcc.

$ echo $LD_LIBRARY_PATH
/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib:/opt/rh/gcc-toolset-9/root/usr/lib64/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib

$ mvn test -Pshelltest -Dtest=x
...
     [exec] Running bats -t hadoop_add_ldlibpath.bats
     [exec] 1..9
     [exec] ok 1 hadoop_add_ldlibpath (simple not exist)
     [exec] ok 2 hadoop_add_ldlibpath (simple exist)
     [exec] ok 3 hadoop_add_ldlibpath (simple dupecheck)
     [exec] ok 4 hadoop_add_ldlibpath (default order)
     [exec] ok 5 hadoop_add_ldlibpath (after order)
     [exec] ok 6 hadoop_add_ldlibpath (before order)
     [exec] ok 7 hadoop_add_ldlibpath (simple dupecheck 2)
     [exec] ok 8 hadoop_add_ldlibpath (dupecheck 3)
     [exec] ok 9 hadoop_add_ldlibpath (complex ordering)
...
[INFO] Executed tasks
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  47.310 s
[INFO] Finished at: 2022-07-28T07:36:11Z
[INFO] ------------------------------------------------------------------------

$ echo $LD_LIBRARY_PATH
/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib:/opt/rh/gcc-toolset-9/root/usr/lib64/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib/dyninst:/opt/rh/gcc-toolset-9/root/usr/lib64:/opt/rh/gcc-toolset-9/root/usr/lib

@hadoop-yetus
Copy link

🎊 +1 overall

Vote Subsystem Runtime Logfile Comment
+0 🆗 reexec 0m 42s Docker mode activated.
_ Prechecks _
+1 💚 dupname 0m 0s No case conflicting files found.
+0 🆗 codespell 0m 0s codespell was not available.
+0 🆗 detsecrets 0m 0s detect-secrets was not available.
+1 💚 @author 0m 0s The patch does not contain any @author tags.
+1 💚 test4tests 0m 0s The patch appears to include 1 new or modified test files.
_ trunk Compile Tests _
+1 💚 mvninstall 38m 0s trunk passed
+1 💚 mvnsite 1m 43s trunk passed
+1 💚 shadedclient 59m 34s branch has no errors when building and testing our client artifacts.
_ Patch Compile Tests _
+1 💚 mvninstall 1m 7s the patch passed
+1 💚 blanks 0m 0s The patch has no blanks issues.
+1 💚 mvnsite 1m 22s the patch passed
+1 💚 shadedclient 20m 45s patch has no errors when building and testing our client artifacts.
_ Other Tests _
+1 💚 unit 1m 17s hadoop-common in the patch passed.
+1 💚 asflicense 0m 52s The patch does not generate ASF License warnings.
86m 36s
Subsystem Report/Notes
Docker ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4652/1/artifact/out/Dockerfile
GITHUB PR #4652
Optional Tests dupname asflicense mvnsite unit codespell detsecrets
uname Linux 4e6c9bcba32e 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19 23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool maven
Personality dev-support/bin/hadoop.sh
git revision trunk / 2d12b41
Test Results https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4652/1/testReport/
Max. process+thread count 562 (vs. ulimit of 5500)
modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-4652/1/console
versions git=2.25.1 maven=3.6.3
Powered by Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

@hotcodemacha
Copy link
Contributor

Changes makes sense. +1 (non-binding)

@iwasakims iwasakims requested a review from aajisaka August 25, 2022 04:13
Copy link
Member

@aajisaka aajisaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1. Thank you @iwasakims and @ashutoshcipher

@iwasakims iwasakims merged commit 22835be into apache:trunk Aug 30, 2022
@iwasakims
Copy link
Member Author

Thanks, @ashutoshcipher and @aajisaka.

iwasakims added a commit that referenced this pull request Aug 30, 2022
iwasakims added a commit that referenced this pull request Aug 30, 2022
HarshitGupta11 pushed a commit to HarshitGupta11/hadoop that referenced this pull request Nov 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants