HDFS-17844. Fix the incorrect collision detection logic that causes nondeterministic behavior of testViewFsMultipleExportPoint()

[Wang-Benjamin]

Problem

There could be a race condition in generating namenode ID. What is problematic is that different URIs resolve to the same namenode ID, which leads to data integrity issues and security problems. The logic of the underlying function used to detect such problem is incorrect as it throws an error whenever the namenode exists even if the URI is the same. As a result, testViewFsMultipleExportPoint() has a nondeterministic behavior when the test execution order and timing change.

Command line used to identify the nondeterminism of testViewFsMultipleExportPoint():

mvn -pl hadoop-hdfs-project/hadoop-hdfs-nfs \
    -Dcheckstyle.skip=true -Drat.skip=true \
    edu.illinois:nondex-maven-plugin:2.1.7:nondex \
    -Dtest=org.apache.hadoop.hdfs.nfs.nfs3.TestExportsTable#testViewFsMultipleExportPoint \
    -DnondexRuns=20 |& tee nondex-$(date +%s).log

Error example:

[ERROR]   TestExportsTable.testViewFsMultipleExportPoint:111 » FileSystem FS:viewfs, Namenode ID collision for path:/hdfs2 nnid:2130740544 uri being added:hdfs://localhost:34111/ existing uri:hdfs://localhost:34111/

Patch

The proposed fix has changed the collision detection logic in prepareAddressMap() so that it will throw errors only when different URIs resolves to the same namenode ID.

In addition, the namenode ID generation logic in getNamenodeId() is modified where a robust hash function combines URI and address strings, so it now reduces the chance of hash collision.

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	8m 38s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+0 🆗	detsecrets	0m 1s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
-1 ❌	mvninstall	228m 40s	/branch-mvninstall-root.txt	root in trunk failed.
+1 💚	compile	0m 19s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	compile	0m 14s		trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚	checkstyle	0m 16s		trunk passed
+1 💚	mvnsite	0m 19s		trunk passed
+1 💚	javadoc	0m 22s		trunk passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 18s		trunk passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚	spotbugs	0m 32s		trunk passed
+1 💚	shadedclient	20m 54s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 13s		the patch passed
+1 💚	compile	0m 12s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javac	0m 12s		the patch passed
+1 💚	compile	0m 10s		the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚	javac	0m 10s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	0m 8s		the patch passed
+1 💚	mvnsite	0m 13s		the patch passed
+1 💚	javadoc	0m 11s		the patch passed with JDK Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 14s		the patch passed with JDK Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
+1 💚	spotbugs	0m 30s		the patch passed
+1 💚	shadedclient	20m 38s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	2m 54s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs-nfs.txt	hadoop-hdfs-nfs in the patch passed.
+1 💚	asflicense	0m 23s		The patch does not generate ASF License warnings.
		288m 17s

Reason	Tests
Failed junit tests	hadoop.hdfs.nfs.nfs3.TestReaddir
	hadoop.hdfs.nfs.nfs3.TestRpcProgramNfs3
	hadoop.hdfs.nfs.nfs3.TestViewfsWithNfs3
	hadoop.hdfs.nfs.nfs3.TestDFSClientCache
	hadoop.hdfs.nfs.nfs3.TestWrites
	hadoop.hdfs.nfs.nfs3.TestClientAccessPrivilege

Subsystem	Report/Notes
Docker	ClientAPI=1.51 ServerAPI=1.51 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8009/1/artifact/out/Dockerfile
GITHUB PR	#8009
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets
uname	Linux 467ca2ee7b91 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 1c1cdef
Default Java	Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.27+6-post-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_452-8u452-gaus1-0ubuntu120.04-b09
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8009/1/testReport/
Max. process+thread count	564 (vs. ulimit of 5500)
modules	C: hadoop-hdfs-project/hadoop-hdfs-nfs U: hadoop-hdfs-project/hadoop-hdfs-nfs
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8009/1/console
versions	git=2.25.1 maven=3.9.11 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.