HADOOP-18027. Include static imports in the maven plugin rules #3733

virajjasani · 2021-11-29T10:03:19Z

Description of PR

Maven enforcer plugin to ban illegal imports require explicit mention of static imports in order to evaluate whether any publicly accessible static entities from the banned classes are directly imported by Hadoop code.

How was this patch tested?

Tested locally.

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?

steveloughran · 2021-11-29T16:25:01Z

+1 pending yetus being happy

hadoop-yetus · 2021-11-30T07:15:53Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 38s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 13s		trunk passed
+1 💚	compile	22m 5s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	19m 20s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	mvnsite	25m 24s		trunk passed
+1 💚	javadoc	7m 57s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	7m 58s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	134m 26s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	21m 16s		the patch passed
+1 💚	compile	21m 17s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	21m 17s		the patch passed
+1 💚	compile	19m 14s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	19m 14s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	21m 0s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	7m 49s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 4s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	36m 27s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	794m 12s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 33s		The patch does not generate ASF License warnings.
		1052m 1s

Reason	Tests
Failed junit tests	hadoop.yarn.csi.client.TestCsiClient
	hadoop.fs.http.server.TestHttpFSAccessControlled
	hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem
	hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem
	hadoop.tools.dynamometer.TestDynamometerInfra

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3733/3/artifact/out/Dockerfile
GITHUB PR	#3733
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml
uname	Linux dfb4faddbff1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `8fdc03e`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3733/3/testReport/
Max. process+thread count	3172 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3733/3/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

virajjasani · 2021-11-30T08:45:04Z

Thanks for the review @steveloughran. Jenkins QA has included all modules in the result since we changed hadoop-main pom. And test failures are known issues (irrelevant to this PR).

hadoop-yetus · 2021-11-30T12:05:18Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 7s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	39m 30s		trunk passed
+1 💚	compile	30m 24s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	24m 42s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	mvnsite	31m 25s		trunk passed
+1 💚	javadoc	10m 31s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 43s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	167m 50s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	24m 6s		the patch passed
+1 💚	compile	23m 6s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	23m 6s		the patch passed
+1 💚	compile	20m 21s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	20m 21s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	21m 58s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	8m 10s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 7s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	39m 30s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	1039m 22s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 33s		The patch does not generate ASF License warnings.
		1341m 22s

Reason	Tests
Failed junit tests	hadoop.tools.dynamometer.TestDynamometerInfra
	hadoop.yarn.csi.client.TestCsiClient
	hadoop.hdfs.rbfbalance.TestRouterDistCpProcedure
	hadoop.hdfs.TestHDFSFileSystemContract
	hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem
	hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3733/2/artifact/out/Dockerfile
GITHUB PR	#3733
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml
uname	Linux fa855445efcd 4.15.0-147-generic #151-Ubuntu SMP Fri Jun 18 19:21:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `8fdc03e`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3733/2/testReport/
Max. process+thread count	2423 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3733/2/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tasanuma · 2021-11-30T13:32:28Z

@virajjasani It seems org.apache.hadoop.thirdparty.com.google.common.io.BaseEncoding has some static methods, and we should ban them too.

virajjasani · 2021-11-30T13:45:18Z

Thanks @tasanuma, you are correct. Added static for BaseEncoding as well. Thanks

The build might take ~24 hr given that it will run for all modules.

tasanuma · 2021-11-30T13:50:19Z

@virajjasani Thanks for updating it quickly. +1, pending CI.

hadoop-yetus · 2021-12-01T07:37:56Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	1m 1s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 1s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	40m 9s		trunk passed
+1 💚	compile	30m 12s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	25m 35s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	mvnsite	30m 24s		trunk passed
+1 💚	javadoc	8m 37s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 14s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	162m 47s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	21m 55s		the patch passed
+1 💚	compile	22m 53s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	22m 53s		the patch passed
+1 💚	compile	19m 53s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	19m 53s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	22m 16s		the patch passed
+1 💚	xml	0m 2s		The patch has no ill-formed XML file.
+1 💚	javadoc	8m 11s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 22s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	37m 35s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	768m 15s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 37s		The patch does not generate ASF License warnings.
		1060m 40s

Reason	Tests
Failed junit tests	hadoop.yarn.csi.client.TestCsiClient
	hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem
	hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem
	hadoop.tools.dynamometer.TestDynamometerInfra

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3733/6/artifact/out/Dockerfile
GITHUB PR	#3733
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml
uname	Linux e347738cb5f0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `5274830`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3733/6/testReport/
Max. process+thread count	3158 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3733/6/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

tasanuma · 2021-12-01T07:48:54Z

Merged. Thanks for your contribution, @virajjasani. Thanks for your review, @steveloughran.

hadoop-yetus · 2021-12-01T07:58:29Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 37s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	33m 9s		trunk passed
+1 💚	compile	23m 29s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	compile	21m 9s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	mvnsite	27m 12s		trunk passed
+1 💚	javadoc	8m 37s		trunk passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 29s		trunk passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	142m 38s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	22m 39s		the patch passed
+1 💚	compile	22m 52s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javac	22m 52s		the patch passed
+1 💚	compile	20m 59s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	javac	20m 59s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	22m 58s		the patch passed
+1 💚	xml	0m 1s		The patch has no ill-formed XML file.
+1 💚	javadoc	8m 37s		the patch passed with JDK Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
+1 💚	javadoc	8m 33s		the patch passed with JDK Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
+1 💚	shadedclient	38m 25s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	818m 19s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 40s		The patch does not generate ASF License warnings.
		1093m 23s

Reason	Tests
Failed junit tests	hadoop.tools.dynamometer.TestDynamometerInfra
	hadoop.yarn.csi.client.TestCsiClient
	hadoop.fs.http.client.TestHttpFSFWithWebhdfsFileSystem
	hadoop.fs.http.client.TestHttpFSFWithSWebhdfsFileSystem
	hadoop.hdfs.TestRollingUpgrade

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3733/4/artifact/out/Dockerfile
GITHUB PR	#3733
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient codespell xml
uname	Linux b9398bce356d 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `5274830`
Default Java	Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3733/4/testReport/
Max. process+thread count	3653 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3733/4/console
versions	git=2.25.1 maven=3.6.3
Powered by	Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

…e#3733) Reviewed-by: Steve Loughran <stevel@apache.org> Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>

HADOOP-18027. Include static imports in the maven plugin rules

8fdc03e

include static for BaseEncoding

5274830

tasanuma approved these changes Dec 1, 2021

View reviewed changes

tasanuma merged commit 4dea4a7 into apache:trunk Dec 1, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HADOOP-18027. Include static imports in the maven plugin rules #3733

HADOOP-18027. Include static imports in the maven plugin rules #3733

Uh oh!

virajjasani commented Nov 29, 2021

Uh oh!

steveloughran commented Nov 29, 2021

Uh oh!

hadoop-yetus commented Nov 30, 2021

Uh oh!

virajjasani commented Nov 30, 2021

Uh oh!

hadoop-yetus commented Nov 30, 2021

Uh oh!

tasanuma commented Nov 30, 2021

Uh oh!

virajjasani commented Nov 30, 2021

Uh oh!

tasanuma commented Nov 30, 2021

Uh oh!

hadoop-yetus commented Dec 1, 2021

Uh oh!

tasanuma commented Dec 1, 2021

Uh oh!

hadoop-yetus commented Dec 1, 2021

Uh oh!

Uh oh!

HADOOP-18027. Include static imports in the maven plugin rules #3733

HADOOP-18027. Include static imports in the maven plugin rules #3733

Uh oh!

Conversation

virajjasani commented Nov 29, 2021

Description of PR

How was this patch tested?

For code changes:

Uh oh!

steveloughran commented Nov 29, 2021

Uh oh!

hadoop-yetus commented Nov 30, 2021

Uh oh!

virajjasani commented Nov 30, 2021

Uh oh!

hadoop-yetus commented Nov 30, 2021

Uh oh!

tasanuma commented Nov 30, 2021

Uh oh!

virajjasani commented Nov 30, 2021

Uh oh!

tasanuma commented Nov 30, 2021

Uh oh!

hadoop-yetus commented Dec 1, 2021

Uh oh!

tasanuma commented Dec 1, 2021

Uh oh!

hadoop-yetus commented Dec 1, 2021

Uh oh!

Uh oh!