Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NIFI-11358 Upgrade Hadoop from 3.3.4 to 3.3.5 #7097

Closed
wants to merge 1 commit into from

Conversation

exceptionfactory
Copy link
Contributor

Summary

NIFI-11358 Upgrades Hadoop dependencies from 3.3.3 and 3.3.4 to 3.3.5 across multiple extension components. Hadoop 3.3.5 includes a number of bug fixes and transitive dependency upgrades.

Changes include dependency management adjustments to align Hadoop dependencies with the expected version, including the following updates:

  • Aligned Iceberg Hadoop version with project Hadoop version going from 3.3.3 to 3.3.5
  • Aligned Atlas dependency on hadoop-hdfs-client 3.3.0 with hadoop-common 3.3.5
  • Aligned Spark Livy bundle with project Hadoop version corresponding to version expected in nifi-hadoop-utils
  • Updated Ranger Hadoop version from 3.3.3 to 3.3.5

Additional changes include the following:

  • Removed unnecessary dependencies from Hive Test Utilities
  • Updated HBase 2 Woodstox Core from 5.3.0 to 5.4.0
  • Suppressed false positive vulnerabilities for HBase client libraries

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

Pull Request Tracking

  • Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
  • Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

  • Pull Request based on current revision of the main branch
  • Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

  • Build completed using mvn clean install -P contrib-check
    • JDK 11
    • JDK 17

Licensing

  • New dependencies are compatible with the Apache License 2.0 according to the License Policy
  • New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

  • Documentation formatting appears as expected in rendered files

- Upgraded Ranger Hadoop dependencies from 3.3.3 to 3.3.5
- Aligned Iceberg Hadoop version with project Hadoop version
- Updated Atlas dependencies to align hadoop-hdfs-client version with hadoop-common
- Updated Ranger hadoop-auth version to align with other Hadoop dependencies
- Updated Spark Livy bundle to align with project Hadoop version
- Removed unnecessary dependencies from Hive Test Utilities
- Updated HBase 2 Woodstox Core from 5.3.0 to 5.4.0
- Suppressed false positive vulnerabilities for HBase client libraries
@asfgit asfgit closed this in dde89c0 Mar 29, 2023
@pvillard31
Copy link
Contributor

@exceptionfactory - I merged this pull request to main but it does not apply cleanly on the support-1.x branch:

CONFLICT (content): Merge conflict in nifi-nar-bundles/nifi-iceberg-bundle/pom.xml
CONFLICT (content): Merge conflict in nifi-nar-bundles/nifi-ranger-bundle/pom.xml

If we feel like this should go in 1.21, can you provide a specific PR for it?
I don't think this is really a blocker for the 1.x release line to be honest, but I leave it up to you.

@exceptionfactory
Copy link
Contributor Author

Thanks @pvillard31! I was able to cherry-pick the commit to the support branch without conflicts, so that conflict may have been an artifact of the previous commit for #7096. I will push the changes to the support branch.

exceptionfactory added a commit that referenced this pull request Mar 29, 2023
- Upgraded Ranger Hadoop dependencies from 3.3.3 to 3.3.5
- Aligned Iceberg Hadoop version with project Hadoop version
- Updated Atlas dependencies to align hadoop-hdfs-client version with hadoop-common
- Updated Ranger hadoop-auth version to align with other Hadoop dependencies
- Updated Spark Livy bundle to align with project Hadoop version
- Removed unnecessary dependencies from Hive Test Utilities
- Updated HBase 2 Woodstox Core from 5.3.0 to 5.4.0
- Suppressed false positive vulnerabilities for HBase client libraries

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes #7097.
r-vandenbos pushed a commit to r-vandenbos/nifi that referenced this pull request Apr 11, 2023
- Upgraded Ranger Hadoop dependencies from 3.3.3 to 3.3.5
- Aligned Iceberg Hadoop version with project Hadoop version
- Updated Atlas dependencies to align hadoop-hdfs-client version with hadoop-common
- Updated Ranger hadoop-auth version to align with other Hadoop dependencies
- Updated Spark Livy bundle to align with project Hadoop version
- Removed unnecessary dependencies from Hive Test Utilities
- Updated HBase 2 Woodstox Core from 5.3.0 to 5.4.0
- Suppressed false positive vulnerabilities for HBase client libraries

Signed-off-by: Pierre Villard <pierre.villard.fr@gmail.com>

This closes apache#7097.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants