HADOOP-19696. hadoop binary distribution to move cloud connectors to hadoop common/lib #7980
+364
−72
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
|
by contrast: 3.4.2 |
|
Having audited the files coming off the cloud connectors, we have about a dozen whose licenses aren't in the binary the analyticsaccelerator is @ahmarsuhail 's work to add to the license, not sure about the others. Proposed: identify which connector the unacknowledged artifacts are coming from, create homework for each team. |
|
💔 -1 overall
This message was automatically generated. |
|
not very familiar with how the packaging stuff works, so finding this a bit difficult to review. How are you testing the packaging? I just ran
are all the same before and after your changes, so I must be doing something wrong. |
|
did you do a
the big distro created under |
…hadoop common/lib * new assembly for hadoop cloud storage * hadoop-cloud-storage does the assembly on -Pdist * layout stitching to move into share/hadoop/common/lib * remove connectors from hadoop-tools-dist * cut old jackson version from huawaei cloud dependency -even though it was being upgraded by our own artifacts, it was a complication.
* add the artifacts found with the relevant hadoop-* modules to the binary license * leave all three with cloud-storage dependencies such that they don't include these in a pull of hadoop-cloud-storage (regression?) * unless specific profiles cos, huawei and aliyun are declared, at which point they're exported by hadoop-cloud-storage and put into the assembly. This avoids dealing with complex dependencies we don't want (okio, more xml parsers,...), let making it straightforward to build a distro with it if you want. bundle.jar is always getting in. Do I do it here iff -Paws is set or do I delay it until the copy to the final distro artifact tree takes place. delay: keeps it as an export of hadoop-cloud-storage pom early: consistent with the rest
steveloughran
force-pushed
the
build/HADOOP-19696-hadoop-cloud-connector-dist
branch
from
c59e351 to
0aaa6ce
Compare
|
💔 -1 overall
This message was automatically generated. |
* Unshade tos * explicit declaration of apache http dependencies, with excludes as needed * updated LICENSE-binary
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
How was this patch tested?
Manual build, review, storediag, hadoop fs commands
For code changes:
LICENSE,LICENSE-binary,NOTICE-binaryfiles?