add a new nexus repository to proxy grade services distribution site

[stitakis]

Problem: missing nexus repository for the gradle service distribution site.

Background:
The spring boot quickstarter uses the gradle wrapper as build tool.
The gradle wrapper downloads the gradle binary from internet.
The url that points to the gradle internet site is defined by the property distributionUrl defined in gradle/wrapper/gradle-wrapper.properties. It current value is:

distributionUrl=https\://services.gradle.org/distributions/gradle-6.7.1-bin.zip

Problem is that developers cannot change this property to point to nexus because a repository for that is missing.

This feature request is for adding automatically on nexus deployment a new repository in nexus that proxies https://services.gradle.org/distributions

[stitakis]

fyi @michaelsauter @renedupont

[michaelsauter]

@stitakis Could Gradle be downloaded through an existing repo? Or do none of the current repos distribute Gradle?

[clemensutschig]

this is super tricky,... especially because you need to pass nexus credentials ... one idea I have though of was to swith to gradle rather than the wrapper - which is already installed on the agent?!

https://stackoverflow.com/questions/45310011/how-do-i-provide-credentials-for-gradle-wrapper-without-embedding-them-in-my-pro

[renedupont]

@clemensutschig not sure about your idea because we only have a certain gradle version on the agent installed. If someone requires a later gradle version or uses old outdated gradle stuff that the installed version does not support anymore their build will fail.

[stitakis]

@michaelsauter I have verified this, gradle could not be downloaded through an existing repo. Apart from this, the current maven repos that are setup in nexus are of format maven2. The new required one would need to be of format raw. As example, I have setup one in BIX nexus called services-gradle-org.

@clemensutschig thanks for the link, it is very useful. I'll test the suggested solution to add the properties systemProp.gradle... to the gradle.properties. It looks to me that it would be easy to add them to gradle.properties in the quickstarter pipeline.

Apart from this, it looks to me that in the corporate environment the download of artifacts in build pipelines works. From security point of view this shouldn't be the case.

[clemensutschig]

@stitakis - we can seed a raw repo - that's not a big deal .. (we do this already with leva-documentation - see: https://github.com/opendevstack/ods-core/blob/master/nexus/json/createRepos.json#L22)

the quickstarter pipeline is not the issue here, but the build pipeline .. (https://github.com/opendevstack/ods-quickstarters/blob/master/be-java-springboot/Jenkinsfile.template#L28)

[stitakis]

yeah, the build pipeline could cause troubles... something like this would work;

def gradleDistributionUrl = context.nexusHost.replace("https://", "https\\://${context.nexusUsername}:${context.nexusPassword}@") + "/repository/services-gradle-org"

...
stage('Build and Unit Test') {
    withEnv(["GRADLE_DISTRIBUTION_URL=$gradleDistributionUrl",...
...
        sh(script: "sed -i 's|distributionUrl=http.*://[^/]*|distributionUrl='${GRADLE_DISTRIBUTION_URL}'|' gradle/wrapper/gradle-wrapper.properties", returnStatus: true)
...
}

With this approach we will be able to inject in gralde/wrapper/gradle-wrapper.properties the correct distributionUrl.

@clemensutschig @michaelsauter @renedupont It looks bit tricky for me but doable... what do you think?

[renedupont]

@stitakis looks good so far, the only thing that I'm not sure about yet is the case when someone wants to run it locally and also use nexus. This wouldn't work because the distributionUrl is altered in the Jenkinsfile which is not used locally. Or is this a case we specifically don't want to cover?

[stitakis]

@renedupont well spotted case! There are 2 scenarios for the nexus runs on a local environment:

1. not behind a proxy: user doesn't notice any problem because gradlew can access the distributionUrl defined in `gradle/wrapper/gradle-wrapper.properties' which is pointing to an internet url.
2. behind a proxy in corporate environment: in this case, the gradlew is not able to access the distributionUrl. This is a problem that currently is causing users some overhead and frustration. As workaround they usually need configure the proxy properties in its gradle.properties. For that they need first to gather the configuration, etc... not an ideal situation and definitely doesn't meet the developer expectation of the quickstarter working locally out of the box.

In the 2nd scenario: once we introduce the new nexus repo proposed by this issue, we could adapt the spring boot quickstarter to modify the distributionUrl to point to the nexus repo if the pipeline is running in a corporate environment. We could also introduce the correct nexus repo properties in the gradle.properties so that running gradlew locally behind the corporate proxy works out of the box.

[renedupont]

@stitakis Thinking about it again I'm not quite convinced for the following couple of reasons:

• The gradle-wrapper.properties file that contains the distributionUrl is auto-generated by gradle commands. I think it is in general not a good idea to overwrite auto-generated files. If someone re-generates this, our customized distributionUrl would be overriden.
• Which distributionUrl would you check into VCS? If you check in the distributionUrl pointing to Nexus, everyone who is NOT in the corporate network won't be able to use it locally, and vice versa.

Looking at threads about the same discussion:
https://discuss.gradle.org/t/support-for-gradle-wrapper-distribution-mirror/26838
https://discuss.gradle.org/t/gradle-wrapper-distributionurl/7116/5
https://discuss.gradle.org/t/gradle-wrapper-in-environments-that-require-controlled-builds/11848

I don't see a good solution or workaround currently as the gradle-wrapper.properties does not provide any mechanisms for customization of the distributionUrl.

My hope and preferred solution would be that they introduce a gradle env var for the gradle.properties file that can be used to set an alternative distributionUrl which overrides the one in the gradle-wrapper.properties file.

Regarding

2. behind a proxy in corporate environment: in this case, the gradlew is not able to access the distributionUrl. This is a problem that currently is causing users some overhead and frustration. As workaround they usually need configure the proxy properties in its gradle.properties. For that they need first to gather the configuration, etc... not an ideal situation and definitely doesn't meet the developer expectation of the quickstarter working locally out of the box.

I am not sure if we will reach the goal to have the quickstarter working locally out of the box this way because nexus settings still need to be configured in gradle.properties, especially if we would implement the solution you described.
One can put the gradle.properties in the gradle home directory and it will be picked up by all repos ... at least it would need to be configured only once and not for every repo. Drawback is of course if you have to connect to different nexus servers and proxies etc.

[stitakis]

@renedupont it was quite a long time since I created this ticket... I think, we need first to define the goal we want to achieve and from there find the best suitable solution. From my point of view, every dependency should be downloaded over nexus, also the gradle wrapper.

My hope and preferred solution would be that they introduce a gradle env var for the gradle.properties file that can be used to set an alternative distributionUrl which overrides the one in the gradle-wrapper.properties file.
This sound interesting, have you tried this?

[renedupont]

This sound interesting, have you tried this?

I meant by this, that it would be my prefered solution to this. This is not implemented (as far as I know).

[henrjk]

Maybe gradle/gradle#8477 is of interest here?

[takeseem]

support request +1