-
Notifications
You must be signed in to change notification settings - Fork 2.7k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #4 in ABFX/maven-forked from incremental-maven-3.6…
….3 to MNG-7129/maven-3.6.3-incremental Squashed commit of the following: commit 21c620a34ddbbf84b3c44014b069a7b222cf91a7 Author: Maximilian Novikov <maximilian.novikov@db.com> Date: Tue Aug 10 19:41:41 2021 +0200 Merge pull request #3 in ABFX/maven-forked from code-cleanup-2 to incremental-maven-3.6.3 Squashed commit of the following: commit 3401739ec9eb957fb03887f6a3c24477c9c3327b Author: Alexander Ashitkin <alexander.ashitkin@db.com> Date: Tue Aug 10 13:37:13 2021 -0400 cleanup db specifics commit 5d32f00559951445ecaf7d5ec4d847541c10e7e9 Author: Alexander Ashitkin <alexander.ashitkin@db.com> Date: Tue Aug 10 11:05:28 2021 -0400 cleanup db specifics commit fcec28f82c0d447de64032c610b39fa75bb344b0 Merge: 6fd401af2 5b0dc1858 Author: Alexander Ashitkin <alexander.ashitkin@db.com> Date: Mon Aug 9 13:22:16 2021 +0100 Pull request #9: Remote cache setup Merge in BND/maven-forked from feature/remote-cache-setup to incremental-maven-3.6.3 * commit '5b0dc1858cf4ca499502f0741b2f3bf2ac55530c': remote cache setup - fail fast, baseline build, diff reporting, documentation. commit 5b0dc1858cf4ca499502f0741b2f3bf2ac55530c Author: Alexander Ashitkin <alexander.ashitkin@db.com> Date: Mon Mar 22 03:30:03 2021 +0100 remote cache setup - fail fast, baseline build, diff reporting, documentation. commit 6fd401af285692df568312ee66ee00edfe21f1cc Author: Maximilian Novikov <maximilian.novikov@db.com> Date: Tue Apr 13 17:36:49 2021 +0100 Pull request #8: Incremental Maven - added non-overrideable cache entries Merge in BND/maven-forked from non-overritable-build-info to incremental-maven-3.6.3 Squashed commit of the following: commit 405bbef8a59cabc0b83e9a64717f65fbd6fb9e3d Author: maximilian.novikov@db.com <maximilian.novikov@db.com> Date: Mon Apr 12 20:18:27 2021 +0200 Incremental Maven - added non-overrideable cache entries - review fixes commit 197704672756e558fb3521e16859d0be324044b8 Author: maximilian.novikov@db.com <maximilian.novikov@db.com> Date: Fri Apr 9 11:35:48 2021 +0200 Incremental Maven - added non-overrideable cache entries commit 64c4b1f5cbdf920ad43560f34cc5be56d4557acf Author: Alexander Ashitkin <alexander.ashitkin@db.com> Date: Tue Mar 9 04:10:20 2021 +0100 injected version commit 727e507ae2fd90dce7011f3f1c76eb4969577d5e Author: maximilian.novikov@db.com <maximilian.novikov@db.com> Date: Wed Jan 13 16:17:25 2021 +0100 Incremental Maven - Java 11 fix commit b2f297566d5afcb0a285985cb5540685a28cafec Author: maximilian.novikov@db.com <maximilian.novikov@db.com> Date: Tue Jan 12 14:10:08 2021 +0100 Incremental Maven - initial commit
- Loading branch information
1 parent
5c36bf5
commit c9a0366
Showing
83 changed files
with
11,472 additions
and
85 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -3,6 +3,7 @@ target/ | |
.classpath | ||
.settings/ | ||
.svn/ | ||
.cache/ | ||
bin/ | ||
# Intellij | ||
*.ipr | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,204 @@ | ||
<!--- | ||
Licensed to the Apache Software Foundation (ASF) under one or more | ||
contributor license agreements. See the NOTICE file distributed with | ||
this work for additional information regarding copyright ownership. | ||
The ASF licenses this file to You under the Apache License, Version 2.0 | ||
(the "License"); you may not use this file except in compliance with | ||
the License. You may obtain a copy of the License at | ||
http://www.apache.org/licenses/LICENSE-2.0 | ||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
--> | ||
|
||
### Overview | ||
|
||
Cache configuration provides you additional control over incremental maven behavior. Follow it step by step to | ||
understand how it works and figure out your optimal config | ||
|
||
### Minimal config | ||
|
||
Absolutely minimal config which enables incremental maven with local cache | ||
|
||
```xml | ||
<?xml version="1.0" encoding="UTF-8" ?> | ||
<cache xmlns="org:apache:maven:cache:config:v1" | ||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
xsi:schemaLocation="org:apache:maven:cache:config:v1 cache-config.xsd"> | ||
|
||
<configuration> | ||
<enabled>true</enabled> | ||
<hashAlgorithm>XX</hashAlgorithm> | ||
</configuration> | ||
|
||
<input> | ||
<global> | ||
<glob>{*.java,*.xml,*.properties}</glob> | ||
</global> | ||
</input> | ||
</cache> | ||
``` | ||
|
||
### Enabling remote cache | ||
|
||
Just add `<remote>` section under `<configuration>` | ||
|
||
```xml | ||
|
||
<configuration> | ||
<enabled>true</enabled> | ||
<hashAlgorithm>XX</hashAlgorithm> | ||
<remote> | ||
<url>https://yourserver:port</url> | ||
</remote> | ||
</configuration> | ||
``` | ||
|
||
### Adding more file types to input | ||
|
||
Add all the project specific source code files in `<glob>`. Scala in this case: | ||
|
||
```xml | ||
|
||
<input> | ||
<global> | ||
<glob>{*.java,*.xml,*.properties,*.scala}</glob> | ||
</global> | ||
</input> | ||
``` | ||
|
||
### Adding source directory for bespoke project layouts | ||
|
||
In most of the cases incremental maven will recognize directories automatically by build introspection. If not, you can | ||
add additional directories with `<include>`. Also you can filter out undesirable dirs and files by using exclude tag | ||
|
||
```xml | ||
|
||
<input> | ||
<global> | ||
<glob>{*.java,*.xml,*.properties,*.scala}</glob> | ||
<include>importantdir/</include> | ||
<exclude>tempfile.out</exclude> | ||
</global> | ||
</input> | ||
``` | ||
|
||
### Plugin property is env specific (breaks checksum and caching) | ||
|
||
Consider to exclude env specific properties: | ||
|
||
```xml | ||
|
||
<input> | ||
<global> | ||
... | ||
</global> | ||
<plugin artifactId="maven-surefire-plugin"> | ||
<effectivePom> | ||
<excludeProperty>argLine</excludeProperty> | ||
</effectivePom> | ||
</plugin> | ||
</input> | ||
``` | ||
|
||
Implications - builds with different `argLine` will have identical checksum. Validate that is semantically valid. | ||
|
||
### Plugin property points to directory where only subset of files is relevant | ||
|
||
If plugin configuration property points to `somedir` it will be scanned with default glob. You can tweak it with custom | ||
processing rule | ||
|
||
```xml | ||
|
||
<input> | ||
<global> | ||
... | ||
</global> | ||
<plugin artifactId="protoc-maven-plugin"> | ||
<dirScan mode="auto"> | ||
<!--<protoBaseDirectory>${basedir}/..</protoBaseDirectory>--> | ||
<tagScanConfig tagName="protoBaseDirectory" recursive="false" glob="{*.proto}"/> | ||
</dirScan> | ||
</plugin> | ||
</input> | ||
``` | ||
|
||
### Local repository is not updated because `install` is cached | ||
|
||
Add `executionControl/runAlways` section | ||
|
||
```xml | ||
<?xml version="1.0" encoding="UTF-8" ?> | ||
<cache xmlns="org:apache:maven:cache:config:v1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
xsi:schemaLocation="org:apache:maven:cache:config:v1 cache-config.xsd"> | ||
<configuration> | ||
... | ||
</configuration> | ||
<input> | ||
... | ||
</input> | ||
<executionControl> | ||
<runAlways> | ||
<plugin artifactId="maven-failsafe-plugin"/> | ||
<execution artifactId="maven-dependency-plugin"> | ||
<execId>unpack-autoupdate</execId> | ||
</execution> | ||
<goals artifactId="maven-install-plugin"> | ||
<goal>install</goal> | ||
</goals> | ||
</runAlways> | ||
</executionControl> | ||
</cache> | ||
``` | ||
|
||
### I occasionally cached build with `-DskipTests=true` and tests do not run now | ||
|
||
If you add command line flags to your build, they do not participate in effective pom - maven defers final value | ||
resolution to plugin runtime. To invalidate build if filed value is different in runtime, add reconciliation section | ||
to `executionControl`: | ||
|
||
```xml | ||
|
||
<executionControl> | ||
<runAlways> | ||
... | ||
</runAlways> | ||
<reconcile> | ||
<plugin artifactId="maven-surefire-plugin" goal="test"> | ||
<reconcile propertyName="skip" skipValue="true"/> | ||
<reconcile propertyName="skipExec" skipValue="true"/> | ||
<reconcile propertyName="skipTests" skipValue="true"/> | ||
<reconcile propertyName="testFailureIgnore" skipValue="true"/> | ||
</plugin> | ||
</reconcile> | ||
</executionControl> | ||
``` | ||
|
||
Please notice `skipValue` attribute. It denotes value which forces skipped execution. | ||
Read `propertyName="skipTests" skipValue="true"` as if property skipTests has value true, plugin will skip execution If | ||
you declare such value incremental maven will reuse appropriate full-build though technically they are different, but | ||
because full-build is better it is safe to reuse | ||
|
||
### How to renormalize line endings in working copy after committing .gitattributes (git 2.16+) | ||
|
||
Ensure you've committed (and ideally pushed everything) - no changes in working copy. After that: | ||
|
||
```shell | ||
# Rewrite objects and update index | ||
git add --renormalize . | ||
# Commit changes | ||
git commit -m "Normalizing line endings" | ||
# Remove working copy paths from git cache | ||
git rm --cached -r . | ||
# Refresh with new line endings | ||
git reset --hard | ||
``` | ||
|
||
### I want to cache interim build and override it later with final version | ||
|
||
Solution: set `-Dremote.cache.save.final=true` to nodes which produce final builds. Such builds will not be overridden | ||
and eventually will replace all interim builds |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,52 @@ | ||
<!--- | ||
Licensed to the Apache Software Foundation (ASF) under one or more | ||
contributor license agreements. See the NOTICE file distributed with | ||
this work for additional information regarding copyright ownership. | ||
The ASF licenses this file to You under the Apache License, Version 2.0 | ||
(the "License"); you may not use this file except in compliance with | ||
the License. You may obtain a copy of the License at | ||
http://www.apache.org/licenses/LICENSE-2.0 | ||
Unless required by applicable law or agreed to in writing, software | ||
distributed under the License is distributed on an "AS IS" BASIS, | ||
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
See the License for the specific language governing permissions and | ||
limitations under the License. | ||
--> | ||
|
||
# Overview | ||
|
||
This documents contains various configuration parameters supported by cache engine | ||
|
||
## Command line flags | ||
|
||
| Parameter | Description | Usage Scenario | | ||
| ----------- | ----------- | ----------- | | ||
| `-Dremote.cache.configPath=true/false` | Location of cache configuration file | Cache config is not in default location | | ||
| `-Dremote.cache.enabled=true/false` | Remote cache and associated features disabled/enabled | To remove noise from logs then remote cache is not available | | ||
| `-Dremote.cache.save.enabled=true/false` | Remote cache save allowed or not | To designate nodes which allowed to push in remote shared cache | | ||
| `-Dremote.cache.save.final=true/false` | Is it allowed or not to override produced cache | To ensure that reference build is not overriden by interim build | | ||
| `-Dremote.cache.failFast=true/false` | Fail on the first module whcih cannot be restored from cache | Remote cache setup/tuning/troubleshooting | | ||
| `-Dremote.cache.baselineUrl=<http url>` | Location of baseline build for comparison | Remote cache setup/tuning/troubleshooting | | ||
|
||
## Project level properties | ||
|
||
Project level parameters allow overriding global parameters on project level Must be specified as project properties: | ||
|
||
```xml | ||
|
||
<pom> | ||
... | ||
<properties> | ||
<remote.cache.input.glob>{*.css}</remote.cache.input.glob> | ||
</properties> | ||
</pom> | ||
``` | ||
|
||
| Parameter | Description | | ||
| ----------------------------- | ----------- | | ||
| `remote.cache.input.glob` | Project specific glob to select sources. Overrides global glob. | | ||
| `remote.cache.input` | Property prefix to mark paths which must be additionally scanned for source code. Value of property starting with this prefix will be treated as path relatively to current project/module | | ||
| `remote.cache.exclude` | Property prefix to mark paths which must be excluded from source code search. Value of property starting with this prefix will be treated as path to current project/module | | ||
| `remote.cache.processPlugins` | Introspect plugins to find inputs or not. Default is true. | |
Oops, something went wrong.