Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Azure SDK Maven build tool #26771

Merged
merged 9 commits into from
Feb 15, 2022
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions sdk/tools/azure-sdk-build-tool/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# Release History

## 1.0.0-beta.1 (Unreleased)

### Features Added
41 changes: 41 additions & 0 deletions sdk/tools/azure-sdk-build-tool/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Azure SDK Maven Build Tool

The Azure SDK for Java project ships a Maven build tool that developers can choose to include in their projects. This tool runs locally and does not transmit any data to Microsoft. It can be configured to generate a report or fail the build when certain conditions are met, which is useful to ensure compliance with numerous best practices. These include:

- Validating the correct use of the azure-sdk-for-java BOM, including using the latest version and relying on it to
define dependency versions on Azure SDK for Java client libraries.
- Validating that historical Azure client libraries are not being used when newer and improved versions exist.
- Providing insight into usage of beta APIs.

The build tool can be configured in a project Maven POM file as such:

```xml
<build>
<plugins>
<plugin>
<groupId>com.azure.tools</groupId>
<artifactId>azure-sdk-build-tool</artifactId>
<version>{latest_version}</version>
<configuration>
...
</configuration>
</plugin>
</plugins>
</build>
```
Within the configuration section, it is possible to configure the settings in the table below if desired, but by default they are configured with the recommended settings. Because of this, it is ok to not have any configuration specified at all.


| Property | Default Value | Description |
|------------------------------------------|---------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| validateAzureSdkBomUsed | true | Ensures that the build has the azure-sdk-for-java BOM referenced appropriately, so that Azure SDK for Java client library dependencies may take their versions from the BOM. |
| validateBomVersionsAreUsed | true | Ensures that where a dependency is available from the azure-sdk-for-java BOM the version is not being manually overridden. |
| validateNoDeprecatedMicrosoftLibraryUsed | true | Ensures that the project does not make use of previous-generation Azure libraries. Using the new and previous-generation libraries in a single project is unlikely to cause any issue, but is will result in a sub-optimal developer experience. |
| validateNoBetaLibraryUsed | false | Some Azure SDK for Java client libraries have beta releases, with version strings in the form x.y.z-beta.n. Enabling this feature will ensure that no beta libraries are being used. |
| validateNoBetaAPIUsed | true | Azure SDK for Java client libraries sometimes do GA releases with methods annotated with @Beta. This check looks to see if any such methods are being used. |
| validateLatestBomVersionUsed | true | Ensures that dependencies are kept up to date by reporting back (or failing the build) if a newer azure-sdk-for-java BOM exists. |
| reportFile | "" | (Optional) Specifies the location to write the build report out to, in JSON format. If not specified, no report will be written (and a summary of the build, or the appropriate build failures), will be shown in the terminal. |
After adding the build tool into a Maven project, the tool can be run by calling `mvn compile azure:run`. Depending on
the configuration provided, you can expect to see build failures or report files generated that can inform you about potential issues before they become more serious.

As the build tool evolves, new releases will be published, and it is recommended that developers frequently check for new releases and update as appropriate.
153 changes: 153 additions & 0 deletions sdk/tools/azure-sdk-build-tool/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>com.azure.tools</groupId>
<artifactId>azure-sdk-build-tool</artifactId>
<packaging>maven-plugin</packaging>
<version>1.0.0-beta.1</version>

<name>Azure Maven Build Tool</name>
srnagar marked this conversation as resolved.
Show resolved Hide resolved
<description>A tool that makes working with the Azure SDK for Java more productive.</description>

<distributionManagement>
<snapshotRepository>
<id>ossrh</id>
<url>https://oss.sonatype.org/content/repositories/snapshots</url>
</snapshotRepository>
<repository>
<id>ossrh</id>
<url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
</repository>
</distributionManagement>

<url>https://github.com/azure/azure-sdk-for-java</url>
<organization>
<name>Microsoft Corporation</name>
<url>http://microsoft.com</url>
</organization>
<licenses>
<license>
<name>The MIT License (MIT)</name>
<url>http://opensource.org/licenses/MIT</url>
<distribution>repo</distribution>
</license>
</licenses>

<developers>
<developer>
<id>microsoft</id>
<name>Microsoft Corporation</name>
</developer>
</developers>

<issueManagement>
<system>GitHub</system>
<url>https://github.com/Azure/azure-sdk-for-java/issues</url>
</issueManagement>

<scm>
<url>https://github.com/Azure/azure-sdk-for-java</url>
<connection>scm:git:https://github.com/Azure/azure-sdk-for-java.git</connection>
<developerConnection/>
<tag>HEAD</tag>
</scm>

<properties>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties>

<dependencies>
<!-- Support for running as a Maven plugin -->
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-plugin-api</artifactId>
<version>3.8.1</version> <!-- {x-version-update;org.apache.maven:maven-plugin-api;external_dependency} -->
</dependency>
<dependency>
<groupId>org.apache.maven.plugin-tools</groupId>
<artifactId>maven-plugin-annotations</artifactId>
<version>3.6.0</version> <!-- {x-version-update;org.apache.maven.plugin-tools:maven-plugin-annotations;external_dependency} -->
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-project</artifactId>
<version>2.2.1</version> <!-- {x-version-update;org.apache.maven:maven-project;external_dependency} -->
<exclusions>
<exclusion>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>com.github.javaparser</groupId>
<artifactId>javaparser-core</artifactId>
<version>3.20.2</version> <!-- {x-version-update;com.github.javaparser:javaparser-core;external_dependency} -->
</dependency>

<!-- Annotation scanning -->
<dependency>
<groupId>net.oneandone.reflections8</groupId>
<artifactId>reflections8</artifactId>
<version>0.11.7</version> <!-- {x-version-update;net.oneandone.reflections8:reflections8;external_dependency} -->
</dependency>

<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-monitor-opentelemetry-exporter</artifactId> <!-- {x-version-update;com.azure:azure-monitor-opentelemetry-exporter;dependency} -->
<version>1.0.0-beta.5</version>
</dependency>

<!-- Unit Test -->
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>5.8.2</version> <!-- {x-version-update;org.junit.jupiter:junit-jupiter-api;external_dependency} -->
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.8.2</version> <!-- {x-version-update;org.junit.jupiter:junit-jupiter-engine;external_dependency} -->
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-params</artifactId>
<version>5.8.2</version> <!-- {x-version-update;org.junit.jupiter:junit-jupiter-params;external_dependency} -->
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.22.2</version> <!-- {x-version-update;org.apache.maven.plugins:maven-surefire-plugin;external_dependency} -->
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-plugin-plugin</artifactId>
<version>3.6.0</version> <!-- {x-version-update;org.apache.maven.plugins:maven-plugin-plugin;external_dependency} -->
<executions>

<execution>
<id>mojo-descriptor</id>
<goals>
<goal>descriptor</goal>
</goals>
</execution>
</executions>
<configuration>
<goalPrefix>azure</goalPrefix>
</configuration>
</plugin>
</plugins>
</build>

</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
package com.azure.sdk.build.tool;

import com.azure.sdk.build.tool.mojo.AzureSdkMojo;
import com.azure.sdk.build.tool.util.AnnotatedMethodCallerResult;
import com.azure.sdk.build.tool.util.logging.Logger;

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.Comparator;
import java.util.Optional;
import java.util.Set;
import java.util.TreeSet;
import java.util.stream.Stream;

import static com.azure.sdk.build.tool.util.AnnotationUtils.findCallsToAnnotatedMethod;
import static com.azure.sdk.build.tool.util.AnnotationUtils.getAnnotation;
import static com.azure.sdk.build.tool.util.AnnotationUtils.getCompleteClassLoader;
import static com.azure.sdk.build.tool.util.MojoUtils.failOrError;
import static com.azure.sdk.build.tool.util.MojoUtils.getAllDependencies;
import static com.azure.sdk.build.tool.util.MojoUtils.getCompileSourceRoots;
import static com.azure.sdk.build.tool.util.MojoUtils.getString;

/**
* Performs the following tasks:
*
* <ul>
* <li>Reporting to the user all use of @ServiceMethods.</li>
* <li>Reporting to the user all use of @Beta-annotated APIs.</li>
* </ul>
*/
public class AnnotationProcessingTool implements Runnable {
private static Logger LOGGER = Logger.getInstance();

/**
* Runs the annotation processing task to look for @ServiceMethod and @Beta usage.
*/
public void run() {
LOGGER.info("Running Annotation Processing Tool");

// We build up a list of packages in the source of the user maven project, so that we only report on the
// usage of annotation methods from code within these packages
final Set<String> interestedPackages = new TreeSet<>(Comparator.comparingInt(String::length));
getCompileSourceRoots().forEach(root -> buildPackageList(root, root, interestedPackages));

final ClassLoader classLoader = getCompleteClassLoader(getAllPaths());

// Collect all calls to methods annotated with the Azure SDK @ServiceMethod annotation
getAnnotation("com.azure.core.annotation.ServiceMethod", classLoader)
.map(a -> findCallsToAnnotatedMethod(a, getAllPaths(), interestedPackages, true))
.ifPresent(AzureSdkMojo.MOJO.getReport()::setServiceMethodCalls);

// Collect all calls to methods annotated with the Azure SDK @Beta annotation
Optional<Set<AnnotatedMethodCallerResult>> annotatedMethodCallerResults = getAnnotation("com.azure.cosmos.util.Beta", classLoader)
.map(a -> findCallsToAnnotatedMethod(a, getAllPaths(), interestedPackages, true));
if (annotatedMethodCallerResults.isPresent()) {
Set<AnnotatedMethodCallerResult> betaMethodCallers = annotatedMethodCallerResults.get();
AzureSdkMojo.MOJO.getReport().setBetaMethodCalls(betaMethodCallers);
if (!betaMethodCallers.isEmpty()) {
StringBuilder message = new StringBuilder();
message.append(getString("betaApiUsed")).append(System.lineSeparator());
betaMethodCallers.forEach(method -> message.append(" - ").append(method.toString()).append(System.lineSeparator()));
failOrError(() -> AzureSdkMojo.MOJO.isValidateNoBetaApiUsed(), message.toString());
}
}
}

private static Stream<Path> getAllPaths() {
// This is the user maven build target directory - we look in here for the compiled source code
final File targetDir = new File(AzureSdkMojo.MOJO.getProject().getBuild().getDirectory() + "/classes/");

// this stream of paths is a stream containing the users maven project compiled class files, as well as all
// jar file dependencies. We use this to analyse the use of annotations and report back to the user.
return Stream.concat(
Stream.of(targetDir.getAbsolutePath()),
getAllDependencies().stream().map(a -> a.getFile().getAbsolutePath()))
.map(Paths::get);
}

private static void buildPackageList(String rootDir, String currentDir, Set<String> packages) {
final File directory = new File(currentDir);

final File[] files = directory.listFiles();
if (files == null) {
return;
}

for (final File file : files) {
if (file.isFile()) {
final String path = file.getPath();
final String packageName = path.substring(rootDir.length() + 1, path.lastIndexOf(File.separator));
packages.add(packageName.replace(File.separatorChar, '.'));
} else if (file.isDirectory()) {
buildPackageList(rootDir, file.getAbsolutePath(), packages);
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,81 @@
package com.azure.sdk.build.tool;

import com.azure.sdk.build.tool.models.OutdatedDependency;
import com.azure.sdk.build.tool.util.MavenUtils;
import com.azure.sdk.build.tool.util.logging.Logger;
import edu.emory.mathcs.backport.java.util.Collections;
import org.apache.maven.artifact.Artifact;

import java.util.Arrays;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.stream.Collectors;

/**
* Contains the mapping for outdated dependencies and it's replacement.
*/
public class AzureDependencyMapping {
private static Logger LOGGER = Logger.getInstance();

private static final String TRACK_ONE_GROUP_ID = "com.microsoft.azure";
private static final String TRACK_TWO_GROUP_ID = "com.azure";

// This map is for all com.microsoft.* group IDs, mapping them into their com.azure equivalents
private static final Map<String, List<String>> TRACK_ONE_REDIRECTS = new HashMap<>();
static {
// Cosmos
TRACK_ONE_REDIRECTS.put("azure-cosmosdb", Collections.singletonList("azure-cosmos"));

// Key Vault - Track 1 KeyVault library is split into three Track 2 libraries
TRACK_ONE_REDIRECTS.put("azure-keyvault",
Arrays.asList("azure-security-keyvault-keys",
"azure-security-keyvault-certificates",
"azure-security-keyvault-secrets"));

// Blob Storage
TRACK_ONE_REDIRECTS.put("azure-storage-blob", Collections.singletonList("azure-storage-blob"));

// Event Hubs
TRACK_ONE_REDIRECTS.put("azure-eventhubs", Collections.singletonList("azure-messaging-eventhubs"));
TRACK_ONE_REDIRECTS.put("azure-eventhubs-eph", Collections.singletonList("azure-messaging-eventhubs-checkpointstore-blob"));

// Service Bus
TRACK_ONE_REDIRECTS.put("azure-servicebus", Collections.singletonList("azure-messaging-servicebus"));

// Event Grid
TRACK_ONE_REDIRECTS.put("azure-eventgrid", Collections.singletonList("azure-messaging-eventgrid"));

// Log Analytics
TRACK_ONE_REDIRECTS.put("azure-loganalytics", Collections.singletonList("azure-monitor-query"));
}

/**
* This method will look to see if we have any recorded guidance on how to replace the given artifact with
* something else.
*
* @param artifact The artifact for which we want to find a replacement
* @return An {@link OutdatedDependency} if a replacement for the given {@code artifact} exists.
*/
public static Optional<OutdatedDependency> lookupReplacement(Artifact artifact) {
String groupId = artifact.getGroupId();
String artifactId = artifact.getArtifactId();

if (TRACK_ONE_GROUP_ID.equals(groupId)) {
if (TRACK_ONE_REDIRECTS.containsKey(artifactId)) {
final List<String> newArtifactIds = TRACK_ONE_REDIRECTS.get(artifactId);

List<String> newGavs = newArtifactIds.stream()
.map(newArtifactId -> TRACK_TWO_GROUP_ID + ":" + newArtifactId + ":" + MavenUtils.getLatestArtifactVersion(TRACK_TWO_GROUP_ID, newArtifactId)).collect(Collectors.toList());
return Optional.of(new OutdatedDependency(MavenUtils.toGAV(artifact), newGavs));
} else {
// we've hit artifact location where we don't know know if the com.microsoft.azure artifact has artifact newer
// replacement...For now we will not give artifact failure
return Optional.empty();
}
}

return Optional.empty();
}
}
Loading