[cli] new 'author template' command

docs/templating.md

@@ -50,13 +50,41 @@ java -cp /path/totemplate-classpath-example-1.0-SNAPSHOT.jar:modules/openapi-gen
 
 Note that our template directory is relative to the resource directory of the JAR defined on the classpath.
 
+### Retrieving Templates
+
+You will need to find and retrieve the templates for your desired generator in order to redefine structures, documentation, or API logic. We cover template customization in the following sections.
+
+In OpenAPI Generator 5.0 and later, you can use the CLI command `author template` to extract embedded templates for your target generator. For example:
+
+```
+openapi-generator author template -g java --library webclient
+```
+
+For OpenAPI Generator versions prior to 5.0, you will want to find the [resources directory](https://github.com/OpenAPITools/openapi-generator/tree/master/modules/openapi-generator/src/main/resources) for the generator you want to extend. This is generally easy to find as directories commonly follow the convention of `resources/<generator name>`. In cases where you're unsure, you will need to find the `embeddedTemplateDir` assignment in your desired generator. This is almost always assigned in the constructor of the generator class. The C# .Net Core generator assigns this as:
+
+```
+embeddedTemplateDir = templateDir = "csharp-netcore";
+```
+
+These templates are in our source repository at [modules/openapi-generator/src/main/resources/csharp-netcore](https://github.com/OpenAPITools/openapi-generator/tree/master/modules/openapi-generator/src/main/resources/csharp-netcore). Be sure to select the tag or branch for the version of OpenAPI Generator you're using before grabbing the templates.
+
+**NOTE** If you have specific logic you'd like to modify such as modifying the generated README, you _only_ need to pull and modify this individual template. OpenAPI Generator will lookup templates in this order:
+
+* User customized library path (e.g. `custom_template/libraries/feign/model.mustache`)
+* User customized generator top-level path (e.g. `custom_template/model.mustache`)
+* Embedded library path (e.g. `resources/Java/libraries/feign/model.mustache`)
+* Embedded top-level path (e.g. `resources/Java/model.mustache`)
+* Common embedded path (e.g. `resources/_common/model.mustache`)
+
 ### Custom Logic
 
 For this example, let's modify a Java client to use AOP via [jcabi/jcabi-aspects](https://github.com/jcabi/jcabi-aspects). We'll log API method execution at the `INFO` level. The jcabi-aspects project could also be used to implement method retries on failures; this would be a great exercise to further play around with templating.
 
 The Java generator supports a `library` option. This option works by defining base templates, then applying library-specific template overrides. This allows for template reuse for libraries sharing the same programming language. Templates defined as a library need only modify or extend the templates concerning the library, and generation falls back to the root templates (the "defaults") when not extended by the library. Generators which support the `library` option will only support the libraries known by the generator at compile time, and will throw a runtime error if you try to provide a custom library name.
 
-To get started, we will need to copy our target generator's directory in full. The directory will be located under `modules/opeanpi-generator/src/main/resources/{generator}`. In general, the generator directory matches the generator name (what you would pass to the `generator` option), but this is not a requirement-- if you are having a hard time finding the template directory, look at the `embeddedTemplateDir` option in your target generator's implementation.
+To get started, we will need to copy our target generator's directory in full.
+
+The directory will be located under `modules/opeanpi-generator/src/main/resources/{generator}`. In general, the generator directory matches the generator name (what you would pass to the `generator` option), but this is not a requirement-- if you are having a hard time finding the template directory, look at the `embeddedTemplateDir` option in your target generator's implementation.
 
 If you've already cloned openapi-generator, find and copy the `modules/opeanpi-generator/src/main/resources/Java` directory. If you have the [Refined GitHub](https://github.com/sindresorhus/refined-github) Chrome or Firefox Extension, you can navigate to this directory on GitHub and click the "Download" button. Or, to pull the directory from latest master:
 

docs/usage.md

@@ -15,13 +15,14 @@ openapi-generator help
 usage: openapi-generator-cli <command> [<args>]
 
 The most commonly used openapi-generator-cli commands are:
+    author        Utilities for authoring generators or customizing templates.
     config-help   Config help for chosen lang
     generate      Generate code with the specified generator.
-    help          Display help information
+    help          Display help information about openapi-generator
     list          Lists the available generators
     meta          MetaGenerator. Generator for creating a new template set and configuration for Codegen.  The output will be based on the language you specify, and includes default templates to include.
     validate      Validate specification
-    version       Show version information
+    version       Show version information used in tooling
 
 See 'openapi-generator-cli help <command>' for more information on a specific
 command.
@@ -670,3 +671,87 @@ EOF
 openapi-generator batch *.yaml
 ```
 
+## author
+
+This command group contains utilities for authoring generators or customizing templates.
+
+```
+openapi-generator help author
+NAME
+        openapi-generator-cli author - Utilities for authoring generators or
+        customizing templates.
+
+SYNOPSIS
+        openapi-generator-cli author
+        openapi-generator-cli author template [(-v | --verbose)]
+                [(-o <output directory> | --output <output directory>)]
+                [--library <library>]
+                (-g <generator name> | --generator-name <generator name>)
+
+OPTIONS
+        --help
+            Display help about the tool
+
+        --version
+            Display full version output
+
+COMMANDS
+        With no arguments, Display help information about openapi-generator
+
+        template
+            Retrieve templates for local modification
+
+            With --verbose option, verbose mode
+
+            With --output option, where to write the template files (defaults to
+            'out')
+
+            With --library option, library template (sub-template)
+
+            With --generator-name option, generator to use (see list command for
+            list)
+```
+
+### template
+
+This command allows user to extract templates from the CLI jar which simplifies customization efforts.
+
+```
+NAME
+        openapi-generator-cli author template - Retrieve templates for local
+        modification
+
+SYNOPSIS
+        openapi-generator-cli author template
+                (-g <generator name> | --generator-name <generator name>)
+                [--library <library>]
+                [(-o <output directory> | --output <output directory>)]
+                [(-v | --verbose)]
+
+OPTIONS
+        -g <generator name>, --generator-name <generator name>
+            generator to use (see list command for list)
+
+        --library <library>
+            library template (sub-template)
+
+        -o <output directory>, --output <output directory>
+            where to write the template files (defaults to 'out')
+
+        -v, --verbose
+            verbose mode
+```
+
+Example:
+
+Extract Java templates, limiting to the `webclient` library.
+
+```
+openapi-generator author template -g java --library webclient
+```
+
+Extract all Java templates:
+
+```
+openapi-generator author template -g java
+```

modules/openapi-generator-cli/src/main/java/org/openapitools/codegen/OpenAPIGenerator.java

@@ -57,6 +57,11 @@ public static void main(String[] args) {
                                 GenerateBatch.class
                         );
 
+        builder.withGroup("author")
+                .withDescription("Utilities for authoring generators or customizing templates.")
+                .withDefaultCommand(HelpCommand.class)
+                .withCommands(AuthorTemplate.class);
+
         try {
             builder.build().parse(args).run();
 

modules/openapi-generator-cli/src/main/java/org/openapitools/codegen/cmd/AuthorTemplate.java

@@ -0,0 +1,168 @@
+package org.openapitools.codegen.cmd;
+
+import io.airlift.airline.Command;
+import io.airlift.airline.Option;
+import org.apache.commons.lang3.StringUtils;
+import org.openapitools.codegen.CodegenConfig;
+import org.openapitools.codegen.CodegenConfigLoader;
+import org.openapitools.codegen.CodegenConstants;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import java.io.File;
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.nio.file.*;
+import java.nio.file.spi.FileSystemProvider;
+import java.util.*;
+import java.util.regex.Pattern;
+import java.util.stream.Stream;
+
+@SuppressWarnings({"FieldMayBeFinal", "FieldCanBeLocal", "unused"})
+@Command(name = "template", description = "Retrieve templates for local modification")
+public class AuthorTemplate extends OpenApiGeneratorCommand {
+
+    private static final Logger LOGGER = LoggerFactory.getLogger(AuthorTemplate.class);
+
+    @Option(name = {"-g", "--generator-name"}, title = "generator name",
+            description = "generator to use (see list command for list)",
+            required = true)
+    private String generatorName;
+
+    @Option(name = {"--library"}, title = "library", description = CodegenConstants.LIBRARY_DESC)
+    private String library;
+
+    @Option(name = {"-o", "--output"}, title = "output directory",
+            description = "where to write the template files (defaults to 'out')")
+    private String output = "";
+
+    @Option(name = {"-v", "--verbose"}, description = "verbose mode")
+    private boolean verbose;
+
+    private Pattern pattern = null;
+
+    @Override
+    void execute() {
+        CodegenConfig config = CodegenConfigLoader.forName(generatorName);
+        String templateDirectory = config.templateDir();
+
+        log("Requesting '{}' from embedded resource directory '{}'", generatorName, templateDirectory);
+
+        Path embeddedTemplatePath;
+        try {
+            URI uri = Objects.requireNonNull(this.getClass().getClassLoader().getResource(templateDirectory)).toURI();
+
+            if ("jar".equals(uri.getScheme())) {
+                Optional<FileSystemProvider> provider = FileSystemProvider.installedProviders()
+                        .stream()
+                        .filter(p -> p.getScheme().equalsIgnoreCase("jar"))
+                        .findFirst();
+
+                if (!provider.isPresent()) {
+                    throw new ProviderNotFoundException("Unable to load jar file system provider");
+                }
+
+                try {
+                    provider.get().getFileSystem(uri);
+                } catch (FileSystemNotFoundException ex) {
+                    // File system wasn't loaded, so create it.
+                    provider.get().newFileSystem(uri, Collections.emptyMap());
+                }
+            }
+
+            embeddedTemplatePath = Paths.get(uri);
+
+            log("Copying from jar location {}", embeddedTemplatePath.toAbsolutePath().toString());
+
+            File outputDir;
+            if (StringUtils.isNotEmpty(output)) {
+                outputDir = new File(output);
+            } else {
+                outputDir = new File("out");
+            }
+
+            Path outputDirPath = outputDir.toPath();
+            if (!Files.exists(outputDirPath)) {
+                Files.createDirectories(outputDirPath);
+            }
+            List<Path> generatedFiles = new ArrayList<>();
+            try (final Stream<Path> templates = Files.walk(embeddedTemplatePath)) {
+                templates.forEach(template -> {
+                    log("Found template: {}", template.toAbsolutePath());
+                    Path relativePath = embeddedTemplatePath.relativize(template);
+                    if (shouldCopy(relativePath)) {
+                        Path target = outputDirPath.resolve(relativePath.toString());
+                        generatedFiles.add(target);
+                        try {
+                            if (Files.isDirectory(template)) {
+                                if (Files.notExists(target)) {
+                                    log("Creating directory: {}", target.toAbsolutePath());
+                                    Files.createDirectories(target);
+                                }
+                            } else {
+                                if (target.getParent() != null && Files.notExists(target.getParent())) {
+                                    log("Creating directory: {}", target.getParent());
+                                    Files.createDirectories(target.getParent());
+                                }
+                                log("Copying to: {}", target.toAbsolutePath());
+                                Files.copy(template, target, StandardCopyOption.REPLACE_EXISTING);
+                            }
+                        } catch (IOException e) {
+                            LOGGER.error("Unable to create target location '{}'.", target);
+                        }
+                    } else {
+                        log("Directory is excluded by library option: {}", relativePath);
+                    }
+                });
+            }
+
+            if (StringUtils.isNotEmpty(library) && !generatedFiles.isEmpty()) {
+                Path librariesPath = outputDirPath.resolve("libraries");
+                Path targetLibrary = librariesPath.resolve(library);
+                String librariesPrefix = librariesPath.toString();
+                if (!Files.isDirectory(targetLibrary)) {
+                    LOGGER.warn("The library '{}' was not extracted. Please verify the spelling and retry.", targetLibrary);
+                }
+                generatedFiles.stream()
+                        .filter(p -> p.startsWith(librariesPrefix))
+                        .forEach(p -> {
+                            if (p.startsWith(targetLibrary)) {
+                                // We don't care about empty directories, and not need to check directory for files.
+                                if (!Files.isDirectory(p)) {
+                                    // warn if the file was not written
+                                    if (Files.notExists(p)) {
+                                        LOGGER.warn("An expected library file was not extracted: {}", p.toAbsolutePath());
+                                    }
+                                }
+                            } else {
+                                LOGGER.warn("The library filter '{}' extracted an unexpected library path: {}", library, p.toAbsolutePath());
+                            }
+                });
+            }
+
+            LOGGER.info("Extracted templates to '{}' directory. Refer to https://openapi-generator.tech/docs/templating for customization details.", outputDirPath);
+        } catch (URISyntaxException | IOException e) {
+            LOGGER.error("Unable to load embedded template directory.", e);
+        }
+    }
+
+    private void log(String format, Object... arguments) {
+        if (verbose) {
+            LOGGER.info(format, arguments);
+        }
+    }
+
+    private boolean shouldCopy(Path relativePath) {
+        String path = relativePath.toString();
+        if (StringUtils.isNotEmpty(library) && path.contains("libraries")) {
+            if (pattern == null) {
+                pattern = Pattern.compile(String.format(Locale.ROOT, "libraries[/\\\\]{1}%s[/\\\\]{1}.*", Pattern.quote(library)));
+            }
+
+            return pattern.matcher(path).matches();
+        }
+
+        return true;
+    }
+}