Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ama-sdk): support of URL as spec-path input #1572

Merged
merged 1 commit into from
Apr 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .yarnrc.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,5 @@ packageExtensions:
probot@*:
dependencies:
body-parser: ^1.20.2
bottleneck: ^2.19.5

yarnPath: .yarn/releases/yarn-4.1.1.cjs
7 changes: 3 additions & 4 deletions packages/@ama-sdk/create/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,7 @@
/* eslint-disable no-console */

import { execSync, spawnSync } from 'node:child_process';
import * as url from 'node:url';
import { dirname, join, relative, resolve } from 'node:path';
import { dirname, join, parse, relative, resolve } from 'node:path';
import * as minimist from 'minimist';

const packageManagerEnv = process.env.npm_config_user_agent?.split('/')[0];
Expand Down Expand Up @@ -74,7 +73,7 @@ const schematicArgs = [
const resolveTargetDirectory = resolve(process.cwd(), targetDirectory);

const run = () => {
const isSpecPathUrl = url.URL.canParse(argv['spec-path']);
const isSpecRelativePath = !!argv['spec-path'] && !parse(argv['spec-path']).root;

const runner = process.platform === 'win32' ? `${packageManager}.cmd` : packageManager;
const steps: { args: string[]; cwd?: string; runner?: string }[] = [
Expand All @@ -89,7 +88,7 @@ const run = () => {
binPath,
`${schematicsPackage}:typescript-core`,
...schematicArgs,
'--spec-path', isSpecPathUrl ? argv['spec-path'] : relative(resolveTargetDirectory, resolve(process.cwd(), argv['spec-path']))
'--spec-path', isSpecRelativePath ? relative(resolveTargetDirectory, resolve(process.cwd(), argv['spec-path'])) : argv['spec-path']
],
cwd: resolveTargetDirectory
}] : [])
Expand Down
59 changes: 43 additions & 16 deletions packages/@ama-sdk/schematics/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,27 +64,53 @@ npx -p @angular/cli ng add @ama-sdk/core

### How to use?

The typescript generator provides 2 generators:
The typescript generator provides 3 generators:

- **shell**: To generate the "shell" of an SDK package
- **core**: To (re)generate the SDK based on a specified Swagger spec
- **core**: To (re)generate the SDK based on a specified OpenApi specification
- **create**: To create a new SDK from scratch (i.e. chain **shell** and **core**)

To generate the `shell` you can run:
You can generate the `shell` in an existing monorepo with the command:
kpanot marked this conversation as resolved.
Show resolved Hide resolved

```shell
yarn schematics @ama-sdk/schematics:typescript-shell
#Monorepo with Otter:
yarn ng g sdk sdkName

# Monorepo without Otter;
yarn schematics @o3r/workspace:sdk sdkName
```

or from scratch, with the NPM initializer:

```shell
npm create @ama-sdk typescript <project-name>
```

If you use `Yarn2+`, you can use the following `scripts` in `package.json`:
The generated package comes with the following script in the package.json:

```json
"resolve": "node -e 'process.stdout.write(require.resolve(process.argv[1]));'",
"generate": "yarn schematics @ama-sdk/schematics:typescript-core --spec-path ./swagger-spec.yaml",
"upgrade:repository": "yarn schematics @ama-sdk/schematics:typescript-shell",
```json5
{
// ...
"generate": "yarn schematics @ama-sdk/schematics:typescript-core --spec-path ./openapi-spec.yaml",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably need to update it in the config we generate with the shell

"upgrade:repository": "yarn schematics @ama-sdk/schematics:typescript-shell"
}
```

Use `generate` to (re)generate your SDK based on the content of `./swagger-spec.yaml` (make sure you have this file at the root of your project) and `upgrade:repository` to regenerate the structure of your project.
> [!NOTE]
> Use `generate` to (re)generate your SDK based on the content of `./openapi-spec.yaml` (make sure you have this file at the root of your project) and `upgrade:repository` to regenerate the structure of your project.

> [!TIP]
> The `--spec-path` parameter supports YAML and JSON file formats based on the file system path or remote URL.

If you use `Yarn2+` with PnP, you can modify the following `scripts` in `package.json` to generate the SDK based on specifications in a dependency package:

```json5
{
// ...
"resolve": "node -e 'process.stdout.write(require.resolve(process.argv[1]));'",
"generate": "yarn schematics @ama-sdk/schematics:typescript-core --spec-path $(yarn resolve @my/dep/spec-file.yaml)",
}
```

#### Light SDK

Expand Down Expand Up @@ -130,14 +156,15 @@ yarn schematics @ama-sdk/schematics:typescript-core --spec-path ./swagger-spec.y

It is possible to configure the SDK code generation by passing parameters to the generator command line to override the default configuration values.
The available parameters are:

- `--spec-path`: Path to the swagger specification used to generate the SDK
- `--spec-config-path`: Path to the spec generation configuration
- `--global-property`: Comma separated string of options to give to the openapi-generator-cli
- `--output-path`: Output path for the generated SDK
- `--generator-custom-path`: Path to a custom generator

Also, another parameter is available called OpenAPI Normalizer which transforms the input OpenAPI specification (which may not perfectly conform) to make it workable
with OpenAPI Generator. There are several rules that are supported which can be found [here](https://openapi-generator.tech/docs/customization/#openapi-normalizer).
Also, another parameter is available called OpenAPI Normalizer which transforms the input OpenAPI specification (which may not perfectly conform) to make it workable
with OpenAPI Generator. There are several rules that are supported which can be found [here](https://openapi-generator.tech/docs/customization/#openapi-normalizer).

This parameter can be passed with `--openapi-normalizer` followed by the rules to be enabled in OpenAPI normalizer in the form of `RULE_1=true,RULE_2=original`.

Expand All @@ -157,7 +184,7 @@ There is also a possibility to configure the SDK code generation in `openapitool
"example-sdk": { // any name you like (can be referenced using --generator-key)
"generatorName": "typescriptFetch",
"output": ".",
"inputSpec": "./swagger-spec.yaml"
"inputSpec": "./swagger-spec.yaml" // supports YAML and JSON formats, based on the file system path or remote URL
}
}
}
Expand All @@ -174,7 +201,7 @@ The properties `generatorName`, `output`, and `inputSpec` are required and addit
[here](https://github.com/OpenAPITools/openapi-generator-cli/blob/master/apps/generator-cli/src/config.schema.json)). For example, we can add the previously
described global properties `stringifyDate` and `allowModelExtension`:

```json
```json5
{
"$schema": "https://raw.githubusercontent.com/OpenAPITools/openapi-generator-cli/master/apps/generator-cli/src/config.schema.json",
"generator-cli": {
Expand All @@ -184,7 +211,7 @@ described global properties `stringifyDate` and `allowModelExtension`:
"example-sdk": {
"generatorName": "typescriptFetch",
"output": ".",
"inputSpec": "./swagger-spec.yaml",
"inputSpec": "./openapi-spec.yaml", // or "./openapi-spec.json" according to the specification format
"globalProperty": {
"stringifyDate": true,
"allowModelExtension": true
Expand Down Expand Up @@ -228,7 +255,7 @@ You can use global property options to pass one or both of the following options
Example:

```shell
yarn schematics @ama-sdk/schematics:typescript-core --spec-path ./swagger-spec.yaml --global-property debugModels,debugOperations
yarn schematics @ama-sdk/schematics:typescript-core --spec-path ./openapi-spec.yaml --global-property debugModels,debugOperations
```

You can also use npx instead of yarn in the command.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ export type CodegenTaskOptions = {
* As is, the CodeGenerator does not implement any actual code generation and needs to be extended to be functional
* @see {@link OpenApiCliGenerator}
*/
export class CodeGenerator<T extends CodegenTaskOptions> {
export abstract class CodeGenerator<T extends CodegenTaskOptions> {
matthieu-crouzet marked this conversation as resolved.
Show resolved Hide resolved
/**
* Refers to the name the {@link Task} will be identified in a {@link Rule} ${@link SchematicContext}
*/
protected generatorName = '';
protected abstract generatorName: string;

/**
* Configure the code generation task
Expand Down Expand Up @@ -73,9 +73,7 @@ export class CodeGenerator<T extends CodegenTaskOptions> {
/**
* Returns the generator specific default options
*/
protected getDefaultOptions(): T {
throw new Error('No implementation, please target an implementation');
}
protected abstract getDefaultOptions(): T;

/**
* Returns the schematic that will run the code generator
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,28 @@ describe('Typescript Core Generator', () => {
expect(tree.readContent('/readme.md')).toContain('Based on OpenAPI spec 1.0.0');
});

it('should update openapitools file with yaml', async () => {
const runner = new SchematicTestRunner('@ama-sdk/schematics', collectionPath);
const tree = await runner.runSchematic('typescript-core', {
specPath: path.join(__dirname, '..', '..', '..', 'testing', 'MOCK_swagger.yaml')
}, baseTree);
const content: any = tree.readJson('/openapitools.json');

expect(content['generator-cli'].generators['test-sdk-sdk'].inputSpec.endsWith('openapi.yaml')).toBe(true);
expect(tree.exists('/openapi.yaml')).toBe(true);
});

it('should update openapitools file with json', async () => {
const runner = new SchematicTestRunner('@ama-sdk/schematics', collectionPath);
const tree = await runner.runSchematic('typescript-core', {
specPath: path.join(__dirname, '..', '..', '..', 'testing', 'MOCK_swagger.json')
}, baseTree);
const content: any = tree.readJson('/openapitools.json');

expect(content['generator-cli'].generators['test-sdk-sdk'].inputSpec.endsWith('openapi.json')).toBe(true);
expect(tree.exists('/openapi.json')).toBe(true);
});

it('should clean previous install', async () => {
baseTree.create('/src/api/my-apy/test.ts', 'fake module');
const runner = new SchematicTestRunner('@ama-sdk/schematics', collectionPath);
Expand Down
72 changes: 53 additions & 19 deletions packages/@ama-sdk/schematics/schematics/typescript/core/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,9 @@ import {
url
} from '@angular-devkit/schematics';
import type { Operation, PathObject } from '@ama-sdk/core';
import { readFileSync } from 'node:fs';
import { existsSync, readFileSync } from 'node:fs';
import * as path from 'node:path';
import { URL } from 'node:url';
import * as semver from 'semver';
import * as sway from 'sway';

Expand All @@ -24,6 +25,7 @@ import { OpenApiCliGenerator } from '../../code-generator/open-api-cli-generator

const JAVA_OPTIONS = ['specPath', 'specConfigPath', 'globalProperty', 'outputPath'];
const OPEN_API_TOOLS_OPTIONS = ['generatorName', 'output', 'inputSpec', 'config', 'globalProperty'];
const LOCAL_SPEC_FILENAME = 'openapi';

interface OpenApiToolsGenerator {
/** Location of the OpenAPI spec, as URL or file */
Expand Down Expand Up @@ -166,24 +168,50 @@ const getGeneratorOptions = (tree: Tree, context: SchematicContext, options: NgG
*/
function ngGenerateTypescriptSDKFn(options: NgGenerateTypescriptSDKCoreSchematicsSchema): Rule {

return (tree, context) => {
return async (tree, context) => {
const targetPath = options.directory || '';
const generatorOptions = getGeneratorOptions(tree, context, options);
let isJson = false;
let specDefaultPath = path.posix.join(targetPath, `${LOCAL_SPEC_FILENAME}.json`);
specDefaultPath = existsSync(specDefaultPath) ? specDefaultPath : path.posix.join(targetPath, `${LOCAL_SPEC_FILENAME}.yaml`);
generatorOptions.specPath ||= specDefaultPath;

let specContent!: string;
if (URL.canParse(generatorOptions.specPath) && (new URL(generatorOptions.specPath)).protocol.startsWith('http')) {
cpaulve-1A marked this conversation as resolved.
Show resolved Hide resolved
cpaulve-1A marked this conversation as resolved.
Show resolved Hide resolved
specContent = await (await fetch(generatorOptions.specPath)).text();
} else {
specContent = readFileSync(generatorOptions.specPath, {encoding: 'utf-8'}).toString();
}

try {
JSON.parse(specContent);
isJson = true;
} catch (e) {
isJson = false;
}
const defaultFileName = `${LOCAL_SPEC_FILENAME}.${isJson ? 'json' : 'yaml'}`;
specDefaultPath = path.posix.join(targetPath, defaultFileName);
generatorOptions.specPath = specDefaultPath;

if (tree.exists(specDefaultPath)) {
tree.overwrite(specDefaultPath, specContent);
} else {
tree.create(specDefaultPath, specContent);
}

/**
* rule to clear previous SDK generation
*/
const clearGeneratedCode = () => {
const clearGeneratedCode: Rule = () => {
treeGlob(tree, path.posix.join(targetPath, 'src', 'api', '**', '*.ts')).forEach((file) => tree.delete(file));
treeGlob(tree, path.posix.join(targetPath, 'src', 'models', 'base', '**', '!(index).ts')).forEach((file) => tree.delete(file));
treeGlob(tree, path.posix.join(targetPath, 'src', 'spec', '!(operation-adapter|index).ts')).forEach((file) => tree.delete(file));
return tree;
};

const generateOperationFinder = async (): Promise<PathObject[]> => {
const swayOptions = {
definition: path.resolve(generatorOptions.specPath!)
};
const definition: any = isJson ? tree.readJson(specDefaultPath) : (await import('js-yaml')).load(tree.readText(specDefaultPath));
const swayOptions = { definition };
const swayApi = await sway.create(swayOptions);
const extraction = swayApi.getPaths().map((obj) => ({
path: obj.path,
Expand All @@ -196,7 +224,7 @@ function ngGenerateTypescriptSDKFn(options: NgGenerateTypescriptSDKCoreSchematic
/**
* rule to update readme and generate mandatory code source
*/
const generateSource = async () => {
const generateSource: Rule = async () => {
const pathObjects = await generateOperationFinder();
const swayOperationAdapter = `[${pathObjects.map((pathObj) => getPathObjectTemplate(pathObj)).join(',')}]`;

Expand All @@ -212,30 +240,35 @@ function ngGenerateTypescriptSDKFn(options: NgGenerateTypescriptSDKCoreSchematic
};

/**
* Update local swagger spec file
* Update readme version
*/
const updateSpec = () => {
const updateSpecVersion: Rule = () => {
const readmeFile = path.posix.join(targetPath, 'readme.md');
const specContent = readFileSync(generatorOptions.specPath!).toString();
if (tree.exists(readmeFile)) {
const specVersion = /version: *([0-9]+\.[0-9]+\.[0-9]+(?:-[^ ]+)?)/.exec(specContent);

if (specVersion) {
const readmeContent = tree.read(readmeFile)!.toString('utf8');
tree.overwrite(readmeFile, readmeContent.replace(/Based on (OpenAPI|Swagger) spec .*/i, `Based on $1 spec ${specVersion[1]}`));
tree.overwrite(readmeFile, readmeContent.replace(/Based on (.+) spec .*/i, `Based on $1 spec ${specVersion[1]}`));
}
}
return tree;
};

if (tree.exists(path.posix.join(targetPath, 'swagger-spec.yaml'))) {
tree.overwrite(path.posix.join(targetPath, 'swagger-spec.yaml'), specContent);
} else {
tree.create(path.posix.join(targetPath, 'swagger-spec.yaml'), specContent);
const adaptDefaultFile: Rule = () => {
const openApiToolsPath = path.posix.join(targetPath, 'openapitools.json');
if (tree.exists(openApiToolsPath)) {
const openApiTools: any = tree.readJson(openApiToolsPath);
Object.keys(openApiTools['generator-cli']?.generators)
.filter((key) => !openApiTools['generator-cli'].generators[key].inputSpec)
.forEach((key) => openApiTools['generator-cli'].generators[key].inputSpec = `./${defaultFileName}`);
tree.overwrite(openApiToolsPath, JSON.stringify(openApiTools, null, 2));
}
return () => tree;
return tree;
};

const runGeneratorRule = () => {
return () => (new OpenApiCliGenerator(options)).getGeneratorRunSchematic(
const runGeneratorRule: Rule = () => {
return (new OpenApiCliGenerator(options)).getGeneratorRunSchematic(
(options.generatorKey && JAVA_OPTIONS.every((optionName) => options[optionName] === undefined)) ?
{
generatorKey: options.generatorKey,
Expand All @@ -249,7 +282,8 @@ function ngGenerateTypescriptSDKFn(options: NgGenerateTypescriptSDKCoreSchematic
return chain([
clearGeneratedCode,
generateSource,
updateSpec,
updateSpecVersion,
adaptDefaultFile,
runGeneratorRule
]);
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
"<%=projectName%>-<%=projectPackageName%>": {
"generatorName": "typescriptFetch",
"output": ".",
"inputSpec": "./swagger-spec.yaml"
"inputSpec": ""
cpaulve-1A marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
Expand Down
Loading
Loading