Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/plugin #1258

Merged
merged 29 commits into from
Jun 15, 2023
Merged
Show file tree
Hide file tree
Changes from 13 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
9985902
Add OpenAPI formatter for PS
peombwa May 8, 2023
f4dee06
Use strongly typed config and options
peombwa May 10, 2023
4eff4c4
Move languageFormat to settings file
peombwa May 12, 2023
8bda200
Merge branch 'po/PowerShellSlicing' into po/PowerShellSlicingExtra
peombwa May 12, 2023
86262b6
Added basic version of filter by manifest
darrelmiller May 15, 2023
ca241c0
Updated with latest Apimanifest changes
darrelmiller May 21, 2023
85b3c0f
Started adding the plugin creation command
darrelmiller May 24, 2023
dc16fed
Merge remote-tracking branch 'origin/vnext' into po/PowerShellSlicing…
darrelmiller May 24, 2023
d1c6fca
Merge branch 'po/PowerShellSlicingExtra' into filterbymanifest
darrelmiller May 24, 2023
fd4c768
Merge branch 'filterbymanifest' into chatgpt
darrelmiller May 24, 2023
0f6a6ca
Added preview plugin command
darrelmiller May 30, 2023
bd3a51e
Changed project reference to package reference for manifest
darrelmiller May 31, 2023
7aefc0a
Fixed issues identified by sonarCube
darrelmiller Jun 2, 2023
fe2b6b9
Fixed more sonarcube issues
darrelmiller Jun 2, 2023
76dcb45
Code quality fixes.
darrelmiller Jun 2, 2023
7c71ad3
Returned OpenAPIService to be non-static due to logging issues
darrelmiller Jun 5, 2023
0e13150
Merge remote-tracking branch 'origin/vnext' into feature/plugin
darrelmiller Jun 8, 2023
eb539de
Addressed code smell issues
darrelmiller Jun 8, 2023
5d6db61
Fixed more smells
darrelmiller Jun 9, 2023
7247f74
More smells
darrelmiller Jun 9, 2023
bab8a96
Shortened assignment of output folder
darrelmiller Jun 12, 2023
e3c1f36
Use static lambda.
peombwa Jun 14, 2023
6cdb8f9
Apply suggestions from code review
peombwa Jun 14, 2023
a2ee29d
Separate concerns in OpenApiFilterService class.
peombwa Jun 14, 2023
3e4bf01
Add PowerShellFormatter tests.
peombwa Jun 14, 2023
ede77de
Update public api.
peombwa Jun 15, 2023
15b4c75
Add plugin command test.
peombwa Jun 15, 2023
038a356
Apply is operation suggestion.
peombwa Jun 15, 2023
181c6c3
Perform case-insensitive operationId and keyType match.
peombwa Jun 15, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,36 @@
// Use IntelliSense to find out which attributes exist for C# debugging
// Use hover for the description of the existing attributes
// For further information visit https://github.com/OmniSharp/omnisharp-vscode/blob/master/debugger-launchjson.md
"name": ".NET Core Launch (console)",
"name": "Launch Hidi",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "build",
// If you have changed target frameworks, make sure to update the program path.
"program": "${workspaceFolder}/src/Microsoft.OpenApi.Hidi/bin/Debug/net7.0/Microsoft.OpenApi.Hidi.dll",
"args": [],
"args": ["plugin",
"-m","C:\\Users\\darrmi\\src\\github\\microsoft\\openapi.net\\test\\Microsoft.OpenApi.Hidi.Tests\\UtilityFiles\\exampleapimanifest.json",
"--of","./output"],
"cwd": "${workspaceFolder}/src/Microsoft.OpenApi.Hidi",
// For more information about the 'console' field, see https://aka.ms/VSCode-CS-LaunchJson-Console
"console": "internalConsole",
"stopAtEntry": false
},
{
// Use IntelliSense to find out which attributes exist for C# debugging
// Use hover for the description of the existing attributes
// For further information visit https://github.com/OmniSharp/omnisharp-vscode/blob/master/debugger-launchjson.md
"name": "Launch Workbench",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "build",
// If you have changed target frameworks, make sure to update the program path.
"program": "${workspaceFolder}/src/Microsoft.OpenApi.WorkBench/bin/Debug/net7.0-windows/Microsoft.OpenApi.Workbench.exe",
"args": [],
"cwd": "${workspaceFolder}/src/Microsoft.OpenApi.Workbench",
// For more information about the 'console' field, see https://aka.ms/VSCode-CS-LaunchJson-Console
"console": "internalConsole",
"stopAtEntry": false
},
{
"name": ".NET Core Attach",
"type": "coreclr",
Expand Down
19 changes: 19 additions & 0 deletions src/Microsoft.OpenApi.Hidi/Extensions/CommandExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license.

using System.Collections.Generic;
using System.CommandLine;

namespace Microsoft.OpenApi.Hidi.Extensions
{
internal static class CommandExtensions
{
public static void AddOptions(this Command command, IReadOnlyList<Option> options)
{
foreach (var option in options)
{
command.AddOption(option);
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
using Microsoft.OpenApi.Any;
using Microsoft.OpenApi.Interfaces;
using System.Collections.Generic;

namespace Microsoft.OpenApi.Hidi.Extensions
{
internal static class OpenApiExtensibleExtensions
{
/// <summary>
/// Gets an extension value from the extensions dictionary.
/// </summary>
/// <param name="extensions">A dictionary of <see cref="IOpenApiExtension"/>.</param>
/// <param name="extensionKey">The key corresponding to the <see cref="IOpenApiExtension"/>.</param>
/// <returns>A <see cref="string"/> value matching the provided extensionKey. Return null when extensionKey is not found. </returns>
public static string GetExtension(this IDictionary<string, IOpenApiExtension> extensions, string extensionKey)
{
string extensionValue = null;
if (extensions.TryGetValue(extensionKey, out var value) && value != null)
{
extensionValue = (value as OpenApiString)?.Value;
}
return extensionValue;
peombwa marked this conversation as resolved.
Show resolved Hide resolved
}
}
}
43 changes: 43 additions & 0 deletions src/Microsoft.OpenApi.Hidi/Extensions/StringExtensions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
using System;
using System.Collections.Generic;
using System.Linq;

namespace Microsoft.OpenApi.Hidi.Extensions
{
/// <summary>
/// Extension class for <see cref="string"/>.
/// </summary>
internal static class StringExtensions
{
/// <summary>
/// Checks if the specified searchValue is equal to the target string based on the specified <see cref="StringComparison"/>.
/// </summary>
/// <param name="target">The target string to commpare to.</param>
/// <param name="searchValue">The search string to seek.</param>
/// <param name="comparison">The <see cref="StringComparison"/> to use. This defaults to <see cref="StringComparison.OrdinalIgnoreCase"/>.</param>
/// <returns>true if the searchValue parameter occurs within this string; otherwise, false.</returns>
public static bool IsEquals(this string target, string searchValue, StringComparison comparison = StringComparison.OrdinalIgnoreCase)
{
if (string.IsNullOrWhiteSpace(target) || string.IsNullOrWhiteSpace(searchValue))
{
return false;
}
return target.Equals(searchValue, comparison);
}

/// <summary>
/// Splits the target string in substrings based on the specified char separator.
/// </summary>
/// <param name="target">The target string to split by char. </param>
/// <param name="separator">The char separator.</param>
/// <returns>An <see cref="IList{String}"/> containing substrings.</returns>
public static IList<string> SplitByChar(this string target, char separator)
{
if (string.IsNullOrWhiteSpace(target))
{
return new List<string>();
}
return target.Split(new char[] { separator }, StringSplitOptions.RemoveEmptyEntries).ToList();
}
}
}
262 changes: 262 additions & 0 deletions src/Microsoft.OpenApi.Hidi/Formatters/PowerShellFormatter.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,262 @@
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using Humanizer;
using Humanizer.Inflections;
using Microsoft.OpenApi.Hidi.Extensions;
using Microsoft.OpenApi.Models;
using Microsoft.OpenApi.Services;

namespace Microsoft.OpenApi.Hidi.Formatters
{
internal class PowerShellFormatter : OpenApiVisitorBase
{
private const string DefaultPutPrefix = ".Update";
private const string PowerShellPutPrefix = ".Set";
private readonly Stack<OpenApiSchema> _schemaLoop = new();
private static readonly Regex s_oDataCastRegex = new("(.*(?<=[a-z]))\\.(As(?=[A-Z]).*)", RegexOptions.Compiled, TimeSpan.FromSeconds(5));
private static readonly Regex s_hashSuffixRegex = new(@"^[^-]+", RegexOptions.Compiled, TimeSpan.FromSeconds(5));
private static readonly Regex s_oDataRefRegex = new("(?<=[a-z])Ref(?=[A-Z])", RegexOptions.Compiled, TimeSpan.FromSeconds(5));

static PowerShellFormatter()
{
// Add singularization exclusions.
// TODO: Read exclusions from a user provided file.
Vocabularies.Default.AddSingular("(drive)s$", "$1"); // drives does not properly singularize to drive.
Vocabularies.Default.AddSingular("(data)$", "$1"); // exclude the following from singularization.
Vocabularies.Default.AddSingular("(delta)$", "$1");
Vocabularies.Default.AddSingular("(quota)$", "$1");
Vocabularies.Default.AddSingular("(statistics)$", "$1");
}

//TODO: FHL taks for PS
// Fixes (Order matters):
// 1. Singularize operationId operationIdSegments.
// 2. Add '_' to verb in an operationId.
// 3. Fix odata cast operationIds.
// 4. Fix hash suffix in operationIds.
// 5. Fix Put operation id should have -> {xxx}_Set{Yyy}
// 5. Fix anyOf and oneOf schema.
// 6. Add AdditionalProperties to object schemas.

public override void Visit(OpenApiSchema schema)
{
AddAddtionalPropertiesToSchema(schema);
ResolveAnyOfSchema(schema);
ResolveOneOfSchema(schema);

base.Visit(schema);
}

public override void Visit(OpenApiPathItem pathItem)
{
if (pathItem.Operations.TryGetValue(OperationType.Put, out var value))
{
var operationId = value.OperationId;
pathItem.Operations[OperationType.Put].OperationId = ResolvePutOperationId(operationId);
}

base.Visit(pathItem);
}

public override void Visit(OpenApiOperation operation)
{
if (string.IsNullOrWhiteSpace(operation.OperationId))
throw new ArgumentNullException(nameof(operation.OperationId), $"OperationId is required {PathString}");

var operationId = operation.OperationId;
var operationTypeExtension = operation.Extensions.GetExtension("x-ms-docs-operation-type");
if (operationTypeExtension.IsEquals("function"))
operation.Parameters = ResolveFunctionParameters(operation.Parameters);

// Order matters. Resolve operationId.
operationId = RemoveHashSuffix(operationId);
if (operationTypeExtension.IsEquals("action") || operationTypeExtension.IsEquals("function"))
operationId = RemoveKeyTypeSegment(operationId, operation.Parameters);
operationId = SingularizeAndDeduplicateOperationId(operationId.SplitByChar('.'));
operationId = ResolveODataCastOperationId(operationId);
operationId = ResolveByRefOperationId(operationId);
// Verb segment resolution should always be last. user.get -> user_Get
operationId = ResolveVerbSegmentInOpertationId(operationId);

operation.OperationId = operationId;
base.Visit(operation);
}

private static string ResolveVerbSegmentInOpertationId(string operationId)
{
var charPos = operationId.LastIndexOf('.', operationId.Length - 1);
if (operationId.Contains('_') || charPos < 0)
return operationId;
// TODO: Optimize this call.
var newOperationId = new StringBuilder(operationId);
newOperationId[charPos] = '_';
operationId = newOperationId.ToString();
return operationId;
}

private static string ResolvePutOperationId(string operationId)
{
return operationId.Contains(DefaultPutPrefix) ?
operationId.Replace(DefaultPutPrefix, PowerShellPutPrefix) : operationId;
}

private static string ResolveByRefOperationId(string operationId)
{
// Update $ref path operationId name
// Ref key word is enclosed between lower-cased and upper-cased letters
// Ex.: applications_GetRefCreatedOnBehalfOf to applications_GetCreatedOnBehalfOfByRef
return s_oDataRefRegex.Match(operationId).Success ? $"{s_oDataRefRegex.Replace(operationId, string.Empty)}ByRef" : operationId;
}

private static string ResolveODataCastOperationId(string operationId)
{
var match = s_oDataCastRegex.Match(operationId);
return match.Success ? $"{match.Groups[1]}{match.Groups[2]}" : operationId;
}

private static string SingularizeAndDeduplicateOperationId(IList<string> operationIdSegments)
{
var segmentsCount = operationIdSegments.Count;
var lastSegmentIndex = segmentsCount - 1;
var singularizedSegments = new List<string>();

for (int x = 0; x < segmentsCount; x++)
baywet marked this conversation as resolved.
Show resolved Hide resolved
{
var segment = operationIdSegments[x].Singularize(inputIsKnownToBePlural: false);

// If a segment name is contained in the previous segment, the latter is considered a duplicate.
// The last segment is ignored as a rule.
if ((x > 0 && x < lastSegmentIndex) && singularizedSegments.Last().Equals(segment, StringComparison.OrdinalIgnoreCase))
continue;

singularizedSegments.Add(segment);
}
return string.Join(".", singularizedSegments);
}

private static string RemoveHashSuffix(string operationId)
{
// Remove hash suffix values from OperationIds.
return s_hashSuffixRegex.Match(operationId).Value;
}

private static string RemoveKeyTypeSegment(string operationId, IList<OpenApiParameter> parameters)
{
var segments = operationId.SplitByChar('.');
foreach (var parameter in parameters)
{
var keyTypeExtension = parameter.Extensions.GetExtension("x-ms-docs-key-type");
if (keyTypeExtension != null && operationId.Contains(keyTypeExtension))
peombwa marked this conversation as resolved.
Show resolved Hide resolved
{
segments.Remove(keyTypeExtension);
}
}
return string.Join(".", segments);
}

private static IList<OpenApiParameter> ResolveFunctionParameters(IList<OpenApiParameter> parameters)
{
foreach (var parameter in parameters.Where(p => p.Content?.Any() ?? false))
peombwa marked this conversation as resolved.
Show resolved Hide resolved
{
// Replace content with a schema object of type array
// for structured or collection-valued function parameters
parameter.Content = null;
parameter.Schema = new OpenApiSchema
{
Type = "array",
Items = new OpenApiSchema
{
Type = "string"
}
};
}
return parameters;
}

private void AddAddtionalPropertiesToSchema(OpenApiSchema schema)
peombwa marked this conversation as resolved.
Show resolved Hide resolved
{
if (schema != null && !_schemaLoop.Contains(schema) && "object".Equals(schema?.Type, StringComparison.OrdinalIgnoreCase))
peombwa marked this conversation as resolved.
Show resolved Hide resolved
{
schema.AdditionalProperties = new OpenApiSchema() { Type = "object" };

/* Because 'additionalProperties' are now being walked,
* we need a way to keep track of visited schemas to avoid
* endlessly creating and walking them in an infinite recursion.
*/
_schemaLoop.Push(schema.AdditionalProperties);
}
}

private static void ResolveOneOfSchema(OpenApiSchema schema)
{
if (schema.OneOf?.Any() ?? false)
{
var newSchema = schema.OneOf.FirstOrDefault();
schema.OneOf = null;
FlattenSchema(schema, newSchema);
}
}

private static void ResolveAnyOfSchema(OpenApiSchema schema)
{
if (schema.AnyOf?.Any() ?? false)
{
var newSchema = schema.AnyOf.FirstOrDefault();
schema.AnyOf = null;
FlattenSchema(schema, newSchema);
}
}

private static void FlattenSchema(OpenApiSchema schema, OpenApiSchema newSchema)
{
if (newSchema != null)
{
if (newSchema.Reference != null)
{
schema.Reference = newSchema.Reference;
schema.UnresolvedReference = true;
}
else
{
// Copies schema properties based on https://github.com/microsoft/OpenAPI.NET.OData/pull/264.
CopySchema(schema, newSchema);
}
}
}

private static void CopySchema(OpenApiSchema schema, OpenApiSchema newSchema)
{
schema.Title ??= newSchema.Title;
schema.Type ??= newSchema.Type;
schema.Format ??= newSchema.Format;
schema.Description ??= newSchema.Description;
schema.Maximum ??= newSchema.Maximum;
schema.ExclusiveMaximum ??= newSchema.ExclusiveMaximum;
schema.Minimum ??= newSchema.Minimum;
schema.ExclusiveMinimum ??= newSchema.ExclusiveMinimum;
schema.MaxLength ??= newSchema.MaxLength;
schema.MinLength ??= newSchema.MinLength;
schema.Pattern ??= newSchema.Pattern;
schema.MultipleOf ??= newSchema.MultipleOf;
schema.Not ??= newSchema.Not;
schema.Required ??= newSchema.Required;
schema.Items ??= newSchema.Items;
schema.MaxItems ??= newSchema.MaxItems;
schema.MinItems ??= newSchema.MinItems;
schema.UniqueItems ??= newSchema.UniqueItems;
schema.Properties ??= newSchema.Properties;
schema.MaxProperties ??= newSchema.MaxProperties;
schema.MinProperties ??= newSchema.MinProperties;
schema.Discriminator ??= newSchema.Discriminator;
schema.ExternalDocs ??= newSchema.ExternalDocs;
schema.Enum ??= newSchema.Enum;
schema.ReadOnly = !schema.ReadOnly ? newSchema.ReadOnly : schema.ReadOnly;
schema.WriteOnly = !schema.WriteOnly ? newSchema.WriteOnly : schema.WriteOnly;
schema.Nullable = !schema.Nullable ? newSchema.Nullable : schema.Nullable;
schema.Deprecated = !schema.Deprecated ? newSchema.Deprecated : schema.Deprecated;
}
}
}
Loading