Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[core-lro] proof-of-concept for https://github.com/Azure/azure-sdk-for-js/issues/32142 #6

Draft
wants to merge 2 commits into
base: keyvault-admin-typespec
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions sdk/core/core-lro/review/core-lro.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ export interface CreateHttpPollerOptions<TResult, TState> {
resolveOnUnsuccessful?: boolean;
resourceLocationConfig?: ResourceLocationConfig;
restoreFrom?: string;
skipFinalGet?: boolean;
Copy link
Owner Author

@maorleger maorleger Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Name TBD: any suggestions? skipFinalGet seems pretty specific. Other options might be

  • withoutResourceLocation
  • skipResourceLocation

or something else - feel free to chime in. Thoughts?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This name could be unclear for someone who is not too familiar with how our pollers work. Could we name it based on impact instead? e.g. noResourceRetrieval. I also wonder if other languages have a similar flag and what name did they choose for it.

Copy link
Owner Author

@maorleger maorleger Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

skipResourceLocation or skipGetResource maybe? Would that make sense? I don't have a good domain understanding of the LRO concepts so I am just brainstorming names 👍 feels like resourceLocation is a known term in LROs?

Copy link

@MaryGao MaryGao Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH I even prefer the specific name skipFinalGet here, because for me LRO would be three steps: initial -> polling -> final get(optional) and the main purpose of this option is to always skip the final step. I guess this would be clear with someone who knows LRO.

My concern for ResourceRetrieval is - skipping the final get doesn't mean we will not retrieve the resources, for example maybe the resource is directly returned from the initial request or non-standard LRO returns the resources directly from polling process?

But open to hear feedbacks also.

updateState?: (state: TState, response: OperationResponse) => void;
withOperationLocation?: (operationLocation: string) => void;
}
Expand Down
4 changes: 4 additions & 0 deletions sdk/core/core-lro/src/http/models.ts
Original file line number Diff line number Diff line change
Expand Up @@ -120,4 +120,8 @@ export interface CreateHttpPollerOptions<TResult, TState> {
* Control whether to throw an exception if the operation failed or was canceled.
*/
resolveOnUnsuccessful?: boolean;
/**
* Skip final get request.
*/
skipFinalGet?: boolean;
}
6 changes: 6 additions & 0 deletions sdk/core/core-lro/src/http/poller.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import {
} from "./operation.js";
import type { CreateHttpPollerOptions } from "./models.js";
import { buildCreatePoller } from "../poller/poller.js";
import { logger } from "../logger.js";

/**
* Creates a poller that can be used to poll a long-running operation.
Expand All @@ -34,6 +35,7 @@ export function createHttpPoller<TResult, TState extends OperationState<TResult>
updateState,
withOperationLocation,
resolveOnUnsuccessful = false,
skipFinalGet = false,
} = options || {};
return buildCreatePoller<OperationResponse, TResult, TState>({
getStatusFromInitialResponse,
Expand All @@ -49,6 +51,10 @@ export function createHttpPoller<TResult, TState extends OperationState<TResult>
init: async () => {
const response = await lro.sendInitialRequest();
const config = inferLroMode(response.rawResponse, resourceLocationConfig);
if (skipFinalGet && config?.resourceLocation) {
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will end up moving into inferLroMode so we are not setting config then overriding it. The proof-of-concept just touches as little code as possible

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change works for me. And I was also thinking of that this could also work in the resumeFrom case.

nit: should we put this logic into inferLroMode?

logger.info("skipFinalGet is enabled, skipping final GET operation of LRO");
config.resourceLocation = undefined;
}
return {
response,
operationLocation: config?.operationLocation,
Expand Down
86 changes: 86 additions & 0 deletions sdk/keyvault/generate.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
#!/usr/bin/env node

const { execSync } = require("child_process");
const fs = require("fs");
const path = require("path");

// Helper to execute shell commands and log output
function execCommand(command) {
try {
execSync(command, { stdio: "inherit" });
} catch (error) {
console.error(`Command failed: ${command}`);
process.exit(1);
}
}

console.log("Setting up the environment...");

// Workaround for src-folder support in emitter:
// End state: src/generated/* contains generated code (instead of src/generated/src/*)

// Step 1: Remove all files in src/generated/*
execCommand("rm -rf src/generated/*");

// Step 2: Copy tsp-location.yaml to src/generated
execCommand("cp tsp-location.yaml src/generated");

// Step 3: Run tsp-client command
// emitter-option as a workaround for https://github.com/Azure/azure-rest-api-specs/issues/31610
execCommand(`tsp-client update -d -o src/generated --emitter-options generateMetadata=false`);
// execCommand(
// "tsp-client update -d -o src/generated --tsp-config ~/workspace/azure-rest-api-specs/specification/keyvault/Security.KeyVault.Keys/tspconfig.yaml --local-spec-repo ~/workspace/azure-rest-api-specs/specification/keyvault/Security.KeyVault.Keys --repo ~/workspace/azure-rest-api-specs --commit 9561bad7d2eed94cc91aa6164d3721b8aa8699fe --emitter-options generateMetadata=false"
// );

// Step 4: Move generated/src/* files to generated until src-folder is supported
execCommand("mv src/generated/src/* src/generated/");

// Step 5: Remove generated/src
execCommand("rm -rf src/generated/src");

// Step 6: Remove tsp-location.yaml from generated folder
execCommand("rm src/generated/tsp-location.yaml");

// Step 7: Read and update package.json
console.log("Updating package.json dependencies...");
const packageJsonPath = path.resolve("./package.json");
const packageJson = JSON.parse(fs.readFileSync(packageJsonPath, "utf8"));

// Remove dependency on @azure/core-client and add @azure-rest/core-client
delete packageJson.dependencies["@azure/core-client"];
packageJson.dependencies["@azure-rest/core-client"] = "^2.0.0";

// Write updated package.json back to disk
fs.writeFileSync(packageJsonPath, JSON.stringify(packageJson, null, 2), "utf8");

// Generated code changes
// Workaround for https://github.com/Azure/autorest.typescript/pull/2135/files
const modelsPath = path.resolve("./src/generated/models/models.ts");
let modelsContent = fs.readFileSync(modelsPath, "utf8");
modelsContent = modelsContent
.replace(
/created: !item\["created"\] \? item\["created"\] : new Date\(item\["created"\]\),/g,
'created: !item["created"] ? item["created"] : new Date(item["created"] * 1000),'
)
.replace(
/updated: !item\["updated"\] \? item\["updated"\] : new Date\(item\["updated"\]\),/g,
'updated: !item["updated"] ? item["updated"] : new Date(item["updated"] * 1000),'
)
.replace(
/notBefore: !item\["nbf"\] \? item\["nbf"\] : new Date\(item\["nbf"\]\),/g,
'notBefore: !item["nbf"] ? item["nbf"] : new Date(item["nbf"] * 1000),'
)
.replace(
/expires: !item\["exp"\] \? item\["exp"\] : new Date\(item\["exp"\]\),/g,
'expires: !item["exp"] ? item["exp"] : new Date(item["exp"] * 1000),'
)
.replace(
/nbf: !item\["notBefore"\] \? item\["notBefore"\] : item\["notBefore"\].getTime\(\),/g,
'nbf: !item["notBefore"] ? item["notBefore"] : item["notBefore"].getTime() / 1000,'
)
.replace(
/exp: !item\["expires"\] \? item\["expires"\] : item\["expires"\].getTime\(\),/g,
'exp: !item["expires"] ? item["expires"] : item["expires"].getTime() / 1000,'
);

fs.writeFileSync(modelsPath, modelsContent, "utf8");
2 changes: 1 addition & 1 deletion sdk/keyvault/keyvault-admin/assets.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@
"AssetsRepo": "Azure/azure-sdk-assets",
"AssetsRepoPrefixPath": "js",
"TagPrefix": "js/keyvault/keyvault-admin",
"Tag": "js/keyvault/keyvault-admin_1ce65643a5"
"Tag": "js/keyvault/keyvault-admin_46c9877128"
}
4 changes: 2 additions & 2 deletions sdk/keyvault/keyvault-admin/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -90,10 +90,10 @@
},
"sideEffects": false,
"dependencies": {
"@azure-rest/core-client": "^2.0.0",
"@azure/abort-controller": "^2.0.0",
"@azure/core-auth": "^1.3.0",
"@azure/core-client": "^1.0.0",
"@azure/core-lro": "^2.2.0",
"@azure/core-lro": "^3.1.0",
"@azure/core-paging": "^1.1.1",
"@azure/core-rest-pipeline": "^1.1.0",
"@azure/core-tracing": "^1.0.0",
Expand Down
30 changes: 18 additions & 12 deletions sdk/keyvault/keyvault-admin/review/keyvault-admin.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,15 +4,15 @@

```ts

import type { CommonClientOptions } from '@azure/core-client';
import type { OperationOptions } from '@azure/core-client';
import type { AbortSignalLike } from '@azure/abort-controller';
import type { CancelOnProgress } from '@azure/core-lro';
import type { ClientOptions } from '@azure-rest/core-client';
import type { OperationOptions } from '@azure-rest/core-client';
import type { PagedAsyncIterableIterator } from '@azure/core-paging';
import type { PollerLike } from '@azure/core-lro';
import type { PollOperationState } from '@azure/core-lro';
import type { TokenCredential } from '@azure/core-auth';

// @public
export interface AccessControlClientOptions extends CommonClientOptions {
export interface AccessControlClientOptions extends ClientOptions {
disableChallengeResourceVerification?: boolean;
serviceVersion?: SUPPORTED_API_VERSIONS;
}
Expand Down Expand Up @@ -65,17 +65,23 @@ export class KeyVaultAccessControlClient {
}

// @public
export interface KeyVaultAdminPollOperationState<TResult> extends PollOperationState<TResult> {
export interface KeyVaultAdminPollOperationState<TResult> {
endTime?: Date;
error?: Error;
isCompleted?: boolean;
isStarted?: boolean;
jobId?: string;
result?: TResult;
startTime?: Date;
status?: string;
// Warning: (ae-forgotten-export) The symbol "OperationStatus" needs to be exported by the entry point index.d.ts
status: OperationStatus;
statusDetails?: string;
}

// @public
export class KeyVaultBackupClient {
constructor(vaultUrl: string, credential: TokenCredential, options?: KeyVaultBackupClientOptions);
// Warning: (ae-forgotten-export) The symbol "PollerLike" needs to be exported by the entry point index.d.ts
beginBackup(blobStorageUri: string, sasToken: string, options?: KeyVaultBeginBackupOptions): Promise<PollerLike<KeyVaultBackupOperationState, KeyVaultBackupResult>>;
beginBackup(blobStorageUri: string, options?: KeyVaultBeginBackupOptions): Promise<PollerLike<KeyVaultBackupOperationState, KeyVaultBackupResult>>;
beginRestore(folderUri: string, sasToken: string, options?: KeyVaultBeginRestoreOptions): Promise<PollerLike<KeyVaultRestoreOperationState, KeyVaultRestoreResult>>;
Expand All @@ -86,7 +92,7 @@ export class KeyVaultBackupClient {
}

// @public
export interface KeyVaultBackupClientOptions extends CommonClientOptions {
export interface KeyVaultBackupClientOptions extends ClientOptions {
disableChallengeResourceVerification?: boolean;
serviceVersion?: SUPPORTED_API_VERSIONS;
}
Expand All @@ -104,7 +110,7 @@ export interface KeyVaultBackupPollerOptions extends OperationOptions {
export interface KeyVaultBackupResult {
endTime?: Date;
folderUri?: string;
startTime: Date;
startTime?: Date;
}

// @public
Expand Down Expand Up @@ -137,7 +143,7 @@ export interface KeyVaultRestoreOperationState extends KeyVaultAdminPollOperatio
// @public
export interface KeyVaultRestoreResult {
endTime?: Date;
startTime: Date;
startTime?: Date;
}

// @public
Expand Down Expand Up @@ -177,7 +183,7 @@ export interface KeyVaultSelectiveKeyRestoreOperationState extends KeyVaultAdmin
// @public
export interface KeyVaultSelectiveKeyRestoreResult {
endTime?: Date;
startTime: Date;
startTime?: Date;
}

// @public
Expand Down Expand Up @@ -283,7 +289,7 @@ export interface SetRoleDefinitionOptions extends OperationOptions {
}

// @public
export interface SettingsClientOptions extends CommonClientOptions {
export interface SettingsClientOptions extends ClientOptions {
disableChallengeResourceVerification?: boolean;
serviceVersion?: SUPPORTED_API_VERSIONS;
}
Expand Down
Loading
Loading