feat: method-by-method OAS doc generation strategy #6023

mingxuanzhangsfdx · 2025-01-23T05:58:08Z

What does this PR do?

Implement the strategy of generating OAS within a class method by method, and combine the separate YAML into one.

What issues does this PR fix or reference?

@W-17587223@

Functionality Before

only generate the OAS doc on class level.

Functionality After

method-by-method strategy is available, which could potentially avoid the prompt size limit.

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/prompts.ts

peternhale · 2025-01-23T17:19:23Z

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/methodByMethodStrategy.ts

+      const responsePromises = this.prompts.filter(p => p?.length > 0).map(prompt => llmService.callLLM(prompt));
+
+      // Execute all LLM calls in parallel and store responses
+      this.llmResponses = await Promise.all(responsePromises);


@mingxuanzhangsfdx If one promise fails then they all do. Is that what you want. If you would like an opportunity to examine each of these, then have a look at Promise.allSettled

peternhale · 2025-01-23T17:21:16Z

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/methodByMethodStrategy.ts

+    }
+  }
+
+  cleanYamlString(input: string): string {


consider adding the @apos; handling here, changing instances to '

this is resolved by Cristi's tip above

peternhale · 2025-01-23T17:30:07Z

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/methodByMethodStrategy.ts

+        Object.assign(combined.paths[path], methods);
+      }
+      // Merge components
+      if (parsed.components?.schemas) {


@mingxuanzhangsfdx should this check if the schema already exists, as is being done above with paths

peternhale · 2025-01-23T17:34:52Z

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/methodByMethodStrategy.ts

+    input += `${prompts.SYSTEM_TAG}\n${prompts['METHOD_BY_METHOD.systemPrompt']}\n${prompts.END_OF_PROMPT_TAG}\n`;
+    input += `${prompts.USER_TAG}\n${prompts.METHOD_BY_METHOD_USER_PROMPT}\n`;
+    input += '\nThis is the Apex method the OpenAPI v3 specification should be generated for:\n```\n';
+    const documentText = fs.readFileSync(new URL(this.metadata.resourceUri.toString()), 'utf8');


Reading the Apex class for each method is expensive. this should be done in constructor.

peternhale · 2025-01-23T17:36:35Z

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/methodByMethodStrategy.ts

+    if (methodContext?.returnType !== undefined) {
+      input += `The return type of the method is ${methodContext.returnType}.\n`;
+    }
+    if (methodContext?.parameterTypes && methodContext.parameterTypes.length > 0) {


Suggested change

if (methodContext?.parameterTypes && methodContext.parameterTypes.length > 0) {

if (methodContext?.parameterTypes?.length > 0) {

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/tmp.ts

mingxuanzhangsfdx · 2025-01-23T21:19:38Z

packages/salesforcedx-vscode-apex/src/commands/apexActionController.ts

@@ -73,7 +73,7 @@ export class ApexActionController {
          // Step 5:Initialize the strategy orchestrator
          const promptGenerationOrchestrator = new PromptGenerationOrchestrator(eligibilityResult, context);
          // Step 6: bid on the strategy, and the best one is available
-          promptGenerationOrchestrator.bid();
+          // promptGenerationOrchestrator.bid();
          // Step 7: use the strateg to generate the OAS
          const openApiDocument = await promptGenerationOrchestrator.generateOASWithStrategySelectedByBidRule(BidRule.MOST_CALLS);


The bid rule is set to always enable method-by-method strategy

packages/salesforcedx-vscode-apex/src/commands/apexActionController.ts

mingxuanzhangsfdx · 2025-01-23T21:22:23Z

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/methodByMethodStrategy.ts

+      const responsePromises = this.prompts.filter(p => p?.length > 0).map(prompt => llmService.callLLM(prompt));
+
+      // Execute all LLM calls in parallel and store responses
+      await Promise.allSettled(responsePromises).then(results => {


use Promise.allSettled to populate the methods as many as possible and not be blocked by failed requests

mingxuanzhangsfdx · 2025-01-23T21:23:29Z

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/methodByMethodStrategy.ts

+      info: {
+        title: this.context.classDetail.name,
+        version: '1.0.0',
+        description: `This is auto-generated OpenAPI v3 spec for ${this.context.classDetail.name}.`


the description is templated before we elaborate how it should look like

mingxuanzhangsfdx · 2025-01-23T21:25:39Z

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/methodByMethodStrategy.ts

+        Object.assign(combined.paths[path], methods);
+        // explicitly define openrationId if missing
+        for (const [method, props] of Object.entries(combined.paths[path] as object)) {
+          if (props?.operationId === undefined) {


I tried several different prompts to remind LLM always enabling operationId, but found out it still failed to fulfill my will constantly. So I added this mandatory check.

an example how it could fail

@RestResource(urlMapping='/simpleResource/*') global with sharing class SimpleRestResource { @HttpGet global static String doGet() { return 'Hello World'; } @HttpPost global static String doPost(String message) { return 'You posted: ' + message; } @HttpPut global static String doPut(String message) { return 'You put: ' + message; } @HttpDelete global static String doDelete() { return 'Goodbye'; } }

mingxuanzhangsfdx · 2025-01-23T21:28:16Z

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/prompts.ts

+      '{ClassName}/{MethodName}. For every `type: object`, generate a `#/components/schemas` entry for that object. The method should have a $ref entry pointing to the generated `#/components/schemas` entry. I do not want AUTHOR_PLACEHOLDER in the result.' +
+      'For each path, you define operations (HTTP methods) that can be used to access that path.' +
+      'IMPORTANT: Each operation includes a MANDATORY *operationId* property, which should be a unique string matching the operations name.' +
+      'Do NOT add any explanations of your answer that are not able to be parsed as YAML!',


have to emphasize the rule No.5 above again here in order to avoid unnecessary explanation in the response to block YAML parsing

mingxuanzhangsfdx · 2025-01-23T21:29:10Z

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/prompts.ts

+      'Generate an OpenAPI v3 specification for the following Apex method. The OpenAPI v3 specification should be a YAML file. The path should be /' +
+      '{ClassName}/{MethodName}. For every `type: object`, generate a `#/components/schemas` entry for that object. The method should have a $ref entry pointing to the generated `#/components/schemas` entry. I do not want AUTHOR_PLACEHOLDER in the result.' +
+      'For each path, you define operations (HTTP methods) that can be used to access that path.' +
+      'IMPORTANT: Each operation includes a MANDATORY *operationId* property, which should be a unique string matching the operations name.' +


Update the rule a bit, but still could see some responses failing. So that's why I added a mandatory check above.

peternhale · 2025-01-24T21:44:37Z

packages/salesforcedx-vscode-apex/src/commands/apexActionController.ts

-
-          // Step 7: use the strategy to generate the OAS
+          // promptGenerationOrchestrator.bid();
+          // Step 7: use the strateg to generate the OAS
          const openApiDocument = await promptGenerationOrchestrator.generateOASWithStrategySelectedByBidRule(


@mingxuanzhangsfdx what happens if there is a tie on number of calls?

* Mz/apply generation strategy (#6018) * feat: integrate apex action controller with prompt generation orchestrator * chore: update framework * chore: add error entries * feat: whole class gen strategy (#6024) * feat: integrate apex action controller with prompt generation orchestrator * feat: whole class gen strategy * chore: sort imports * chore: asdfghj --------- Co-authored-by: mingxuanzhang <mingxuanzhang@salesforce.com> * feat: update jar with changes in lsp to add comments in context (#6028) * feat: update jar with changes in lsp to add comments in context (#6025) * feat: update jar with changes in lsp to add comments in context * feat: update jar with changes in lsp to add comments in context * fix: add comment property to context (#6026) @W-17645605@ moved comments from ApexClassOASGatherContextResponse and added comment proeprty to class/method/property --------- Co-authored-by: peternhale <peternhale@users.noreply.github.com> * feat: method-by-method OAS doc generation strategy (#6023) * feat: create open api doc method by method * chore: update for generating ESR * chore: refine * chore: refine prompt structure * feat: enable comments for prompt generation * chore: sync jorje jar * chore: sync generation strategy with latest change (#6031) * feat: update jar with changes in lsp to add comments in context (#6025) * feat: update jar with changes in lsp to add comments in context * feat: update jar with changes in lsp to add comments in context * fix: add comment property to context (#6026) @W-17645605@ moved comments from ApexClassOASGatherContextResponse and added comment proeprty to class/method/property * chore: remove deploy and retrieve commands for oas and hide create oas from method command (#6029) * Madhur/w 17619393 (#6017) * feat: adding spectral library and validation based on OAS3 spec * feat: adding spectral library and validation based on OAS3 spec * feat: adding spectral library and validation based on OAS3 spec * feat: adding spectral library and validation based on OAS3 spec * fix: increase minimum supported node version to v20.17.0 * feat: adding spectral library and validation based on OAS3 spec --------- Co-authored-by: Daphne Yang <daphne.yang@salesforce.com> --------- Co-authored-by: Cristina Cañizales <113132642+CristiCanizales@users.noreply.github.com> Co-authored-by: peternhale <peternhale@users.noreply.github.com> Co-authored-by: madhur310 <madhur.shrivastava@salesforce.com> Co-authored-by: Daphne Yang <daphne.yang@salesforce.com> * chore: remove eslint disable * chore: move schemas to oas folder * chore: sort imports --------- Co-authored-by: Mingxuan Zhang <132491513+mingxuanzhangsfdx@users.noreply.github.com> Co-authored-by: mingxuanzhang <mingxuanzhang@salesforce.com> Co-authored-by: peternhale <peternhale@users.noreply.github.com> Co-authored-by: madhur310 <madhur.shrivastava@salesforce.com> Co-authored-by: Daphne Yang <daphne.yang@salesforce.com>

feat: create open api doc method by method

5db2945

mingxuanzhangsfdx changed the base branch from feat/apply-generation-strategy to feat/generation-strategy January 23, 2025 05:59

chore: update for generating ESR

80bf99c

CristiCanizales requested changes Jan 23, 2025

View reviewed changes

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/prompts.ts Outdated Show resolved Hide resolved

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/prompts.ts Outdated Show resolved Hide resolved

CristiCanizales reviewed Jan 23, 2025

View reviewed changes

packages/salesforcedx-vscode-apex/src/oas/generationStrategy/prompts.ts Outdated Show resolved Hide resolved

peternhale requested changes Jan 23, 2025

View reviewed changes

mingxuanzhangsfdx added 2 commits January 23, 2025 13:13

chore: refine

39f6f96

chore: refine prompt structure

485bf90

mingxuanzhangsfdx commented Jan 23, 2025

View reviewed changes

packages/salesforcedx-vscode-apex/src/commands/apexActionController.ts Outdated Show resolved Hide resolved

mingxuanzhangsfdx commented Jan 23, 2025

View reviewed changes

mingxuanzhangsfdx requested review from CristiCanizales and peternhale January 23, 2025 21:31

mingxuanzhangsfdx changed the title ~~Mz/methodbymethod~~ feat: method-by-method OAS doc generation strategy Jan 23, 2025

mingxuanzhangsfdx added 3 commits January 24, 2025 10:21

Merge branch 'feat/generation-strategy' into mz/methodbymethod

38e43b3

feat: enable comments for prompt generation

e93b83b

chore: sync jorje jar

af4ef2f

mingxuanzhangsfdx requested a review from daphne-sfdc January 24, 2025 19:58

peternhale reviewed Jan 24, 2025

View reviewed changes

peternhale approved these changes Jan 24, 2025

View reviewed changes

peternhale marked this pull request as ready for review January 24, 2025 22:08

peternhale requested a review from a team as a code owner January 24, 2025 22:08

mingxuanzhangsfdx merged commit 8147b1a into feat/generation-strategy Jan 24, 2025
6 checks passed

mingxuanzhangsfdx deleted the mz/methodbymethod branch January 24, 2025 23:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: method-by-method OAS doc generation strategy #6023

feat: method-by-method OAS doc generation strategy #6023

mingxuanzhangsfdx commented Jan 23, 2025 •

edited

Loading

peternhale Jan 23, 2025

peternhale Jan 23, 2025

mingxuanzhangsfdx Jan 23, 2025

peternhale Jan 23, 2025

peternhale Jan 23, 2025

peternhale Jan 23, 2025

mingxuanzhangsfdx Jan 23, 2025

mingxuanzhangsfdx Jan 23, 2025

mingxuanzhangsfdx Jan 23, 2025

mingxuanzhangsfdx Jan 23, 2025

mingxuanzhangsfdx Jan 23, 2025

mingxuanzhangsfdx Jan 23, 2025

mingxuanzhangsfdx Jan 23, 2025

peternhale Jan 24, 2025

	if (methodContext?.parameterTypes && methodContext.parameterTypes.length > 0) {
	if (methodContext?.parameterTypes?.length > 0) {

feat: method-by-method OAS doc generation strategy #6023

feat: method-by-method OAS doc generation strategy #6023

Conversation

mingxuanzhangsfdx commented Jan 23, 2025 • edited Loading

What does this PR do?

What issues does this PR fix or reference?

Functionality Before

Functionality After

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mingxuanzhangsfdx commented Jan 23, 2025 •

edited

Loading