Merge latest upstream python extension

--------------------
Commit message for microsoft/vscode-python@fc72be9:

Show `Python: Report issue` command in palette regardless of whether a Python file is opened (microsoft/vscode-python#20726)

Closes microsoft/vscode-python#20723
--------------------
Commit message for microsoft/vscode-python@c18e8c9:

Detect ActiveState Python runtimes (microsoft/vscode-python#20534)

Closes microsoft/vscode-python#20532
--------------------
Commit message for microsoft/vscode-python@2152cd9:

Don't set `formatOnType` for auto-indent experiment if it's already set (microsoft/vscode-python#20710)


--------------------
Commit message for microsoft/vscode-python@995b0bc:

Add support for 'back' to all create env UI. (microsoft/vscode-python#20693)

Closes microsoft/vscode-python#20274


### Usage

This change allows callers of the Create Environment command to handle
`Back` and `Cancel`:
``` typescript
let result: CreateEnvironmentResult | undefined;
try {
    const result = await commands.executeCommand("python.createEnvironment", {showBackButton: true});
} catch(e) {
   // error while creating environment
}

if (result?.action === 'Back') {
    // user clicked Back
}

if (result?.action === 'Cancel') {
    // user pressed escape or Cancel
}
```
I decided to go with `result?.action` because we don't have a npm
package for python extension API so catching particular exception might
be error prone with `ex instanceof <error>`. We will provide a proper
interface via `api.environments` for create environment, and
contribution to create environment. Until that point this command will
provide the stop gap.

### Notes

1. I did not use the multi-step input that is used in the rest of the
extension because, the existing implementation does not have context.
Consider the following scenario: venv -> workspace select -> python
select -> packages. Assume that there is only one workspace, and we
don't show the workspace selection UI, that decision is done inside the
workspace step. So, if there is only 1 workspace it is a short circuit
to next step. User is on python selection and clicks `back`, workspace
selection short circuits to next step which is python selection. So,
from user perspective, back does not work. This can be fixed by sending
context that the reason control moved to previous step was because user
clicked on back.
2. This makes a change to old multi step API to rethrow the exception,
if user hits `back` and the current step has no steps to go back to.
--------------------
Commit message for microsoft/vscode-python@f3ecbf5:

Fix-conda-version-parsing (microsoft/vscode-python#20674)


--------------------
Commit message for microsoft/vscode-python@a6a6f50:

Inactive pytest run command (microsoft/vscode-python#20653)

Here the new flow is created but kept inactive for the pytest execution
--------------------
Commit message for microsoft/vscode-python@2202fbe:

Call the correct API to determine if a user is in treatment or control group (microsoft/vscode-python#20690)

Closes microsoft/vscode-python#20183
--------------------
Commit message for microsoft/vscode-python@b0ab10d:

Only use activated environment from terminal if VSCode was launched via CLI (microsoft/vscode-python#20667)

Closes microsoft/vscode-python#20644
--------------------
Commit message for microsoft/vscode-python@02a92fc:

Ensure interpreter path isn't truncated for workspace-relative paths when storing value (microsoft/vscode-python#20661)

For microsoft/vscode-python#20660

I'm not quite sure why this was done. It doesn't make sense to do this
only for display.
--------------------
Commit message for microsoft/vscode-python@377067f:

Use correct API to get interpreter path for language servers (microsoft/vscode-python#20656)

For microsoft/vscode-python#20644 closes
microsoft/vscode-python#20657
--------------------
Commit message for microsoft/vscode-python@cd6ca9d:

Remove `isort` extension dependency (microsoft/vscode-python#20577)

Closes microsoft/vscode-python#20586

Lead-authored-by: Kartik Raj <karraj@microsoft.com>
Co-authored-by: Erik De Bonte <erikd@microsoft.com>
Co-authored-by: Karthik Nadig <kanadig@microsoft.com>
Co-authored-by: mitchell <mitchellb@activestate.com>
Co-authored-by: Pete Farland <pete.farland@posit.co>
Co-authored-by: Eleanor Boyd <eleanorboyd@microsoft.com>

extensions/positron-python/.github/workflows/pr-check.yml

@@ -266,15 +266,6 @@ jobs:
           working-directory: ${{ env.special-working-directory }}
         if: matrix.test-suite == 'single-workspace'
 
-      - name: Run multi-workspace tests
-        env:
-          CI_PYTHON_VERSION: ${{ matrix.python }}
-        uses: GabrielBB/xvfb-action@v1.6
-        with:
-          run: npm run testMultiWorkspace
-          working-directory: ${{ env.special-working-directory }}
-        if: matrix.test-suite == 'multi-workspace'
-
       - name: Run debugger tests
         env:
           CI_PYTHON_VERSION: ${{ matrix.python }}

extensions/positron-python/gulpfile.js

@@ -82,7 +82,7 @@ async function addExtensionPackDependencies() {
     // extension dependencies need not be installed during development
     const packageJsonContents = await fsExtra.readFile('package.json', 'utf-8');
     const packageJson = JSON.parse(packageJsonContents);
-    packageJson.extensionPack = ['ms-toolsai.jupyter', 'ms-python.vscode-pylance', 'ms-python.isort'].concat(
+    packageJson.extensionPack = ['ms-toolsai.jupyter', 'ms-python.vscode-pylance'].concat(
         packageJson.extensionPack ? packageJson.extensionPack : [],
     );
     await fsExtra.writeFile('package.json', JSON.stringify(packageJson, null, 4), 'utf-8');

extensions/positron-python/package.json

@@ -383,6 +383,12 @@
                     "scope": "window",
                     "type": "string"
                 },
+                "python.activeStateToolPath": {
+                    "default": "state",
+                    "description": "%python.activeStateToolPath.description%",
+                    "scope": "machine-overridable",
+                    "type": "string"
+                },
                 "python.autoComplete.extraPaths": {
                     "default": [],
                     "description": "%python.autoComplete.extraPaths.description%",
@@ -1621,7 +1627,7 @@
                     "category": "Python",
                     "command": "python.reportIssue",
                     "title": "%python.command.python.reportIssue.title%",
-                    "when": "!virtualWorkspace && shellExecutionSupported && editorLangId == python"
+                    "when": "!virtualWorkspace && shellExecutionSupported"
                 },
                 {
                     "category": "Test",

extensions/positron-python/package.nls.json

@@ -27,6 +27,7 @@
     "python.command.python.launchTensorBoard.title": "Launch TensorBoard",
     "python.command.python.refreshTensorBoard.title": "Refresh TensorBoard",
     "python.menu.createNewFile.title": "Python File",
+    "python.activeStateToolPath.description": "Path to the State Tool executable for ActiveState runtimes (version 0.36+).",
     "python.autoComplete.extraPaths.description": "List of paths to libraries and the like that need to be imported by auto complete engine. E.g. when using Google App SDK, the paths are not in system path, hence need to be added into this list.",
     "python.condaPath.description": "Path to the conda executable to use for activation (version 4.4+).",
     "python.defaultInterpreterPath.description": "Path to default Python to use when extension loads up for the first time, no longer used once an interpreter is selected for the workspace. See [here](https://aka.ms/AAfekmf) to understand when this is used",

extensions/positron-python/pythonFiles/create_venv.py

@@ -154,14 +154,14 @@ def main(argv: Optional[Sequence[str]] = None) -> None:
     if pip_installed:
         upgrade_pip(venv_path)
 
-    if args.requirements:
-        print(f"VENV_INSTALLING_REQUIREMENTS: {args.requirements}")
-        install_requirements(venv_path, args.requirements)
-
     if args.toml:
         print(f"VENV_INSTALLING_PYPROJECT: {args.toml}")
         install_toml(venv_path, args.extras)
 
+    if args.requirements:
+        print(f"VENV_INSTALLING_REQUIREMENTS: {args.requirements}")
+        install_requirements(venv_path, args.requirements)
+
 
 if __name__ == "__main__":
     main(sys.argv[1:])

extensions/positron-python/resources/report_issue_user_settings.json

@@ -7,6 +7,7 @@
     "envFile": "placeholder",
     "venvPath": "placeholder",
     "venvFolders": "placeholder",
+    "activeStateToolPath": "placeholder",
     "condaPath": "placeholder",
     "pipenvPath": "placeholder",
     "poetryPath": "placeholder",

extensions/positron-python/src/client/activation/languageClientMiddlewareBase.ts

@@ -13,10 +13,10 @@ import {
 import { ConfigurationItem } from 'vscode-languageserver-protocol';
 
 import { HiddenFilePrefix } from '../common/constants';
-import { IConfigurationService } from '../common/types';
 import { createDeferred, isThenable } from '../common/utils/async';
 import { StopWatch } from '../common/utils/stopWatch';
 import { IEnvironmentVariablesProvider } from '../common/variables/types';
+import { IInterpreterService } from '../interpreter/contracts';
 import { IServiceContainer } from '../ioc/types';
 import { EventName } from '../telemetry/constants';
 import { LanguageServerType } from './types';
@@ -66,7 +66,7 @@ export class LanguageClientMiddlewareBase implements Middleware {
                 return next(params, token);
             }
 
-            const configService = this.serviceContainer.get<IConfigurationService>(IConfigurationService);
+            const interpreterService = this.serviceContainer.get<IInterpreterService>(IInterpreterService);
             const envService = this.serviceContainer.get<IEnvironmentVariablesProvider>(IEnvironmentVariablesProvider);
 
             let settings = next(params, token);
@@ -87,9 +87,10 @@ export class LanguageClientMiddlewareBase implements Middleware {
                     const settingDict: LSPObject & { pythonPath: string; _envPYTHONPATH: string } = settings[
                         i
                     ] as LSPObject & { pythonPath: string; _envPYTHONPATH: string };
-
                     settingDict.pythonPath =
-                        (await this.getPythonPathOverride(uri)) ?? configService.getSettings(uri).pythonPath;
+                        (await this.getPythonPathOverride(uri)) ??
+                        (await interpreterService.getActiveInterpreter(uri))?.path ??
+                        'python';
 
                     const env = await envService.getEnvironmentVariables(uri);
                     const envPYTHONPATH = env.PYTHONPATH;

extensions/positron-python/src/client/activation/node/analysisOptions.ts

@@ -43,22 +43,41 @@ export class NodeLanguageServerAnalysisOptions extends LanguageServerAnalysisOpt
     }
 
     private async isAutoIndentEnabled() {
-        const editorConfig = this.getPythonSpecificEditorSection();
-        const formatOnTypeInspect = editorConfig.inspect(FORMAT_ON_TYPE_CONFIG_SETTING);
-        const formatOnTypeSetForPython = formatOnTypeInspect?.globalLanguageValue !== undefined;
+        let editorConfig = this.getPythonSpecificEditorSection();
 
-        const inExperiment = await this.isInAutoIndentExperiment();
-        // only explicitly enable formatOnType for those who are in the experiment
+        // Only explicitly enable formatOnType for those who are in the experiment
         // but have not explicitly given a value for the setting
-        if (!formatOnTypeSetForPython && inExperiment) {
-            await NodeLanguageServerAnalysisOptions.setPythonSpecificFormatOnType(editorConfig, true);
+        if (!NodeLanguageServerAnalysisOptions.isConfigSettingSetByUser(editorConfig, FORMAT_ON_TYPE_CONFIG_SETTING)) {
+            const inExperiment = await this.isInAutoIndentExperiment();
+            if (inExperiment) {
+                await NodeLanguageServerAnalysisOptions.setPythonSpecificFormatOnType(editorConfig, true);
+
+                // Refresh our view of the config settings.
+                editorConfig = this.getPythonSpecificEditorSection();
+            }
         }
 
-        const formatOnTypeEffectiveValue = this.getPythonSpecificEditorSection().get(FORMAT_ON_TYPE_CONFIG_SETTING);
+        const formatOnTypeEffectiveValue = editorConfig.get(FORMAT_ON_TYPE_CONFIG_SETTING);
 
         return formatOnTypeEffectiveValue;
     }
 
+    private static isConfigSettingSetByUser(configuration: WorkspaceConfiguration, setting: string): boolean {
+        const inspect = configuration.inspect(setting);
+        if (inspect === undefined) {
+            return false;
+        }
+
+        return (
+            inspect.globalValue !== undefined ||
+            inspect.workspaceValue !== undefined ||
+            inspect.workspaceFolderValue !== undefined ||
+            inspect.globalLanguageValue !== undefined ||
+            inspect.workspaceLanguageValue !== undefined ||
+            inspect.workspaceFolderLanguageValue !== undefined
+        );
+    }
+
     private async isInAutoIndentExperiment(): Promise<boolean> {
         if (await this.experimentService.inExperiment('pylanceAutoIndent')) {
             return true;

extensions/positron-python/src/client/common/configSettings.ts

@@ -98,6 +98,8 @@ export class PythonSettings implements IPythonSettings {
 
     public venvFolders: string[] = [];
 
+    public activeStateToolPath = '';
+
     public condaPath = '';
 
     public pipenvPath = '';
@@ -254,6 +256,11 @@ export class PythonSettings implements IPythonSettings {
 
         this.venvPath = systemVariables.resolveAny(pythonSettings.get<string>('venvPath'))!;
         this.venvFolders = systemVariables.resolveAny(pythonSettings.get<string[]>('venvFolders'))!;
+        const activeStateToolPath = systemVariables.resolveAny(pythonSettings.get<string>('activeStateToolPath'))!;
+        this.activeStateToolPath =
+            activeStateToolPath && activeStateToolPath.length > 0
+                ? getAbsolutePath(activeStateToolPath, workspaceRoot)
+                : activeStateToolPath;
         const condaPath = systemVariables.resolveAny(pythonSettings.get<string>('condaPath'))!;
         this.condaPath = condaPath && condaPath.length > 0 ? getAbsolutePath(condaPath, workspaceRoot) : condaPath;
         const pipenvPath = systemVariables.resolveAny(pythonSettings.get<string>('pipenvPath'))!;

extensions/positron-python/src/client/common/experiments/service.ts

@@ -142,7 +142,7 @@ export class ExperimentService implements IExperimentService {
         // it means that the value for this experiment was not found on the server.
         const treatmentVariable = this.experimentationService.getTreatmentVariable(EXP_CONFIG_ID, experiment);
 
-        return treatmentVariable !== undefined;
+        return treatmentVariable === true;
     }
 
     public async getExperimentValue<T extends boolean | number | string>(experiment: string): Promise<T | undefined> {

extensions/positron-python/src/client/common/process/internal/scripts/index.ts

@@ -58,23 +58,6 @@ export function interpreterInfo(): [string[], (out: string) => InterpreterInfoJs
     return [args, parse];
 }
 
-// sortImports.py
-
-export function sortImports(filename: string, sortArgs?: string[]): [string[], (out: string) => string] {
-    const script = path.join(SCRIPTS_DIR, 'sortImports.py');
-    const args = [script, filename, '--diff'];
-    if (sortArgs) {
-        args.push(...sortArgs);
-    }
-
-    function parse(out: string) {
-        // It should just be a diff that the extension will use directly.
-        return out;
-    }
-
-    return [args, parse];
-}
-
 // normalizeSelection.py
 
 export function normalizeSelection(): [string[], (out: string) => string] {

extensions/positron-python/src/client/common/types.ts

@@ -184,6 +184,7 @@ export interface IPythonSettings {
     readonly pythonPath: string;
     readonly venvPath: string;
     readonly venvFolders: string[];
+    readonly activeStateToolPath: string;
     readonly condaPath: string;
     readonly pipenvPath: string;
     readonly poetryPath: string;

extensions/positron-python/src/client/common/utils/async.ts

@@ -27,7 +27,7 @@ export interface Deferred<T> {
     readonly rejected: boolean;
     readonly completed: boolean;
     resolve(value?: T | PromiseLike<T>): void;
-    reject(reason?: string | Error | Record<string, unknown>): void;
+    reject(reason?: string | Error | Record<string, unknown> | unknown): void;
 }
 
 class DeferredImpl<T> implements Deferred<T> {

extensions/positron-python/src/client/common/utils/localize.ts

@@ -466,6 +466,10 @@ export namespace ToolsExtensions {
     export const pylintPromptMessage = l10n.t(
         'Use the Pylint extension to enable easier configuration and new features such as quick fixes.',
     );
+    export const isortPromptMessage = l10n.t(
+        'To use sort imports, please install the isort extension. It provides easier configuration and new features such as code actions.',
+    );
     export const installPylintExtension = l10n.t('Install Pylint extension');
     export const installFlake8Extension = l10n.t('Install Flake8 extension');
+    export const installISortExtension = l10n.t('Install isort extension');
 }

extensions/positron-python/src/client/common/utils/multiStepInput.ts

@@ -8,6 +8,7 @@
 import { inject, injectable } from 'inversify';
 import { Disposable, QuickInput, QuickInputButton, QuickInputButtons, QuickPick, QuickPickItem, Event } from 'vscode';
 import { IApplicationShell } from '../application/types';
+import { createDeferred } from './async';
 
 // Borrowed from https://github.com/Microsoft/vscode-extension-samples/blob/master/quickinput-sample/src/multiStepInput.ts
 // Why re-invent the wheel :)
@@ -29,7 +30,7 @@ export type InputStep<T extends any> = (input: MultiStepInput<T>, state: T) => P
 
 type buttonCallbackType<T extends QuickPickItem> = (quickPick: QuickPick<T>) => void;
 
-type QuickInputButtonSetup = {
+export type QuickInputButtonSetup = {
     /**
      * Button for an action in a QuickPick.
      */
@@ -164,35 +165,41 @@ export class MultiStepInput<S> implements IMultiStepInput<S> {
         // so do it after initialization. This ensures quickpick starts with the active
         // item in focus when this is true, instead of having scroll position at top.
         input.keepScrollPosition = keepScrollPosition;
-        try {
-            return await new Promise<MultiStepInputQuickPicResponseType<T, P>>((resolve, reject) => {
-                disposables.push(
-                    input.onDidTriggerButton(async (item) => {
-                        if (item === QuickInputButtons.Back) {
-                            reject(InputFlowAction.back);
-                        }
-                        if (customButtonSetups) {
-                            for (const customButtonSetup of customButtonSetups) {
-                                if (JSON.stringify(item) === JSON.stringify(customButtonSetup?.button)) {
-                                    await customButtonSetup?.callback(input);
-                                }
-                            }
+
+        const deferred = createDeferred<T>();
+
+        disposables.push(
+            input.onDidTriggerButton(async (item) => {
+                if (item === QuickInputButtons.Back) {
+                    deferred.reject(InputFlowAction.back);
+                    input.hide();
+                }
+                if (customButtonSetups) {
+                    for (const customButtonSetup of customButtonSetups) {
+                        if (JSON.stringify(item) === JSON.stringify(customButtonSetup?.button)) {
+                            await customButtonSetup?.callback(input);
                         }
-                    }),
-                    input.onDidChangeSelection((selectedItems) => resolve(selectedItems[0])),
-                    input.onDidHide(() => {
-                        resolve(undefined);
-                    }),
-                );
-                if (acceptFilterBoxTextAsSelection) {
-                    disposables.push(
-                        input.onDidAccept(() => {
-                            // eslint-disable-next-line @typescript-eslint/no-explicit-any
-                            resolve(input.value as any);
-                        }),
-                    );
+                    }
                 }
-            });
+            }),
+            input.onDidChangeSelection((selectedItems) => deferred.resolve(selectedItems[0])),
+            input.onDidHide(() => {
+                if (!deferred.completed) {
+                    deferred.resolve(undefined);
+                }
+            }),
+        );
+        if (acceptFilterBoxTextAsSelection) {
+            disposables.push(
+                input.onDidAccept(() => {
+                    // eslint-disable-next-line @typescript-eslint/no-explicit-any
+                    deferred.resolve(input.value as any);
+                }),
+            );
+        }
+
+        try {
+            return await deferred.promise;
         } finally {
             disposables.forEach((d) => d.dispose());
         }
@@ -277,6 +284,9 @@ export class MultiStepInput<S> implements IMultiStepInput<S> {
                 if (err === InputFlowAction.back) {
                     this.steps.pop();
                     step = this.steps.pop();
+                    if (step === undefined) {
+                        throw err;
+                    }
                 } else if (err === InputFlowAction.resume) {
                     step = this.steps.pop();
                 } else if (err === InputFlowAction.cancel) {

extensions/positron-python/src/client/common/vscodeApis/extensionsApi.ts

@@ -0,0 +1,30 @@
+// Copyright (c) Microsoft Corporation. All rights reserved.
+// Licensed under the MIT License.
+
+import * as path from 'path';
+import * as fs from 'fs-extra';
+import { Extension, extensions } from 'vscode';
+import { PVSC_EXTENSION_ID } from '../constants';
+
+export function getExtension<T = unknown>(extensionId: string): Extension<T> | undefined {
+    return extensions.getExtension(extensionId);
+}
+
+export function isExtensionEnabled(extensionId: string): boolean {
+    return extensions.getExtension(extensionId) !== undefined;
+}
+
+export function isExtensionDisabled(extensionId: string): boolean {
+    // We need an enabled extension to find the extensions dir.
+    const pythonExt = getExtension(PVSC_EXTENSION_ID);
+    if (pythonExt) {
+        let found = false;
+        fs.readdirSync(path.dirname(pythonExt.extensionPath), { withFileTypes: false }).forEach((s) => {
+            if (s.toString().startsWith(extensionId)) {
+                found = true;
+            }
+        });
+        return found;
+    }
+    return false;
+}