Test adoption of Python API with the New and Old Kernel Picker

[DonJayamanne]

Testing adoption of the new Python Extension API for discovering Python Enviornmnents

Refs: #11904

• anyOS @rebornix
• Windows @amunger
• WSL @roblourens
• macOS @jrieken
• anyOS @weinand

Complexity: 5

Authors: @DonJayamanne , @rebornix

Create Issue

---

Notes

Use the setting Kernel Picker Type to toggle between regular kernel picker (all) and the new one (mru)

Pre-Test

• Keep track of the kernels displayed in the kernel picker

Requirements

• Install pre-release Jupyter extension
• Install pre-release Python extension
• Install VS Code Insiders

Scenarios/items to test (Note: test with old and new kernel pickers)

• Verify you can use a Julia or .Net kernel with an ipynb notebook

• Verify discovery status is displayed in the kernel status (where you see selected kernel)

  • Reload vscode and this should show up
  • Clear jupyter and python extension cache and reload, and this progress indicator should still show up (this time a bit longer than usual)

• Verify the same kernels (kernel specs and Python environments) are still displayed in the kernel picker

• New Virtual environments are detected and displayed in the Kernel picker
  Note: You need to have already executed a cell in some notebook
  Then create a virtual environment within VS Code

  • Verify the new env shows up in kernel picker without having to refresh nor reload vscode.
    I.e. the goal is to test detection of new virtual envs after the extension has been used for a while
  • Verify you are prompted to install IPyKernel into this Python enviornment when running a Notebook cell against this kernel
  • Verify you are prompted to install IPyKernel into this Python enviornment when running an Interactive Window cell such a new Python Environment (either re-create a new env or uninstall IPyKernel)
  • Verify you are prompted to install Pandas data frame into this Python enviornment when using the Data Frame viewer
    To test this, create a few variables, at least one with a list e.g. a = [1,2,3] and use the variable viewer to open the variable a

• New Conda environments are detected and displayed in the Kernle picker
  Note: You need to have already executed a cell in some notebook
  Then create a conda environment within VS Code

  • Verify the new env shows up in kernel picker without having to refresh nor reload vscode.
    I.e. the goal is to test detection of new virtual envs after the extension has been used for a while
  • Verify you are prompted to install IPyKernel into this Python enviornment when running a Notebook cell against this kernel
  • Verify you are prompted to install IPyKernel into this Python enviornment when running an Interactive Window cell such a new Python Environment (either re-create a new env or uninstall IPyKernel)
  • Verify you are prompted to install Pandas data frame into this Python enviornment when using the Data Frame viewer
    To test this, create a few variables, at least one with a list e.g. a = [1,2,3] and use the variable viewer to open the variable a

• Re-load VS Code and verify you can pick a Kernel Spec before Python environments
  I'm hoping some machines are slow enough and we can see Kernel Specifications before Python environments in the kernel picker.
  In the past we regressed in this space and Jupyter extension waited for ALL python environnents to be loaded before displaying Kernel Specs on disc.
  We're verifyhing the fact that some kernels will appear before others

• Verify local kernel specs appera before Remote Kernel specs
  Connect to a local Jupyter server after having started a jupyter server using jupyter notebook --no-browser --NotebookApp.allow_origin=*
  In the past we regressed in this space and Jupyter extension waited for ALL python environnents to be loaded before displaying Kernel Specs on disc.
  We're verifyhing the fact that some kernels will appear before others

• Run a cell against a virtual env/or conda env and then delete that virtual/conda env and verify the selected kernel no longer points to that deleted environment.

• Delete a virtual python environment and veriy the kernel is no longer listed in your kernel picker

  • Verify it doesn't show up in mru
  • Verify it doesn't show up in python environments (within the kernel picker)

• Delete a conda python environment and veriy the kernel is no longer listed in your kernel picker

  • Verify it doesn't show up in mru
  • Verify it doesn't show up in python environments (within the kernel picker)

• Connect to multiple remote jupyter servers

  • Verify reloading vscode will display the last used remote kernels in MRU

• Test the new Kernel Picker (go into settings and toggle kernel picker to all -> mru or mru -> all)

  • Try all of the above
  • Try clearing the cache in Jupyer and Python extension
  • To clear cache in Jupyter and add "jupyter.development": true, to your user settings and then use the command Clear Cache (dev)
  • To clear cache in Python use the command Python: Clear Cache and Reload Window

[DonJayamanne]

@weinand there's a lot here, but what ever you can do to help would be great.
even sharing your getting started experience , ie no need to complete the whole plan (as you might not know about most of the steps)

[weinand]

@DonJayamanne I had a notebook setup from some previous experiments and I tried to understand the differences between the "traditional" and the "experimental" picker.

"traditional":

"experimental":

From my very limited experience and understanding of notebooks kernels I prefer the "traditional" over the "experimental" picker:

The "traditional" picker is a real picker because it shows what is available and I can actually "pick" something.

The new selection is more like a "Get Info" window that shows the currently selected values.
I find it confusing that I can only select a kernel, but not the Python version. Also, I don't understand why an entry from the "PyEnv env" section is displayed when the currently selected Python version is from the "Global Env" section. In the "traditional" selection, this problem does not exist, since the two sections show "all" items, but only one is marked as "Currently Selected".

[roblourens]

My impression of the slowness that @DonJayamanne mentioned

In stable and insiders, I think the run button shows up at about the same time. Around 7-9s from "reload window" to the run button appearing, hard to tell whether stable is slightly faster.

But after it appears, and the preferred kernel has been discovered, I see "Discovering Kernels" for another 5s or more, which could lead to an impression of slowness if the user doesn't realize they can go ahead and run a cell at this point.

[roblourens]

Clear jupyter and python extension cache

Is this just running Python: Clear Cache and reload window?

[amunger]

@roblourens - jupyter has a clear cache command as well if you add the setting "jupyter.development": true

[roblourens]

I did the best I could testing on WSL, but it's hard just due to everything in the windows VM being really slow. On the latest attempt, "Detecting Kernels" has been running for half an hour. So I did a lot of testing on mac and with SSH as well

I saw this once or twice, but I can't really repro it. Let me know if you want an issue:

• Opening the kernel picker and seeing the kernel source picker immediately, as if there are no mru entries, like in Empty MRU- back button does nothing vscode#167768, but there should have been mru entries. Closing/reopening the notebook, then seeing the mru entries appear