Native server does not work with Jupyter Notebook without configuration #546

Xiao-Chenguang · 2024-07-21T18:19:57Z

Ruff not function in jupyter notebooks when set ruff.nativeServer to auto or on. It seems there are some communication problem between the nativeserver and notebook. When use ruff-lsp by setting ruff.nativeServer to off, all ruff functions works as expected.

According to the server log, the nativeserver started normally but receive nothing while I type format command.

2024-07-21 16:44:25.149 [info] Name: Ruff
2024-07-21 16:44:25.149 [info] Module: ruff
2024-07-21 16:44:25.149 [info] Python extension loading
2024-07-21 16:44:25.149 [info] Waiting for interpreter from python extension.
2024-07-21 16:44:25.149 [info] Python extension loaded
2024-07-21 16:44:25.242 [info] Falling back to bundled executable: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:44:25.265 [info] Resolved 'ruff.nativeServer: auto' to use the native server
2024-07-21 16:44:25.266 [info] Found Ruff 0.5.3 at /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:44:25.266 [info] Server run command: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff server
2024-07-21 16:44:25.266 [info] Server: Start requested.

The behaviour of setting nativeServer off is:

2024-07-21 16:48:31.470 [info] Name: Ruff
2024-07-21 16:48:31.470 [info] Module: ruff
2024-07-21 16:48:31.470 [info] Python extension loading
2024-07-21 16:48:31.470 [info] Waiting for interpreter from python extension.
2024-07-21 16:48:31.470 [info] Python extension loaded
2024-07-21 16:48:31.470 [info] Server run command: /Users/cg/miniconda3/envs/fl/bin/python /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/tool/server.py
2024-07-21 16:48:31.470 [info] Server: Start requested.
2024-07-21 16:48:31.934 [info] 2024-07-21 16:48:31,934 INFO Starting IO server

2024-07-21 16:48:31.975 [info] Workspace settings: []
2024-07-21 16:48:31.975 [info] Global settings: {
    "nativeServer": "off",
    "cwd": "/",
    "workspace": "/",
    "path": [],
    "ignoreStandardLibrary": true,
    "interpreter": [],
    "configuration": null,
    "importStrategy": "fromEnvironment",
    "codeAction": {
        "fixViolation": {
            "enable": true
        },
        "disableRuleComment": {
            "enable": true
        }
    },
    "lint": {
        "enable": true,
        "run": "onType",
        "args": []
    },
    "format": {
        "args": []
    },
    "enable": true,
    "organizeImports": true,
    "fixAll": true,
    "showNotifications": "off",
    "configurationPreference": "editorFirst",
    "showSyntaxErrors": true
}
2024-07-21 16:48:31.990 [info] Interpreter executable (/Users/cg/miniconda3/envs/fl/bin/ruff) not found
2024-07-21 16:48:31.990 [info] Falling back to bundled executable: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:48:32.025 [info] Inferred version 0.5.3 for: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:48:32.025 [info] Found ruff 0.5.3 at /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff
2024-07-21 16:48:32.025 [info] Running Ruff with: /Users/cg/.vscode/extensions/charliermarsh.ruff-2024.34.0-darwin-arm64/bundled/libs/bin/ruff ['check', '--force-exclude', '--no-cache', '--no-fix', '--quiet', '--output-format', 'json', '-', '--stdin-filename', '/Users/cg/Desktop/test.ipynb']

other ruff setting: all default
ruff version: v2024.34.0
OS: Mac
Python version: 3.10

Mininal example:

# test.ipynb
a = [[1, 2, 3, 4, 5], [6, 7, 8, 9, 10], [11, 12, 13, 14, 15], [16, 17, 18, 19, 20], [21, 22, 23, 24, 25]]

The text was updated successfully, but these errors were encountered:

charliermarsh · 2024-07-21T18:31:28Z

\cc @dhruvmanila

dhruvmanila · 2024-07-22T04:40:34Z

Can you provide the contents of the Ruff config in pyproject.toml / ruff.toml / .ruff.toml file? After astral-sh/ruff#12252, Ruff would only include the notebook files if it's included via either include or extend-include.

The reason it works for ruff-lsp is because it uses the ruff CLI directly via stdin and it would always provide diagnostics / formatting for a file which is passed directly:

2024-07-22 10:08:25.812 [info] Running Ruff with: /Users/dhruv/work/astral/ruff/target/debug/ruff ['check', '--force-exclude', '--no-cache', '--no-fix', '--quiet', '--output-format', 'json', '-', '--stdin-filename', '/Users/dhruv/playground/ruff/notebooks/test.ipynb']

This doesn't consider the include / extend-include config because the --force-exclude flag doesn't consider those settings.

dhruvmanila · 2024-07-22T04:44:05Z

You can also enabled debug logs for the native server using:

{
  "ruff.trace.server": "messages",
  "ruff.logLevel": "debug"
}

which outputs:

2024-07-22 10:03:54.657 [info] Using 'path' setting: /Users/dhruv/work/astral/ruff/target/debug/ruff
2024-07-22 10:03:55.471 [info] Found Ruff 0.5.2 at /Users/dhruv/work/astral/ruff/target/debug/ruff
2024-07-22 10:03:55.471 [info] Server run command: /Users/dhruv/work/astral/ruff/target/debug/ruff server --preview
2024-07-22 10:03:55.472 [info] Server: Start requested.
2024-07-22 10:03:55.476 [info] [Trace - 10:03:55 AM] Sending request 'initialize - (0)'.
2024-07-22 10:03:55.488 [info] [Trace - 10:03:55 AM] Received response 'initialize - (0)' in 12ms.
2024-07-22 10:03:55.489 [info] [Trace - 10:03:55 AM] Sending notification 'initialized'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending notification 'textDocument/didOpen'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/diagnostic - (1)'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending notification 'notebookDocument/didOpen'.
2024-07-22 10:03:55.492 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (2)'.
2024-07-22 10:03:55.493 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (3)'.
2024-07-22 10:03:55.494 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (4)'.
2024-07-22 10:03:55.510 [info]    0.019833583s DEBUG ThreadId(13) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode

2024-07-22 10:03:55.519 [info] [Trace - 10:03:55 AM] Received request 'client/registerCapability - (1)'.
2024-07-22 10:03:55.519 [info] [Trace - 10:03:55 AM] Sending response 'client/registerCapability - (1)'. Processing request took 1ms
2024-07-22 10:03:55.519 [info]    0.028470583s DEBUG ruff:worker:0 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/src/play.py

2024-07-22 10:03:55.520 [info]    0.029595292s  INFO     ruff:main ruff_server::server: Configuration file watcher successfully registered

2024-07-22 10:03:55.520 [info] [Trace - 10:03:55 AM] Received response 'textDocument/codeAction - (4)' in 26ms.

You can see that it only included the Python file (search for "Included path via") and not the notebook file.

But, once you include it in the config:

[tool.ruff]
extend-include = ["*.ipynb"]

It'll start including the notebook files as well.

Xiao-Chenguang · 2024-07-22T06:37:55Z

You can also enabled debug logs for the native server using:

{
  "ruff.trace.server": "messages",
  "ruff.logLevel": "debug"
}

which outputs:

2024-07-22 10:03:54.657 [info] Using 'path' setting: /Users/dhruv/work/astral/ruff/target/debug/ruff
2024-07-22 10:03:55.471 [info] Found Ruff 0.5.2 at /Users/dhruv/work/astral/ruff/target/debug/ruff
2024-07-22 10:03:55.471 [info] Server run command: /Users/dhruv/work/astral/ruff/target/debug/ruff server --preview
2024-07-22 10:03:55.472 [info] Server: Start requested.
2024-07-22 10:03:55.476 [info] [Trace - 10:03:55 AM] Sending request 'initialize - (0)'.
2024-07-22 10:03:55.488 [info] [Trace - 10:03:55 AM] Received response 'initialize - (0)' in 12ms.
2024-07-22 10:03:55.489 [info] [Trace - 10:03:55 AM] Sending notification 'initialized'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending notification 'textDocument/didOpen'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/diagnostic - (1)'.
2024-07-22 10:03:55.490 [info] [Trace - 10:03:55 AM] Sending notification 'notebookDocument/didOpen'.
2024-07-22 10:03:55.492 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (2)'.
2024-07-22 10:03:55.493 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (3)'.
2024-07-22 10:03:55.494 [info] [Trace - 10:03:55 AM] Sending request 'textDocument/codeAction - (4)'.
2024-07-22 10:03:55.510 [info]    0.019833583s DEBUG ThreadId(13) ruff_server::session::index::ruff_settings: Ignored path via `exclude`: /Users/dhruv/playground/ruff/.vscode

2024-07-22 10:03:55.519 [info] [Trace - 10:03:55 AM] Received request 'client/registerCapability - (1)'.
2024-07-22 10:03:55.519 [info] [Trace - 10:03:55 AM] Sending response 'client/registerCapability - (1)'. Processing request took 1ms
2024-07-22 10:03:55.519 [info]    0.028470583s DEBUG ruff:worker:0 ruff_server::resolve: Included path via `include`: /Users/dhruv/playground/ruff/src/play.py

2024-07-22 10:03:55.520 [info]    0.029595292s  INFO     ruff:main ruff_server::server: Configuration file watcher successfully registered

2024-07-22 10:03:55.520 [info] [Trace - 10:03:55 AM] Received response 'textDocument/codeAction - (4)' in 26ms.

You can see that it only included the Python file (search for "Included path via") and not the notebook file.

But, once you include it in the config:

[tool.ruff]
extend-include = ["*.ipynb"]

It'll start including the notebook files as well.

Thank you @dhruvmanila , setting extend-include = ["*.ipynb"] in a ruff.toml file works for me. I didn't use ruff config file before and I would suggest you put this as a notice at some place obvious.

juanitorduz · 2024-07-22T07:40:56Z

Thanks for the tip! I was having the same problem and

[tool.ruff]
extend-include = ["*.ipynb"]

solved it for me as well!

dhruvmanila · 2024-07-22T09:02:44Z

No worries. It makes sense for this change to be documented. I'm thinking of adding it to:

Do you have any other places in mind which will be more visible?

* make nb ruff astral-sh/ruff-vscode#546 * fix docs * fix quickstart * add types

Xiao-Chenguang · 2024-07-22T12:44:33Z

That's really helpful. Use it as an extension, I would first check the extension readme as you mentioned and then the extension setting. It may benefit users by adding this into extension setting and the notebook feature can be enabled global without need of toml file per project.

dhruvmanila · 2024-07-22T13:19:01Z

It may benefit users by adding this into extension setting and the notebook feature can be enabled global without need of toml file per project.

Yeah, I had a similar thought earlier but that would mean that running Ruff from the command-line would give different results compared to the editor. I'm not sure if there are any opinions on this but as the notebook files were always included for linting and formatting via ruff-lsp and there haven't been any user report, I guess that should be fine (?).

Xiao-Chenguang · 2024-07-22T13:40:12Z

I'm not sure if there are any opinions on this but as the notebook files were always included for linting and formatting via ruff-lsp and there haven't been any user report, I guess that should be fine (?).

Totally agree. Just keep the notebook behaviour similar to ruff-lsp before there is a better solution.

## Summary Similar to astral-sh/ruff-vscode#547 but for the online docs. Refer to astral-sh/ruff-vscode#546 ## Preview <img width="1728" alt="Screenshot 2024-07-22 at 14 51 40" src="https://github.com/user-attachments/assets/39014278-c868-45b0-9058-42858a060fd8">

## Summary This PR adds a new section in the README to guide users to opt-in for Jupyter Notebook support for the native server. This is required because, unlike `ruff-lsp`, the native server looks at the `include` / `extend-include` setting. Refer #546

dhruvmanila · 2024-07-22T16:27:32Z

We might just include Jupyter Notebooks in Ruff's inclusion by default (astral-sh/ruff#12456).

rnckp · 2024-07-26T08:37:19Z

@dhruvmanila

I still have difficulties making it work, even when I add extend-include = ["*.ipynb"] in ~.vscode/extensions/charliermarsh.ruff-2024.36.0-darwin-arm64/pyproject.toml.

.py files in VS Code are being formatted and linted properly. However, jupyer notebooks are not.

Tried to restart, reinstall from clean state (VS Code, settings and extensions) to no avail. Any recommendations to make it work again? I noticed that I stopped working a couple of days ago.

I use all latest versions of VS Code and extensions. Besides Ruff I have installed Python, Pylance, Python Debugger (all MS), Jupyter, Jupyter Keymap and Jupyter Notebook Renderers.

dhruvmanila · 2024-07-26T12:00:33Z

I still have difficulties making it work, even when I add extend-include = ["*.ipynb"] in ~.vscode/extensions/charliermarsh.ruff-2024.36.0-darwin-arm64/pyproject.toml.

I think you'd need to add that in the pyproject.toml file within the current project you're in. That directory is where the extension resides. For example, if you opened VS Code in ~/projects/ruff, then the root config file would be ~/projects/ruff/pyproject.toml.

rnckp · 2024-07-26T12:37:42Z

That did the trick! Thanks so much! You saved my day... 🙏☀️

A detail: In my case it was sufficient to create pyproject.toml just in the root folder of my project (aka the top code folder that I opened in my VS Code workspace). Subfolder /ruff/ didn't seem to be necessary.

dhruvmanila · 2024-07-26T12:52:19Z

~/projects/ruff is the root folder of the "ruff" project ;)

rnckp · 2024-07-26T14:07:27Z

Got it! Thanks! 😊

DGrady · 2024-07-31T23:50:20Z

We might just include Jupyter Notebooks in Ruff's inclusion by default (astral-sh/ruff#12456).

Upvoting this! Speaking as someone who mainly uses Ruff from within Jupyter notebooks running in VSCode, my experience here was: a few months ago, very happy that I got to replace two slow extensions with one fast one; then recent frustration when the fast one stopped working with no obvious errors.

Co-authored-by: Alex Waygood <alex.waygood@gmail.com> Closes: #12456 Closes: astral-sh/ruff-vscode#546

* make nb ruff astral-sh/ruff-vscode#546 * fix docs * fix quickstart * add types

charliermarsh assigned dhruvmanila Jul 21, 2024

This was referenced Jul 22, 2024

Guide to opt-in to Notebook support in native server #547

Merged

Add note to include notebook files for native server astral-sh/ruff#12449

Merged

juanitorduz added a commit to pymc-labs/pymc-marketing that referenced this issue Jul 22, 2024

make nb ruff astral-sh/ruff-vscode#546

218fa4c

juanitorduz mentioned this issue Jul 22, 2024

Various MMM small documentation fixes pymc-labs/pymc-marketing#854

Merged

wd60622 pushed a commit to pymc-labs/pymc-marketing that referenced this issue Jul 22, 2024

Various MMM small documentation fixes (#854)

e6f844f

* make nb ruff astral-sh/ruff-vscode#546 * fix docs * fix quickstart * add types

dhruvmanila mentioned this issue Jul 22, 2024

Add Jupyter Notebooks to Ruff's default inclusion astral-sh/ruff#12456

Closed

dhruvmanila added the notebook Related to (Jupyter) notebooks label Jul 23, 2024

dhruvmanila changed the title ~~nativeServer not work with jupyter notebook~~ Native server does not work with Jupyter Notebook without configuration Jul 23, 2024

dhruvmanila added the native-server Related to the native server label Jul 23, 2024

dhruvmanila mentioned this issue Aug 14, 2024

Stabilize support for Jupyter Notebooks astral-sh/ruff#12878

Merged

3 tasks

AlexWaygood added a commit to astral-sh/ruff that referenced this issue Aug 14, 2024

Stabilize support for Jupyter Notebooks (#12878)

524a86a

Co-authored-by: Alex Waygood <alex.waygood@gmail.com> Closes: #12456 Closes: astral-sh/ruff-vscode#546

AlexWaygood closed this as completed Aug 14, 2024

MichaReiser pushed a commit to astral-sh/ruff that referenced this issue Aug 14, 2024

Stabilize support for Jupyter Notebooks (#12878)

8da1c12

Co-authored-by: Alex Waygood <alex.waygood@gmail.com> Closes: #12456 Closes: astral-sh/ruff-vscode#546

MichaReiser pushed a commit to astral-sh/ruff that referenced this issue Aug 14, 2024

Stabilize support for Jupyter Notebooks (#12878)

73160dc

Co-authored-by: Alex Waygood <alex.waygood@gmail.com> Closes: #12456 Closes: astral-sh/ruff-vscode#546

twiecki pushed a commit to pymc-labs/pymc-marketing that referenced this issue Sep 10, 2024

Various MMM small documentation fixes (#854)

ae91e8f

* make nb ruff astral-sh/ruff-vscode#546 * fix docs * fix quickstart * add types

twiecki pushed a commit to pymc-labs/pymc-marketing that referenced this issue Sep 10, 2024

Various MMM small documentation fixes (#854)

15d993b

* make nb ruff astral-sh/ruff-vscode#546 * fix docs * fix quickstart * add types

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Native server does not work with Jupyter Notebook without configuration #546

Native server does not work with Jupyter Notebook without configuration #546

Xiao-Chenguang commented Jul 21, 2024 •

edited

Loading

charliermarsh commented Jul 21, 2024

dhruvmanila commented Jul 22, 2024

dhruvmanila commented Jul 22, 2024

Xiao-Chenguang commented Jul 22, 2024

juanitorduz commented Jul 22, 2024

dhruvmanila commented Jul 22, 2024

Xiao-Chenguang commented Jul 22, 2024

dhruvmanila commented Jul 22, 2024

Xiao-Chenguang commented Jul 22, 2024

dhruvmanila commented Jul 22, 2024

rnckp commented Jul 26, 2024 •

edited

Loading

dhruvmanila commented Jul 26, 2024

rnckp commented Jul 26, 2024 •

edited

Loading

dhruvmanila commented Jul 26, 2024

rnckp commented Jul 26, 2024 •

edited

Loading

DGrady commented Jul 31, 2024

Native server does not work with Jupyter Notebook without configuration #546

Native server does not work with Jupyter Notebook without configuration #546

Comments

Xiao-Chenguang commented Jul 21, 2024 • edited Loading

charliermarsh commented Jul 21, 2024

dhruvmanila commented Jul 22, 2024

dhruvmanila commented Jul 22, 2024

Xiao-Chenguang commented Jul 22, 2024

juanitorduz commented Jul 22, 2024

dhruvmanila commented Jul 22, 2024

Xiao-Chenguang commented Jul 22, 2024

dhruvmanila commented Jul 22, 2024

Xiao-Chenguang commented Jul 22, 2024

dhruvmanila commented Jul 22, 2024

rnckp commented Jul 26, 2024 • edited Loading

dhruvmanila commented Jul 26, 2024

rnckp commented Jul 26, 2024 • edited Loading

dhruvmanila commented Jul 26, 2024

rnckp commented Jul 26, 2024 • edited Loading

DGrady commented Jul 31, 2024

Xiao-Chenguang commented Jul 21, 2024 •

edited

Loading

rnckp commented Jul 26, 2024 •

edited

Loading

rnckp commented Jul 26, 2024 •

edited

Loading

rnckp commented Jul 26, 2024 •

edited

Loading