Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Xeus incorporation #44

Merged
merged 3 commits into from
Dec 14, 2019
Merged

Conversation

SylvainCorlay
Copy link
Member

@SylvainCorlay SylvainCorlay commented Nov 30, 2019

Copy link
Member

@minrk minrk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is great, and happy to include Xeus. The only downside I see to including xeus is that incorporating it into Jupyter makes the vibrant third-party kernel community look a little less great, as we bring one of the best bits in as first-party :).

I know naming things is hard, but jupyter-native seems like a confusing name. What's it native to? It's not native to Jupyter, etc. I know it's native to the processor because it's compiled, but the phrase "jupyter-native" doesn't communicate that as well, and the term "native kernels" also doesn't seem a good fit for describing kernels that are compiled. Can we not just keep the name xeus? We could use jupyter-xeus for the GitHub org if folks want to make the Jupyter subproject relationship clear, but still call everything xeus.

@SylvainCorlay
Copy link
Member Author

I know naming things is hard, but jupyter-native seems like a confusing name. What's it native to? It's not native to Jupyter, etc. I know it's native to the processor because it's compiled, but the phrase "jupyter-native" doesn't communicate that as well, and the term "native kernels" also doesn't seem a good fit for describing kernels that are compiled. Can we not just keep the name xeus? We could use jupyter-xeus for the GitHub org if folks want to make the Jupyter subproject relationship clear, but still call everything xeus.

I am not opinionated at all about that name. We thought of "native" because it is mostly compiled code. I am also OK with jupyter-xeus...

@jasongrout
Copy link
Member

jasongrout commented Dec 3, 2019

Point of order: I just added a voting checklist to the OP. Now we have three ways to vote, the emojis, the github approval system, and the checklist. I've updated the checklist to merge each of these results.

  • emoji: Easiest to vote, but provides no audit trail of the vote and votes can be silently removed
  • checklist: Easy to vote, pings all steering committee members. Can record votes for others if they approved elsewhere or in other ways, and there is an audit trail of who recorded the vote, though the audit trail (the comment history) is not very easy to see (so perhaps is not so useful in practicality). Can be used on issues and PRs.
  • github approvals: Nicely summarized in issue and in sidebar, has very understandable audit trail. Hardest to vote and OP cannot vote. Reviews can be requested, but seems like we can't request the entire steering council easily. Can only be used on PRs.

@jasongrout
Copy link
Member

I know naming things is hard, but jupyter-native seems like a confusing name. What's it native to? It's not native to Jupyter, etc. I know it's native to the processor because it's compiled, but the phrase "jupyter-native" doesn't communicate that as well, and the term "native kernels" also doesn't seem a good fit for describing kernels that are compiled. Can we not just keep the name xeus? We could use jupyter-xeus for the GitHub org if folks want to make the Jupyter subproject relationship clear, but still call everything xeus.

I agree that jupyter-xeus would be a better name for the Github org, or even just xeus.

@saulshanabrook
Copy link

saulshanabrook commented Dec 11, 2019

I opened a discussion earlier today on the Jupyter Debugger repo about using xeus Python kernel vs ipykernel (jupyterlab/debugger#274). At the JupyterLab dev meeting, some folks suggested I move the discussion here to encourage more participation and visibility.

The impetus is that we (@Quansight) have a client who wants us to help move forward the debugging support in JupyterLab. Currently, if they were to begin adopting it with a Python kernel, they would have to switch to xeus-python instead of ipykernel, because xeus-python has support for the debugging protocol and ipykernel does not. So the question is, should we put our resources towards making sure xeus-python works for them or on adding debugger support for ipykernel?

More broadly, in the JupyterLab context, when Jupyter debugger is stable and we start encouraging use of it, we will have the same question. Of course technically we can tell users that if they want debugging support they have to switch kernels and if they want any features that ipykernel has that are not supported in xeus-python, then they should stick with that, but that isn't a satisfying answer from a UX perspective. When a new user comes to JupyterLab, eventually we would like them to be able to start debugging in Python with minimal friction.

My understanding is that adding debugging support to ipykernel would require a significant refactor. So my hope is to get a sense from folks who are currently, or have been, active in ipykernel and those in xeus-python (cc @minrk @ivanov @Carreau). whether we can put together a plan to either:

  1. End up with xeus-python as the default python kernel
  2. Refactor ipykernel so it can support the debugging protocol

I don't have a horse in the race, but just want to understand which would be useful for us to allocate resources towards, as client needs come up that affect the Python kernel.

On a meta level, I am open to any feedback or suggestions on where this conversation should be held and how to move it forward in a way that is inclusive and transparent.

EDIT: Here are some links to

I am not sure what it would take to make xeus-python feature-complete with ipykernel. Someone mentioned something about magics offhand, but I personally have never used xeus-python and am not sure about its status.

@meeseeksmachine
Copy link

This pull request has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/ipykernel-vs-xeus-kernel-discussion/2877/1

@SylvainCorlay
Copy link
Member Author

Thanks for opening this @saulshanabrook although I don't think that the JEP for incorporating xeus is the right place because for this. The JEP is not making a statement about an official recommended Python kernel, and it covers a broader set of projects, including xeus itself (the C++ library), xeus-cling and companion projects.

@saulshanabrook
Copy link

@SylvainCorlay Could you make a recommendation for where we should discuss this then?

@SylvainCorlay
Copy link
Member Author

SylvainCorlay commented Dec 11, 2019

@SylvainCorlay Could you make a recommendation for where we should discuss this then?

Sure, I think that both ipykernel or xeus-python are fine. Although it seems you just opened issues and threads on all channels already

  • discourse
  • the mailing list
  • twitter
  • the xeus-python repository
  • the debugger repository
  • the ipykernel repository
  • and here.

so I would just pick one of them and close the others!

@saulshanabrook
Copy link

Although it seems you just opened issues and threads on all channels already (discourse, xeus-python, debugger, ipykernel and here) so I would just pick one of them and close the others!

OK since I opened an issue on the debugger already, we can have the conversation over there: jupyterlab/debugger#274

@SylvainCorlay
Copy link
Member Author

OK since I opened an issue on the debugger already, we can have the conversation over there: jupyterlab/debugger#274

OK thanks @saulshanabrook

@willingc
Copy link
Member

It may make sense to remove reference to the debugger in the JEP. I was a bit surprised by the wording since it seems to imply movement from ipykernel to the xeus-python kernel. If this is not the case, I think this would read more clearly by removing the following part, which does not seem critical to the JEP itself. Alternatively, I would be more explicit about what is meant by the core library and where ipykernel fits in relationship to xeus-python.

Beyond the xeus-based kernels, there is a new push in the project motivated by the development of the JupyterLab visual debugger. We think that the core library is fairly complete already and that most of the work will be done in xeus-based language kernels such as xeus-cling and xeus-python.

Copy link
Member

@willingc willingc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I approve conditionally on clarification re: ipykernel and xeus-python in core library of debugger or removal of that wording.

@SylvainCorlay
Copy link
Member Author

@willingc I have modified the wording into

Beyond the xeus-based kernels, there is a new push in the project motivated by the development of the JupyterLab visual debugger. We think that the core library is fairly complete already and that most of the work already done will apply to other xeus-based language kernels such as xeus-cling.

Are you fine with this?

@willingc
Copy link
Member

Pretty much @SylvainCorlay. Perhaps clarify which core library. The visual debugger core library or xeus core library. Thanks!

@SylvainCorlay
Copy link
Member Author

Pretty much @SylvainCorlay. Perhaps clarify which core library. The visual debugger core library or xeus core library. Thanks!

Done. I meant the core xeus library (not xeus-python). The front-end debugger UI will work with any kernel supporting the protocol.

@Carreau
Copy link
Member

Carreau commented Dec 11, 2019

Can (Could) xeus-python use IPython as an execution engine ? What needs to change for this to happen ? My guess is that users do not really care about the ipykernel features which are super-minimal but the IPython features (magics, tracebacks, etc...)

Voted +1 in principle about adopting xeus.

@SylvainCorlay
Copy link
Member Author

Can (Could) xeus-python use IPython as an execution engine ?

Probably, although it would probably make the codebase more complicated, and may not be the best approach to reach full feature parity (cf second part of the answer).

(magics, tracebacks, etc...)

We already support rich tracebacks, autocompletion, rich mimetype rendering, widgets, but not magics yet.

  • although we have looked into enabling all IPython magics in xeus-python when IPython is present in the environment.
  • I would like to have two levels of magics in xeus-based kernels: language-agnostic magics that we could implement once for any kernel, and kernel-specific magics such as the IPython ones.

@blink1073
Copy link
Contributor

Given that we have 11 approvals from SC members and no disagreements, I propose we merge this tomorrow. We can continue discussions about ipykernel vs. xeus-kernel and debugging on other channels.

@Carreau
Copy link
Member

Carreau commented Dec 13, 2019

We already support rich tracebacks, autocompletion, rich mimetype rendering, widgets, but not magics yet.

Let me rephrase then; I'm concern about the difficulty to maintain similar behavior in two different project. For example current work in IPython to get better stacktraces: ipython/ipython#11827

I think there is a good case to pull some of the functionalities into a ipython_core that could be reuse and guaranty identical behavior between IPython/Xeus-Python(/maybe xonsh)

But all of that is an implementation detail.

@SylvainCorlay
Copy link
Member Author

I think there is a good case to pull some of the functionalities into a ipython_core that could be reuse and guaranty identical behavior between IPython/Xeus-Python(/maybe xonsh)

Yes - similarly, many libraries meant to be use in Jupyter make use of IPython APIs (for example display, get_ipython) I was looking at a common denominator that could be defined as an abstract API... This is not quite about converging on behavior though.

@rgbkrk
Copy link
Member

rgbkrk commented Jan 6, 2020

Belated 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants