can you please add Rstudio type dataframe-variable tree explorer?

[data-steve]

Admittedly this is a feature request, not a bug issue, but still....

So much value could be created with this one feature addition of having a dedicated pane to some of these project level views. None of the other python IDEs do this well and have any speed or aesthetic.

Noone has really given jupyter a run for that intermediate space between RStudio as the best IDE and Jupyter allowing you to work in multiple data science stacks/languages.

I love that you guys aren't looked into the cells behavior (R Notebooks went that way too which is better in my opinion).

[lgeiger]

@data-steve That would be a awesome feature, though that's quite hard to do in a language agnostic way. We discussed something similar in #64.

That would be a excellent plugin idea, which then can be language specific for R or Python and easier to implement. We could totally expose more functionality in our plugin API for this.

Sadly I must say don't hold your breath. Unless you or someone else takes a stab at implementing something like this (which would be super cool 😄) , it is currently on the back burner.

[BenRussert]

Agreed this would be great. I think react inspector might be a good starting place.

I might experiment with this as a plugin and hopefully get some discussion going here if anyone else has some ideas/code.

[data-steve]

Awesome! As I said, if someone can reproduce the power of RStudio for other languages, they will get the kind of lock-in that RStudio is getting and building a business on top of. I just posted a similar request at Continuum, the biggest real player in fully open-source for data science, aside from RStudio. Continuum is multi-lingual at least in their language distribution service . Feels like Jupyter overcommitted to the notebook concept and is dropping the ball on this one. Visual Studio is doing Rtools and PythonTools plugins but still isn’t that compelling. Rodeo is too slow and buggy, freezes up when data is of any real size. ~ Steve Sent via telepathy

…

On Mar 23, 2017, at 10:01 AM, BenRussert ***@***.***> wrote: Agreed this would be great. I think react inspector <http://xyc.github.io/react-inspector/> might be a good starting place. I might experiment with this as a plugin and hopefully get some discussion going here if anyone else has some ideas/code. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#628 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHefi5kMB6Zjav1gFmZgIy10ja2mjgcbks5rontPgaJpZM4MQjCX>.

[lgeiger]

Rereading the issue makes me think that we may be able to support this simply by adding support for the application/json output type and utilizing the watch sidebar.
This way we don't have a complete variable explorer that shows all available variables from the current scope, instead one can add custom variables or expression to watch, similar to a debugger (we already support this). This has the advantage of being language agnostic as long as the library supports the application/json mime type.

nteract already has support for interacting with objects and exports the relevant React components in @nteract/display-area and @nteract/tranforms using react-json-tree. I think using this together with our Watch Sidebar would bring us very close to the desired behavior.

@BenRussert @data-steve Awesome that you are interested in experimenting with this! I started #655 which tries to bring in @nteract/tranforms to adds support for the above mentioned object output and more.
I would be super happy to have more people collaborating on this effort. 🎉

[data-steve]

I wish I could help more than just cheering and wishlisting things. I have zero experience with React and barely any experience with javascript beyond basic web and d3 stuff. I’m very happy to be a guinea pig though. Hydrogen has become my default editor for a lot of quick stuff. One question related to what you mention in terms of a sidebar. Everytime I saved the file I’m working inall the visualizations / cross tabs etc disappear which makes it hard to do much long-running experimentation because I have to keep going back to re-run plots/crosstabs and possibly underlying data has changed along the way. That’s one big of advantage of having a plot viewer on the side or console/output in Jupyter. Is that a sidebar thing as well? ~ Steve Sent via telepathy

…

On Mar 23, 2017, at 10:44 AM, Lukas Geiger ***@***.***> wrote: Rereading the issue makes me think that we may be able to support this simply by adding support for the application/json output type and utilizing the watch sidebar. This way we don't have a complete variable explorer that shows all available variables from the current scope, instead one can add custom variables or expression to watch, similar to a debugger (we already support this). nteract <https://github.com/nteract/nteract> already has support for interacting with objects <https://cloud.githubusercontent.com/assets/836375/19217520/db6e4fc8-8daa-11e6-98fb-266c15466a65.png> and exports the relevant React components in @nteract/display-area <https://github.com/nteract/nteract/tree/master/packages/display-area> and @nteract/tranforms <https://github.com/nteract/nteract/tree/master/packages/transforms> using react-json-tree <https://github.com/alexkuz/react-json-tree>. I think using this together with our Watch Sidebar would bring us very close to the desired behavior. @BenRussert <https://github.com/BenRussert> @data-steve <https://github.com/data-steve> Awesome that you are interested in experimenting with this! I started #655 <#655> which tries to bring in @nteract/tranforms to adds support for the above mentioned object output and more. I would be super happy to have more people collaborating on this effort. 🎉 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#628 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHefi_oG5x9INsUmirCYYx5nKOXMeiLYks5rooVAgaJpZM4MQjCX>.

[BenRussert]

simply by adding support for the application/json output type and utilizing the watch sidebar.

@lgeiger I was really hoping you'd say something like this! I think this is a good fit for Hydrogen because it allows you to visualize data and objects and think about their structure as you write code.

[rgbkrk]

This is something very common that I think we should be able to support across languages. However, when I say "we" I mean across the Jupyter ecosystem (kernels and frontends like Hydrogen).

We should start drafting a new message for the Jupyter protocol that could request available data frames or variables that are able to be exposed. Here's an example, using the same simplified psuedo-format as the building on jupyter post:

vars_request: {
  variables: [ // optional, would request all marked as available
    'x' // only request x
  ],
}

vars_response: {
  available: {
    x: { // some representation (could be mimetype style)
    }
  }
}

I'm thinking the frontend would cache as much as possible and when the user tries to expand a variable (or some other way of listing it), it would request to get the latest version.

---

As an aside, python provides both dir() for listing currently available objects and there's a __dir__ method that can be provided on an object to only show certain attributes for listing.

[lgeiger]

I wish I could help more than just cheering and wishlisting things. I have zero experience with React and barely any experience with javascript beyond basic web and d3 stuff.

No problem, don't hesitate to hack around in Hydrogen and ask questions. I'm just getting started with React as well. We're very happy to receive PRs even (or better: especially) if they are titled "This breaks everything but it's a cool idea" 😄
Plus their are a lot of super valuable ways to get involved, beyond just writing code. If you're interested please take a look at our new-contributor-friendly issues 🎉

Is that a sidebar thing as well?

I don't fully understand, but this GIF describes the behavior best: https://github.com/nteract/hydrogen#watch-expressions
Every time code is run, the plot is recomputed.

I was really hoping you'd say something like this!

Cool! Happy to have you on board! 👍

[lgeiger]

@rgbkrk Indeed a vars_request would be really cool and would allow frontends to implement a variable inspector supporting multiple languages like the one proposed in #64.

Based on your pseudo message, a general question: How would a vars_request for a single variable x differ from a execute_request with content x?

[rgbkrk]

Based on your pseudo message, a general question: How would a vars_request for a single variable x differ from a execute_request with content x?

😆 Pretty much the same.

I was thinking that an empty vars_request would return a response with all of them

vars_request: {
}

However, there's a semantic difference here between execute and "vars", which is that execute's intention is to run code. Here we want to look at a variable in memory (this is up to the kernel). As a user, running:

>>> x

is "looking" at the variable, yet could have side effects. By posting the vars_request message, we're explicitly saying to view (and hope to be free of side effects).

⚠️ I'm just throwing these ideas around, so I wouldn't hold any of my thoughts here as concrete. ⚠️

[lgeiger]

By posting the vars_request message, we're explicitly saying to view (and hope to be free of side effects).

True, that may be very valuable for a kernel to know in certain cases.

I'm just throwing these ideas around, so I wouldn't hold any of my thoughts here as concrete

😂 Sure, me too

[data-steve]

On Mar 23, 2017, at 1:27 PM, Lukas Geiger ***@***.***> wrote: Is that a sidebar thing as well? I don't fully understand, but this GIF describes the behavior best: https://github.com/nteract/hydrogen#watch-expressions <https://github.com/nteract/hydrogen#watch-expressions> Every time code is run, the plot is recomputed.

I guess I just mean, like in RStudio, you run code from your script or console and it opens in a plot viewer, from my understanding of this Toggle Watches feature https://github.com/nteract/hydrogen#watch-expressions <https://github.com/nteract/hydrogen#watch-expressions> that viewer only tracks registered variables? But in a data science workflow, it much more common to want to keep track of generated plots on a variety of variables than necessarily track how only a tracked variable changes as the same variable updates. More generally, I’d say the Watches panel would be much more useful as just a general purpose plot viewer / trackers as in the gif above. In RStudio you can download/delete/etc the plots directly from the viewer. Also, this seems like a nice enough feature to button-ize because I’d be using it all the time. Having to Control Panel and search for such an odd phrasing like Hydrogen: Toggle Watches is hard to remember ~ Steve Sent via telepathy

[lgeiger]

Definitely something that would be great to look into.

Also, this seems like a nice enough feature to button-ize because I’d be using it all the time. Having to Control Panel and search for such an odd phrasing like Hydrogen: Toggle Watches is hard to remember

In Atom you can allways add custom keybindings by editing the keymaps.cson file. Though you're right a default shortcut would be handy. Any ideas?

[data-steve]

Also, is there a way to export the resulting script with visualizations and cross tabs having been run as a html file?

…

On Mar 23, 2017, at 5:05 PM, Lukas Geiger ***@***.***> wrote: Definitely something that would be great to look into. Also, this seems like a nice enough feature to button-ize because I’d be using it all the time. Having to Control Panel and search for such an odd phrasing like Hydrogen: Toggle Watches is hard to remember In Atom you can allways add custom keybindings by editing the keymaps.cson file. Though you're right a default shortcut would be handy. Any ideas? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#628 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHefi2y2cyl_kFEfCYatfqmfeTQRfvbMks5rot6pgaJpZM4MQjCX>.

[abalter]

I believe that the JupyterLab roadmap lists a variable inspector as a future development. And JupyterLab is under very active development. Maybe sharing some ideas with them could be useful.

[lgeiger]

@abalter That sounds great!

@data-steve In the next release we'll have support for JSON data structures:

To keep it clear, I'm going to close this issue.
We can continue the awesome discussion about the variable explorer in #64 and open new issues for other ideas.