HTML system diagrams

[BenPortner]

This PR adds an html output format for system diagrams. The format is equivalent to svg except it adds informative tooltips:

Open issues:

• use CDN versions of digraph.js and digraph.css to make them work independent of output file location
• add tooltips for other unit types (currently only compressors)
• find out why tippy.js adds white space to node tooltips
• add tooltips only if output format is html (Yoel edit: or svg?)
• make sure it works for save_digraph and display_digraph (Fixed! Just needed IFrame for html with javascript)
• add essential unit parameters (e.g. efficiency) to tooltip
• add GIF to home page of readthedocs and show how to save diagram in tutorial (Yoel will do this in a couple of days)
• make stream tooltips a nice pandas series (or 1 column dataframe).

Help needed

@yoelcortes: Do you know a clever way to include other unit types without manually modifying each _graphics.tailor_xyz_node function? Perhaps we can make these part of the class definitions and have _unit.Unit provide a default tooltip function?

[yoelcortes]

@BenPortner, this is awesome. It is also great to see you added it for streams too. A good way to make this work for all units is to add it in the Unit.get_node method.

Regarding adding tooltips only if format is html, we can have a new biosteam.preferences.diagram_format attribute that defaults to 'html'. Then in System.diagram, have something like this:

        preferences = bst.preferences
        with preferences.temporary(): # This makes any changes to preferences temporary
            if number is not None: preferences.number_path = number
            if label is not None: preferences.label_streams = label
            if profile is not None: preferences.profile = profile
            if format is not None: preferences.diagram_format = format # Override if not given
            if kind == 0 or kind == 'cluster':
                f = self._cluster_digraph(graph_attrs)
            elif kind == 1 or kind == 'thorough':
                f = self._thorough_digraph(graph_attrs)
            elif kind == 2 or kind == 'surface':
                f = self._surface_digraph(graph_attrs)
            elif kind == 3 or kind == 'minimal':
                f = self._minimal_digraph(graph_attrs)
            else:
                raise ValueError("kind must be one of the following: "
                                 "0 or 'cluster', 1 or 'thorough', 2 or 'surface', "
                                 "3 or 'minimal'")
            if display or file:
                # Format will not be passed to finalize_diagram anymore.
                # Instead, the format should be accessed from biosteam.preferences.
                finalize_digraph(f, file)
            else:
                return f

I have a question regarding output format and tooltips. In System.diagram, we have the following code:

if format == 'html': graph_attrs['format'] = 'svg' # because html will cause error

Is this temporary? I thought the format would need to be 'html' to inject js in finalize_digraph

Thanks!

[BenPortner]

Creating tooltips in Unit.get_node(): That should work. I'll try this later!

Having a biosteam.preferences.diagram_format attribute is fine. However, I think the priority order should be opposite. All functions, which need a format attribute should accept one in their signature. Only if None is given, the format should be taken from the preferences. More generally, I think this should be adopted by all functions and all preferences. It makes the functions more readable and makes sure they can be tested without having to create a global preference object first.

I have a question regarding output format and tooltips. In System.diagram, we have the following code:

if format == 'html': graph_attrs['format'] = 'svg' # because html will cause error

Is this temporary? I thought the format would need to be 'html' to inject js in finalize_digraph

Unfortunately, overwriting format in the graph_attrs is necessary to make graphviz work (graphviz does not accept format=html). However, one could remove the format field from the graph_attrs altogether. Then it should work.

[yoelcortes]

@BenPortner, that makes sense, we can keep the format as an input parameter to format_digraph finalize_digraph and still have the default in preferences. Note that the graphviz digraph is created outside finalize_digraph, so (although it is not ideal) the functions that create the nodes and edges will be using the values listed in preferences to decide whether to add tooltips (among other things like the color/style/numbering).

Yes, let's remove the format from graph_attrs, it seems like it is not needed since we set the final format anyway.

Thanks a bunch!

[BenPortner]

@yoelcortes: It would be nice to include essential initialization parameters in the tooltips (e.g. efficiency of a turbine, number of stages for a multicompressor). Do you know a clever way to include these while preventing duplication of the result() data?

[yoelcortes]

@BenPortner, yes! I meant to mention this to you earlier but forgot. Thanks for asking! We can extend the results() dataframe using the _get_design_info() -> Sequence[tuple[name, value, units]] method. This method will add rows in the results() dataframe as if they were additional items in the design_results dictionary. For example:

def _get_design_info(self):
      return [
      ('Residence time', self.tau, 'hr'),
      ('Conversion efficiency', self.efficiency, ''),
      ('Working volume fraction', 0.8, '')
] 

Hope this helps!

[BenPortner]

Hi @yoelcortes,

Thanks for your help with this! I'm a little bit swamped right now, so I cannot pour as much time into it as I'd like.

Just a quick question: Did you tick the "make sure it works for save_digraph and display_digraph" box? Last time I tried, the HTML output did not work in notebooks because of this bug, which I am waiting to be resolved.

Concerning your edit "add tooltips only if output format is html (Yoel edit: or svg?)": Although SVG offers a simple tooltip functionality, displaying large amounts of data like the ones returned by Unit.results() requires advanced formatting (e.g. HTML tables) to keep the output readable. Unfortunately, the default SVG tooltips do not support such advanced formatting so I would not recommend attempting this for now.

Ben

[yoelcortes]

@BenPortner, thanks for the notes on this! The html output doesn't show tooltips in jupyter notebooks (yet) unfortunately, but it is still working for me.

Your right about the svg formating, I just tried it jaja. I'll make tooltips only available for html format.

I think I'll set the default format back to svg since html is still not working on jupyter notebooks

Thanks!

[yoelcortes]

I'm super stoked about these new features. Thanks, @BenPortner! We still got a couple of items left to add, but I'll go ahead and merge to not have branches diverge too much.

[yoelcortes]

I was able to get html diagrams to show on jupyter notebooks, but it requires to save the file:

biosteam/biosteam/digraph/digraph.py

Line 525 in 8f66a0b

temp_file = os.path.join('.', name + '.html')

I wasn't able to get the file to save in a path other than directly in the same folder (everything else I tried failed...). It would me nice if they could be saved as part of jupyter notebook ipynb checkpoints... or something of the sort.

EDIT: Nevermind, I think I found the solution:

jupyter/nbconvert#1166 (comment)

import urllib
data_uri = 'data:text/html;charset=utf-8,' + urllib.parse.quote(html_str)
display(IFrame(data_uri, width='100%', height='250'))

[BenPortner]

Hi @yoelcortes,

Just to clarify: You don't see the HTML diagrams in the notebook if you don't save it? That is unusual, because last time I tried, it worked for me. The only thing that didn't work were the tooltips.

Which error do you get when saving in another folder? (I guess this is solved)

[yoelcortes]

@BenPortner, Yes! This is solved already. I had a ton of fun getting it to work and updating the documentation over the weekend. Here are a couple of notes:

1.The default does not insert javascript and outputs diagrams as svg with tooltips (which has no issues on jupyter notebooks). For tooltips with full results, set bst.prefrences.tooltips_full_results = True and bst.prefrences.graphviz_format = 'html' (https://biosteam.readthedocs.io/en/latest/tutorial/Preferences.html#Diagram-display-preferences).
2. There is still an issue with the html tooltips with javascript in jupyter notebooks: the tables are long and detailed (which is good) but we cannot scroll down to view them. Would it be possible keep interacting with the tables (such as copying data from them) while hovering on it? This would also allow users to scroll down while hovering on the table (not just the label). I believe there is an option for "iteractive buttons" in tippytools but I could not figure it out.
3. There still some aspects of the code that could be made better, like the ability to pass more options to diagram functions instead of depending only on biosteam.preferences.

Thanks!