Allow LaTeX in text3d

[slel]

From #29758-comment:13.

CC: @jcamp0x2a @paulmasson @slel @egourgoulhon

Component: graphics

Issue created by migration from https://trac.sagemath.org/ticket/30226

[jcamp0x2a]

comment:1

Attachment: mathjax2canvas_threejs.html.gz

I've attached a proof of concept of rendering MathJax output inside a Three.js scene. It would correspond to Sage code like:

t = text3d(r"x = {-b \pm \sqrt{b^2-4ac} \over 2a}", (0,0,0))
show(t, axis_labels=[r"\alpha", r"\beta", r"\gamma"])

It's got some clipping issues in Chrome that don't appear in FireFox, and it takes awhile to generate the images. I'm not sure how much I could speed that up, but it does seem like it would be feasible to support LaTeX in the viewer.

This example renders every piece of text as LaTeX. I imagine we'd want to add an additional parameter to text3d to control that as well as an extra viewer option to control axis_labels? Or just try to auto-detect the presence of LaTeX in the strings?

[slel]

comment:2

Replying to @jcamp0x2a:

This example renders every piece of text as LaTeX.
I imagine we'd want to add an additional parameter
to text3d to control that as well as an extra viewer option
to control axis_labels? Or just try to auto-detect
the presence of LaTeX in the strings?

Maybe mimic how sage.plot.text.text detects LaTeX?
How does it work btw?

[paulmasson]

comment:3

Replying to @slel:

Replying to @jcamp0x2a:

This example renders every piece of text as LaTeX.
I imagine we'd want to add an additional parameter
to text3d to control that as well as an extra viewer option
to control axis_labels? Or just try to auto-detect
the presence of LaTeX in the strings?

Maybe mimic how sage.plot.text.text detects LaTeX?
How does it work btw?

The is no detection of LaTeX in Sage proper that I can see. That all occurs in matplotlib.

[paulmasson]

comment:4

Replying to @jcamp0x2a:

I've attached a proof of concept of rendering MathJax output inside a Three.js scene. It would correspond to Sage code like:

t = text3d(r"x = {-b \pm \sqrt{b^2-4ac} \over 2a}", (0,0,0))
show(t, axis_labels=[r"\alpha", r"\beta", r"\gamma"])

It's got some clipping issues in Chrome that don't appear in FireFox, and it takes awhile to generate the images. I'm not sure how much I could speed that up, but it does seem like it would be feasible to support LaTeX in the viewer.

The slow rendering probably comes from explicitly calling the MathJax queue. For one of my VR projects I figured out how to render an entire div inside an SVG that was then used as a texture. The code for that is here, and if you have VR equipment you can see it in action in any of these special function visualizations. I was thinking of replicating that here and including the MathJax tag in the SVG: might be faster.

Another thing to think about is how to make this work offline. I've had plenty of negative feedback about defaulting to the CDN: quite a number of developers consider that an invasion of privacy. That's why Three.js was made a standard package to begin with. Are we planning to add MathJax as a standard package or just declare that this feature only works in a notebook?

This example renders every piece of text as LaTeX. I imagine we'd want to add an additional parameter to text3d to control that as well as an extra viewer option to control axis_labels? Or just try to auto-detect the presence of LaTeX in the strings?

We probably should do some auto-detection on text3d. We could require that users include the
MathJax tags $$ in the string and then search for them to decide how to process the text in the renderer.

I'm not in favor of applying these changes to axes labels, since one can already include characters as Unicode either by copying from a character map or using \u formatting, e.g. alpha = \u03b1.

P.S. you can avoid the weird appearance of names with internal capital letters by prefacing them with an exclamation point.

[paulmasson]

comment:5

Getting math to display in 2D plots requires dollar signs around the LaTeX. Should do the same here for consistency. That way could assume every string starting with dollar sign should be rendered with MathJax.

[paulmasson]

comment:6

MathJax is already a standard package...

[jcamp0x2a]

comment:7

Replying to @paulmasson:

The slow rendering probably comes from explicitly calling the MathJax queue. For one of my VR projects I figured out how to render an entire div inside an SVG that was then used as a texture. The code for that is here, and if you have VR equipment you can see it in action in any of these special function visualizations. I was thinking of replicating that here and including the MathJax tag in the SVG: might be faster.

Cool stuff! Yea, I'm pretty sure my approach is not the ideal and would love an alternative. (Off-topic, but any chance of VR support in the viewer at some point?)

Another thing to think about is how to make this work offline. I've had plenty of negative feedback about defaulting to the CDN: quite a number of developers consider that an invasion of privacy. That's why Three.js was made a standard package to begin with. Are we planning to add MathJax as a standard package or just declare that this feature only works in a notebook?

As you mentioned in a follow-up comment, MathJax seems to already be a standard package. In the proof of concept, I used the same version from the CDN that's currently in Sage, 2.7.4 I think.

I'm not in favor of applying these changes to axes labels, since one can already include characters as Unicode either by copying from a character map or using \u formatting, e.g. alpha = \u03b1.

True, but it may seem awkward/inconsistent from the user's perspective. I think a common use case would be displaying an equation adjacent to its plot so one may end up having to represent the same character in two different ways. As a user with no knowledge of the internals, I'd be perplexed as to why.

P.S. you can avoid the weird appearance of names with internal capital letters by prefacing them with an exclamation point.

Thanks!

[paulmasson]

comment:8

Replying to @jcamp0x2a:

Replying to @paulmasson:

The slow rendering probably comes from explicitly calling the MathJax queue. For one of my VR projects I figured out how to render an entire div inside an SVG that was then used as a texture. The code for that is here, and if you have VR equipment you can see it in action in any of these special function visualizations. I was thinking of replicating that here and including the MathJax tag in the SVG: might be faster.

Cool stuff! Yea, I'm pretty sure my approach is not the ideal and would love an alternative. (Off-topic, but any chance of VR support in the viewer at some point?)

The changes needed to get WebXR working are rather simple. The real problem is that the files require a secure server, i.e. https, in order to enter VR. Even local files needed to be served through Chrome in order to be viewed. Making something like that work for a diverse crowd would be a challenge...

[mkoeppe]

comment:11

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.