Trac #25833: Upgrade to MathJax 3 and configure Sage

Mathjax v3.2.0 was released on 2021-06-17, see
[[https://github.com/mathjax/MathJax]].

This will be done in three steps:

- Step 1: Fix Sage to be compatible with MathJax3. Done in #31517.
- Step 2: Provide mathjax spkg upgraded to MathJax3.
- Step 3: Configure Sage to use MathJax3.

This ticket focuses on the Steps 2 and 3.

To install mathjax, you can download the tarball from a temporary URL:

  https://drive.google.com/file/d/1-tMjjIwftzoGngFh549_kUO3Chmy3qhI/view
?usp=sharing

and place the tarball in `upstream/` and `make`, or you can just do
`make` since the tarball will be automatically downloaded from the
attachment of this ticket after unsuccessful attempts to download from
sage mirrors.

We can now choose whether to use mathjax CDN or not. The default is not
to. To use CDN, do
{{{
export SAGE_DOCBUILD_OPTS="--use-cdns"
}}}
before `make` online doc.

Related tickets:

- #30296 System package information and spkg-configure for mathjax

- #33600 Configure mathjax in sphinx config and upgrade to Mathjax v3

URL: https://trac.sagemath.org/25833
Reported by: slelievre
Ticket author(s): Tobias Diez, Kwankyu Lee
Reviewer(s): Tobias Diez, Kwankyu Lee

build/pkgs/mathjax/SPKG.rst

@@ -6,7 +6,7 @@ Description
 
 MathJax is a JavaScript library for displaying mathematical formulas.
 
-MathJax is used by the Jupyter notebook and the Sphinx documentation.
+MathJax is used in the Sage documentation built by Sphinx.
 
 License
 -------
@@ -30,9 +30,8 @@ Special Update/Build Instructions
 
 None.
 
+
 Patches
 -------
 
--  nopng_config.patch: prevent font warning messages since png files are
-   removed. See section "Trimming II -- not strictly necessary" of
-   https://github.com/mathjax/MathJax-docs/wiki/Guide%3A-reducing-size-of-a-mathjax-installation
+None.

build/pkgs/mathjax/checksums.ini

@@ -1,4 +1,5 @@
 tarball=mathjax-VERSION.tar.gz
-sha1=a1b31066a541167c65d6b43f24f3a06f03760662
-md5=02d1067dc06d9ca7d380457a49404766
-cksum=1204309472
+sha1=3f7abecf8cacd7f5d7f9ae6c3baca7739101c17d
+md5=ba1a65ab58aaad6c84f39735c619bc34
+cksum=1142131398
+upstream_url=https://trac.sagemath.org/raw-attachment/ticket/25833/mathjax-3.2.0.tar.gz

build/pkgs/mathjax/package-version.txt

@@ -1 +1 @@
-2.7.4.p0
+3.2.0

build/pkgs/mathjax/spkg-src

@@ -4,57 +4,31 @@ set -e
 
 [ -n "${SAGE_ROOT}" ] || SAGE_ROOT="$(pwd)/../../../"
 
-
-# determine latest version.
-GIT_VERSION="$(curl http://docs.mathjax.org/en/latest/installation.html | grep 'Current Version' | sed 's|^.*archive/||g' | sed 's/.zip".*//g')"
+# determine the latest version
+GIT_VERSION="$(curl https://github.com/mathjax/MathJax/releases | grep  'MathJax v' | head -1 | sed 's|^.*MathJax v||g' | sed 's/\s*$//g')"
 echo "GIT_VERSION=$GIT_VERSION"
 
-# fetch and rename latest version.
-URL="https://github.com/mathjax/MathJax/archive/${GIT_VERSION}.zip"
+# fetch and rename the latest version
+URL="https://github.com/mathjax/MathJax/archive/refs/tags/${GIT_VERSION}.zip"
 echo "Downloading $URL"
 rm -rf src
 if [ -z "$UPSTREAM_SOURCE_TARBALL" ]; then
     tar xzf <( curl -L "$URL" )
 else
     tar xzf "$UPSTREAM_SOURCE_TARBALL"
 fi
-mv MathJax-${GIT_VERSION} src
-
-# Strip mathjax according to
-# https://github.com/mathjax/MathJax-docs/wiki/Guide%3A-reducing-size-of-a-mathjax-installation
-
-# Trimming I -- removing files unnecessary for deployment
-FILEDIRS_TO_REMOVE='docs/ test/ unpacked/ .gitignore README-branch.txt README.md bower.json'
-for filedir in ${FILEDIRS_TO_REMOVE} ; do
-    rm -rf "src/${filedir}"
-done
-
-# Trimming II -- not strictly necessary (requires the patch nopng_config.patch)
-rm -rf 'src/fonts/HTML-CSS/TeX/png/'
-
-# Trimming III -- fonts
-FONTS_TO_REMOVE='Asana-Math Gyre-Pagella Gyre-Termes Latin-Modern Neo-Euler'
-for font in ${FONTS_TO_REMOVE} ; do
-    find . -type d -name "${font}" -prune -exec rm -rf {} \;
-done
-
-FONT_FORMATS_TO_REMOVE='eot otf svg'
-for fontformat in ${FONT_FORMATS_TO_REMOVE} ; do
-    find . -type d -name "${fontformat}" -prune -exec rm -rf {} \;
-done
-
-# Trimming IV -- reducing input and output options
-OUTPUT_OPTIONS_TO_REMOVE='NativeMML SVG'
-for output in ${OUTPUT_OPTIONS_TO_REMOVE} ; do
-    rm -rf "src/jax/output/${output}"
-done
-
-# repack
-tar czf "$SAGE_ROOT/upstream/mathjax-${GIT_VERSION}.tar.gz" src
+
+# put files under mathjax directory
+mkdir src
+mv MathJax-${GIT_VERSION}/es5 src/mathjax
+rm -r MathJax-${GIT_VERSION}
+
+PACKAGE_VERSION=${GIT_VERSION}
+
+# repackage
+tar czf "$SAGE_ROOT/upstream/mathjax-${PACKAGE_VERSION}.tar.gz" src
 rm -rf src
 
 # update package info
-echo "${GIT_VERSION}" > 'package-version.txt'
+echo "${PACKAGE_VERSION}" > 'package-version.txt'
 "$SAGE_ROOT"/sage --package fix-checksum mathjax
-
-

src/doc/common/themes/sage-classic/theme.conf

@@ -40,5 +40,3 @@ linkcolor        = #45529B
 # Background color for code blocks: very pale yellow
 codebgcolor      = #FFFFE5
 
-# MathJax settings filled in by conf.py
-mathjax_macros =

src/doc/en/reference/footer.txt

@@ -1,13 +1,13 @@
 Indices and Tables
 ==================
 
+* `Index <../genindex.html>`_
+* `Module Index <../py-modindex.html>`_
+* `Search Page <../search.html>`_
+
 ..
   comment: the following math environment forces Sphinx to load MathJax
   in the index.rst pages. Do not delete it!
 
 .. math::
    :nowrap:
-
-* `Index <../genindex.html>`_
-* `Module Index <../py-modindex.html>`_
-* `Search Page <../search.html>`_

src/doc/en/reference/index.rst

@@ -7,7 +7,7 @@ Welcome to the Sage Reference Manual!
 Here you find documentation for all of `Sage <http://www.sagemath.org/>`_'s
 features, illustrated with lots of examples.  A thematic index follows.
 
-This documentation is licensed under the `Creative Commons Attribution-Share Alike 3.0 License`__:math:`.`
+This documentation is licensed under the `Creative Commons Attribution-Share Alike 3.0 License`__.
 
 __ http://creativecommons.org/licenses/by-sa/3.0/
 
@@ -175,3 +175,10 @@ Indices and Tables
 * :ref:`genindex`
 * :ref:`modindex`
 * :ref:`search`
+
+..
+  comment: the following math environment forces Sphinx to load MathJax
+  in the index.rst pages. Do not delete it!
+
+.. math::
+   :nowrap:

src/doc/en/website/templates/index.html

@@ -15,6 +15,13 @@
     display: none;
   {%- endif %}
   }
+  table.contentstable {
+    align: center;
+    border-spacing: 20px;
+  }
+  table.contentstable span {  {# trac #33600 comment:22 #}
+    border-spacing: initial;
+  }
 </style>
 {% endblock %}
 
@@ -33,7 +40,7 @@ <h2>
     Tutorials and FAQ
   </strong>
 </h2>
-<table class="contentstable" align="center" cellspacing="20">
+<table class="contentstable">
    <tr valign="top">
     <td width="50%">
       <p class="biglink">

src/sage/docs/conf.py

@@ -4,7 +4,8 @@
 import sys
 import os
 import sphinx
-from sage.env import SAGE_DOC_SRC, SAGE_DOC, SAGE_SRC, THEBE_DIR, PPLPY_DOCS, MATHJAX_DIR
+from sage.env import SAGE_DOC_SRC, SAGE_DOC, THEBE_DIR, PPLPY_DOCS, MATHJAX_DIR
+from sage.misc.latex_macros import sage_mathjax_macros
 import sage.version
 from sage.misc.sagedoc import extlinks
 import dateutil.parser
@@ -15,19 +16,23 @@
 import sphinx.ext.intersphinx as intersphinx
 from IPython.lib.lexers import IPythonConsoleLexer, IPyLexer
 
+
 # General configuration
 # ---------------------
 
 # Add any Sphinx extension module names here, as strings. They can be extensions
 # coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
-extensions = ['sage_docbuild.ext.inventory_builder',
-              'sage_docbuild.ext.multidocs',
-              'sage_docbuild.ext.sage_autodoc',
-              'sphinx.ext.todo',
-              'sphinx.ext.extlinks',
-              'IPython.sphinxext.ipython_directive',
-              'matplotlib.sphinxext.plot_directive',
-              'jupyter_sphinx']
+extensions = [
+    'sage_docbuild.ext.inventory_builder',
+    'sage_docbuild.ext.multidocs',
+    'sage_docbuild.ext.sage_autodoc',
+    'sphinx.ext.todo',
+    'sphinx.ext.extlinks',
+    'sphinx.ext.mathjax',
+    'IPython.sphinxext.ipython_directive',
+    'matplotlib.sphinxext.plot_directive',
+    'jupyter_sphinx',
+]
 
 jupyter_execute_default_kernel = 'sagemath'
 
@@ -263,21 +268,36 @@ def set_intersphinx_mappings(app, config):
 html_common_static_path = [os.path.join(SAGE_DOC_SRC, 'common', 'static'),
                            THEBE_DIR, 'static']
 
-# We use MathJax to build the documentation.
-extensions.append('sphinx.ext.mathjax')
-mathjax_path = 'MathJax.js?config=TeX-AMS_HTML-full,../mathjax_sage.js'
-
-from sage.misc.latex_macros import sage_mathjax_macros
-html_theme_options['mathjax_macros'] = sage_mathjax_macros()
+# Configure MathJax
+# https://docs.mathjax.org/en/latest/options/input/tex.html
+mathjax3_config = {
+    "tex": {
+        # Add custom sage macros
+        # http://docs.mathjax.org/en/latest/input/tex/macros.html
+        "macros": sage_mathjax_macros(),
+        # Add $...$ as possible inline math
+        # https://docs.mathjax.org/en/latest/input/tex/delimiters.html#tex-and-latex-math-delimiters
+        "inlineMath": [["$", "$"], ["\\(", "\\)"]],
+        # Increase the limit the size of the string to be processed
+        # https://docs.mathjax.org/en/latest/options/input/tex.html#option-descriptions
+        "maxBuffer": 50 * 1024,
+        # Use colorv2 extension instead of built-in color extension
+        # https://docs.mathjax.org/en/latest/input/tex/extensions/autoload.html#tex-autoload-options
+        # https://docs.mathjax.org/en/latest/input/tex/extensions/colorv2.html#tex-colorv2
+        "autoload": {"color": [], "colorv2": ["color"]},
+    },
+}
 
-mathjax_relative = os.path.basename(MATHJAX_DIR)
+if os.environ.get('SAGE_USE_CDNS', 'no') == 'yes':
+    mathjax_path = "https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-chtml.js"
+else:
+    mathjax_path = 'mathjax/tex-chtml.js'
+    html_common_static_path += [MATHJAX_DIR]
 
-# It would be really nice if sphinx would copy the entire mathjax
-# directory, (so we could have a _static/mathjax directory), rather than
-# the contents of the directory
-html_common_static_path.append(MATHJAX_DIR)
-exclude_patterns += ['**/' + os.path.join(mathjax_relative, i)
-                     for i in ('docs', 'README*', 'test', 'unpacked', 'LICENSE')]
+# A list of glob-style patterns that should be excluded when looking for source
+# files. They are matched against the source file names relative to the
+# source directory, using slashes as directory separators on all platforms.
+exclude_patterns = []
 
 # If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
 # using the given strftime format.

src/sage/misc/latex_macros.py

@@ -43,14 +43,16 @@
 contain '\newcommand' lines for each of the entries in ``macros``.
 """
 
+
 def produce_latex_macro(name, *sample_args):
     r"""
     Produce a string defining a LaTeX macro.
 
     INPUT:
 
-    -  ``name`` - name of macro to be defined, also name of corresponding Sage object
-    -  ``sample_args`` - (optional) sample arguments for this Sage object
+    -  ``name`` -- name of macro to be defined, also name of corresponding Sage object
+
+    -  ``sample_args`` -- (optional) sample arguments for this Sage object
 
     EXAMPLES::
 
@@ -108,37 +110,38 @@ def convert_latex_macro_to_mathjax(macro):
 
     INPUT:
 
-    -  ``macro`` - LaTeX macro definition
+    -  ``macro`` -- LaTeX macro definition
 
     See the web page
-    http://www.mathjax.org/docs/1.1/options/TeX.html for a
+    https://docs.mathjax.org/en/latest/input/tex/macros.html for a
     description of the format for MathJax macros.
 
     EXAMPLES::
 
         sage: from sage.misc.latex_macros import convert_latex_macro_to_mathjax
         sage: convert_latex_macro_to_mathjax('\\newcommand{\\ZZ}{\\Bold{Z}}')
-        'ZZ: "\\\\Bold{Z}"'
+        ('ZZ', '\\Bold{Z}')
         sage: convert_latex_macro_to_mathjax('\\newcommand{\\GF}[1]{\\Bold{F}_{#1}}')
-        'GF: ["\\\\Bold{F}_{#1}",1]'
+        ('GF', ['\\Bold{F}_{#1}', 1])
     """
     left_bracket = macro.find('[')
     right_bracket = macro.find('[')
     if left_bracket >= 0:
         right_bracket = macro.find(']')
-        num_args = macro[left_bracket+1:right_bracket]
+        num_args = int(macro[left_bracket + 1 : right_bracket])
     else:
         num_args = 0
     start_name = macro.find('{') + 1  # add one to go past the backslash
     end_name = macro.find('}')
-    name = macro[start_name+1:end_name]
+    name = macro[start_name + 1 : end_name]
     start_defn = macro.find('{', end_name)
     end_defn = macro.rfind('}')
-    defn = macro[start_defn+1: end_defn].replace('\\', '\\\\')
+    defn = macro[start_defn + 1 : end_defn]
     if num_args == 0:
-        return name + ': "' + defn + '"'
+        return name, defn
     else:
-        return name + ': ["' + defn + '",' + str(num_args) + ']'
+        return name, [defn, num_args]
+
 
 # To add a new macro for use in the Sage documentation, add a list or
 # tuple to the following list.  Each list (or tuple) should have the
@@ -176,6 +179,7 @@ def convert_latex_macro_to_mathjax(macro):
 # mathbf vs mathbb.  See latex.py for more information.
 sage_configurable_latex_macros = [r"\newcommand{\Bold}[1]{\mathbf{#1}}"]
 
+
 def sage_latex_macros():
     r"""
     Return list of LaTeX macros for Sage. This just runs the function
@@ -193,15 +197,15 @@ def sage_latex_macros():
 
 def sage_mathjax_macros():
     r"""
-    Return list of MathJax macro definitions for Sage as
-    JavaScript. This feeds each item output by
-    :func:`sage_latex_macros` to
+    Return Sage's macro definitions for usage with MathJax.
+
+    This feeds each item output by :func:`sage_latex_macros` to
     :func:`convert_latex_macro_to_mathjax`.
 
     EXAMPLES::
 
         sage: from sage.misc.latex_macros import sage_mathjax_macros
         sage: sage_mathjax_macros()
-        ['ZZ: "\\\\Bold{Z}"', 'NN: "\\\\Bold{N}"', ...
+        {'Bold': ['\\mathbf{#1}', 1], 'CC': '\\Bold{C}', ...
     """
-    return [convert_latex_macro_to_mathjax(m) for m in sage_latex_macros()]
+    return dict(convert_latex_macro_to_mathjax(m) for m in sage_latex_macros())

src/sage/misc/sagedoc.py

@@ -1486,7 +1486,6 @@ def __call__(self, obj, output='html', view=True):
     </script>
     <script type="text/javascript" src="%(static_path)s/jquery.js"></script>
     <script type="text/javascript" src="%(static_path)s/doctools.js"></script>
-    <script type="text/javascript" src="%(static_path)s/mathjax_sage.js"></script>
     <link rel="shortcut icon" href="%(static_path)s/favicon.ico" />
     <link rel="icon" href="%(static_path)s/sageicon.png" type="image/x-icon" />
   </head>

src/sage_docbuild/__init__.py

@@ -1 +0,0 @@
-