-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
LaTeX: background gray in PDF code-blocks has a lesser contrast for xelatex
with non-highlighted tokens
#11269
Comments
Instead of trying to figure out which background gray to choose, I suggest changing the font in a tt-like environments and fallback to \newfontfamily\sphinxttfamily{txtt}
\renewcommand{\ttfamily}{\sphinxttfamily}
% if fontspec implements \setfontfamily
% \setfontfamily\ttfamily{txtt} Alternatively, we could change the font used by the |
Fair enough (I though that
I don't think we can do it but it's worth researching. I think |
It is the LaTeX default by LaTeX upstream maintainers for historical reasons (Latin Modern was developed by Polish TeX user groups, and was widely promoted at time of creation, but Computer Modern Unicode is much much closer to the original Knuth Computer Modern) but Latin Modern as its name indicates supports only Latin scripts. In contrast the GNU FreeFont supports many more scripts (in particular Greek and Cyrillic). and the crucial fact here is that LaTeX has no built-in support for automatic rescue when a glyph is not supported in a font, and they will not change that because they prefer a PDF with rectangles for missing glyphs than glyphs picked up from fall-back fonts; all fall-back has to be configured manually by user if needed, and 99% of LaTeX users do not know how to do that because the "New" LaTeX font system from perhaps 30 years ago is basically documented only at package author level and we as Sphinx LaTeX maintainers can little about this; besides missing glyphs if they happen will typically be overlooked by Sphinx latex users because except if one does
Yes, I didn't and don't expect now to find a fontspec way, I think it may be possible in lualatex which can manipulate font (I recall seeing some FakeBold somewhere). An extreme option would be to use FreeMono Bold, I did not yet try it out, but this is probably a no-go, as it will make Pygments highlighting partially hampered. A better way goes like you said: adopt some other font. Match with FreeSerif and FreeSans is not fundamental (after all Sphinx user times+helvetica+courier for long time, until this was modernized and courier replaced by txtt) but the font must have wide Unicode coverage and be available widely. (and come with TeXLive) |
I don't know if this is a suitable choice, but what about Noto? Or more generally, what about choosing specific fonts when a document contains special glyphs not supported by the default font ? Like, we could (on the builder's side) check that the characters being written are within some correct unicode range, and if they are not, we would switch to a font family that supports all characters in the output. The user could also specify the "default" fallback font to check in priority (for instance, if the whole document is in huge code-block in Chinese, it's probably better to directly use a font supporting this range instead of first trying all other fonts).
Yes, so I think we need to find an alternative.
For unsupported characters, we could actually do it on the builder side by checking unicode ranges of the text being written. The idea is to load the font file at the beginning of the process and when writing, we check the user content and emit warnings if needed. Since this might be quite costly for large documents, this could be enabled on demand, or only if we encounter non-UTF8 characters (again, we could enable it depending on the language of the document being produced or depending on other configuration values). |
An option could be NewComputerModern which is a TeXLive provided font. Here we only pick the monospace family: diff --git a/sphinx/builders/latex/constants.py b/sphinx/builders/latex/constants.py
index ce646d024..1f7252764 100644
--- a/sphinx/builders/latex/constants.py
+++ b/sphinx/builders/latex/constants.py
@@ -49,12 +49,12 @@ XELATEX_DEFAULT_FONTPKG = r'''
BoldFont = *Bold,
BoldItalicFont = *BoldOblique,
]
-\setmonofont{FreeMono}[
- Extension = .otf,
- UprightFont = *,
- ItalicFont = *Oblique,
- BoldFont = *Bold,
- BoldItalicFont = *BoldOblique,
+\setmonofont{NewCMMono10}[
+ Path,
+ UprightFont = *-Regular,
+ ItalicFont = *-Italic,
+ BoldFont = *-Bold,
+ BoldItalicFont = *-BoldOblique,
]
''' (in the above I use as it gives sphinx/sphinx/builders/latex/constants.py Line 145 in e2f66ce
to use fontsize=auto , not tested). edit I tested and it is definitely better with fontsize=auto .
but I got Latexmk: ====List of undefined refs and citations:
Missing character: There is no ⊞ (U+229E) in font [NewCMMono10-Regular]/OT:script=latn;language=dflt;!
Missing character: There is no ⊞ (U+229E) in font [NewCMMono10-Regular]/OT:script=latn;language=dflt;! for our document so I needed diff --git a/doc/conf.py b/doc/conf.py
index 19b8b2a2e..b9b81c2d7 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -58,16 +58,18 @@ epub_show_urls = 'inline'
epub_use_index = False
epub_description = 'Sphinx documentation generator system manual'
+latex_engine = 'xelatex'
latex_documents = [('index', 'sphinx.tex', 'Sphinx Documentation',
'the Sphinx developers', 'manual', 1)]
latex_logo = '_static/sphinx.png'
latex_elements = {
- 'fontenc': r'\usepackage[LGR,X2,T1]{fontenc}',
+# 'fontenc': r'\usepackage[LGR,X2,T1]{fontenc}',
'passoptionstopackages': r'''
\PassOptionsToPackage{svgnames}{xcolor}
''',
'preamble': r'''
-\DeclareUnicodeCharacter{229E}{\ensuremath{\boxplus}}
+%\DeclareUnicodeCharacter{229E}{\ensuremath{\boxplus}}
+\catcode`⊞\active\protected\def⊞{\ensuremath{\boxplus}}
\setcounter{tocdepth}{3}% depth of what main TOC shows (3=subsubsection)
\setcounter{secnumdepth}{1}% depth of section numbering
\setlength{\tymin}{2cm}% avoid too cramped table columns which is specific to our document but maybe indicative of less extensive Unicode coverage compared to FreeMono. Looking at Ubuntu packages I got this i.e. it is (at least for kinetic ok also in focal but not in bionic). It is not in bionic which is 18.04LTS which I think we support (not checked). Anyway it is only one of possible Unicode monospace font to adopt if we decide. But so far nobody had complained about the lightness of FreeMono (which is inherited from Courier from which it seems to take inspiration). |
Why not. If it is distributed by TeXLive I guess it qualifies as opensource compatible font. And it has very extensive coverage of course.
I was going to tag this issue "help wanted" and will definitely do so now ;-). I think this is ambitious goal
Help wanted! Lots of maintenance there!
Chinese users I think have their own ways using xeCJK package probably... Help wanted!
Unfortunately I think even if we embark upon this ambitious goal we will not have a smooth sailing. I will link to an issue that the translate-to-Japanese team of Python docs encountered:python/python-docs-ja#35 . They ended up using luatex-ja. (the above link is a bit confusing here, sorry, because their starting point was not xelatex, but platex or rather probably uplatex which is used by Sphinx for Japanese; sorry about that but leaving the link anyhow as it contains some aspects of complex font problems in LaTeX) |
@picnixz thanks for feedback and ideas 👍 I forgot to say I currently have been using for some time Source Code Pro in my Emacs and even in Firefox. (but it is wide) |
@picnixz I may be wrong but it appears NotoSansMono from TeXLive has no Italic or BoldItalic variant. The documentation from |
Oh, didn't think that it would be the case. It appears that it is a feature request but probably not under consideration (notofonts/latin-greek-cyrillic#203)... |
For people willing to help: we need a monospace Unicode font supporting at least Latin Greek and Cyrillic (CJK goes via separate pathways a priori but it would be a plus to have simplified Chinese) as well as for example Unicode code points involved in output of Linux tree command, ideally does not require texlive-fonts-extra for Ubuntu and if it does was already included there with 18.04LTS, is also available via TeXLive, has been tested with fontspec |
I asked one of my friend and he recommended me The issue is that it is not provided by TeXlive. One possibility is to ask for the NF maintainers to do this, or alternatively, shipout the font in the Sphinx installation directly. Or make it an optional dependency when installing Sphinx via |
Thanks for info @picnixz. As a complement I report here on searching in my TeXLive tree for suitable fonts. For this I used the albatross tool (which must be at release 0.5.0 or newer as formerly the tacit "and" for searched characters acted as "or"). This is command line tool but unfortunately it is not completely suited for piping its output into grep for example which complicates a bit things. Details:
Here is result. But I don't think I have a complete TeXLive installation so I may be lacking some fonts. (there is no albatross option for a github markdown flavor table syntax... but as shown here it supports reST grid tables...) I also manually added after Font name the number of glyphs it has according to FontBook on macos (only the regular variant). And finally, as I had exceeded my time quote I went the extra mile and added screenshots of a test. And I added the magic incantation to be sure it works on all systems via font filenames.
Regarding the ⊞ which our own documentation uses twice there is a simple work-around commented above if we wanted to have it, assuming here we were to build our docs with xelatex.
Actually I encountered a problem with SourceCodePro when a long code line wraps, as happens in example above when not using See the red crosses above. The LaTeX log contains:
This is is the OPEN BOX glyph and I have to check why XeLaTeX wants to use it as visible space (some one, two, or three years ago, LaTeX changed something in this context). Because it is definitely not Sphinx which tells it to. Ah yes Sphinx has
so the
Really a pity that Source Code Pro does not have the glyph! (I have read somewhere that Adobe supplied fonts usable for free have often problems with things such as placement of apostrophe, but I am not sure). So now I have to check that all above other fonts have the glyph...
Hence the list appears to be reduced to (except if we do not use
|
|
Regarding
so it is only a matter of adding the info. General philosophy here is that people willing to use modernistic engines If we decide to use |
I personally use DejaVu because some years ago, installing some free font entirely broke my desktop environment (I honestly don't know what happened and I still don't know what I did. I just know that it worked again if I removed the package I installed). So I would be in favor of using DejaVu family. NewComputerModernMono10 looks nicer to me because DejaVu seems more compact, but since it's too recent we might wait for another 5 years until we use it as a default font ? Concerning DejaVu Sans and Serif, I think it's good to have a uniform font (and that's also what I personally use). Still need other opinions though. |
Describe the bug
Example with
'xelatex'
forlatex_engine
:Same with default
'pdflatex'
:The choice of default background gray was done with testing with pdflatex. If testing with xelatex, it could have been chosen possibly even lighter.
But this issue is mainly applying to the few non highlighted tokens.
It is a font issue, the one used for xelatex/lualatex is thinner than the one used for pdflatex. It may also be compounded with a PDF viewer anti-aliasing effect, for these thin glyphs on colored background.
How to Reproduce
Any project with code-blocks
Environment Information
Sphinx extensions
No response
Additional context
LaTeX installation from TeXLive 2023.
The text was updated successfully, but these errors were encountered: