UnicodeEncodeError: 'charmap' codec can't encode character '\u2265'

[thorbjornwolf]

Hi Jake!

Thank you sooooooo much for your work on, well, everything. Here, in particular on altair ❤️

I was thrilled to discover altair_saver, as I just happened to need a way of saving plots to HTML with the javascript dependencies inline: altair_saver.save(chart, "boom.html", inline=True)!

Problem

At first try, it fails on my machine:

>>> import altair
>>> import altair_saver
>>> import vega_datasets
>>> 
>>> chart = altair.Chart(vega_datasets.data.cars.url).mark_point().encode(
>>>     x='Horsepower:Q',
>>>     y='Miles_per_Gallon:Q',
>>>     color='Origin:N'
>>> )
>>> altair_saver.save(chart, "cars.html", inline=True)

---------------------------------------------------------------------------
UnicodeEncodeError                        Traceback (most recent call last)
<ipython-input-2-526fd86c54d9> in <module>
----> 1 altair_saver.save(chart, "cars.html", inline=True)

~\AppData\Local\Continuum\miniconda3\envs\plotter\lib\site-packages\altair_saver\_core.py in save(chart, fp, fmt, mode, method, **kwargs)
     75     saver = Saver(spec, mode=mode, **kwargs)
     76 
---> 77     saver.save(fp=fp, fmt=fmt)
     78 
     79 

~\AppData\Local\Continuum\miniconda3\envs\plotter\lib\site-packages\altair_saver\savers\_saver.py in save(self, fp, fmt)
     92         elif isinstance(content, str):
     93             with maybe_open(fp, "w") as f:
---> 94                 f.write(content)
     95         elif isinstance(content, bytes):
     96             with maybe_open(fp, "wb") as f:

~\AppData\Local\Continuum\miniconda3\envs\plotter\lib\encodings\cp1252.py in encode(self, input, final)
     17 class IncrementalEncoder(codecs.IncrementalEncoder):
     18     def encode(self, input, final=False):
---> 19         return codecs.charmap_encode(input,self.errors,encoding_table)[0]
     20 
     21 class IncrementalDecoder(codecs.IncrementalDecoder):

UnicodeEncodeError: 'charmap' codec can't encode character '\u2265' in position 246219: character maps to <undefined>

• '\u2265' is the innocent-looking GREATER-THAN OR EQUAL TO sign, ≥
  • It seems to come from the source for vega.js:

  (Pdb)  input.find("// vega.js v")
  74
  (Pdb)  input.find("// vega-lite.js v")
  481314
  

• The encoding happens in a module named "cp1252.py", which seems to indicate the Windows 1252 encoding
  • How my machine ends up thinking 1252 is the way to go is beyond me: I checked sys.getdefaultencoding() and got utf-8.

Context

• altair 4.0.1
• altair_saver 0.1.0
• vega_datasets 0.8.0

conda-installed python 3.7.6
Windows 10

Monkeypatching solution

I found that I could "solve" the problem (i.e., successfully save a usable html) by monkeypatching altair_saver.savers._saver.maybe_open to explicitly use utf-8, like so:

import contextlib
from typing import Union, Iterator, IO

@contextlib.contextmanager
def maybe_open(fp: Union[IO, str], mode: str = "w") -> Iterator[IO]:
    """Write to string or file-like object"""
    if isinstance(fp, str):
        with open(fp, mode, encoding="UTF-8") as f:    # --------- I added the encoding kwarg
            yield f
    elif isinstance(fp, io.TextIOBase) and "b" in mode:
        raise ValueError("File expected to be opened in binary mode.")
    elif isinstance(fp, io.BufferedIOBase) and "b" not in mode:
        raise ValueError("File expected to be opened in text mode")
    else:
        yield fp
        
from altair_saver.savers import _saver
_saver.maybe_open = maybe_open

I know far too little about HTML encodings to tell if this is a generally good idea, so... draw your own conclusions 😉

[joelostblom]

Since Altair 5.2, the functionality of Altair Saver is now available in Altair via the vl-convert package. Most of the functionality has been available since 5.0, and the main addition in 5.2 was PDF export. See the docs on how to save charts for more details.

We are going to archive this repo, so I'm closing all the open issues and PRs before doing so. Try out the new options for saving charts mentioned above and if you run into issues, please open an issue directly in the altair or vl-convert repo.