:UTF-8 stream decoding error on maxplot.svg

[ymei]

When the maxplot.svg contains some special characters, UTF-8 stream decoding error is raised. Here is a minimal case to cause this issue:

set_plot_option([svg_file, "maxplot.svg"])$ /* Allow the generated figures to be shown in this jupyter notebook. */
set_plot_option([gnuplot_preamble,"set format y \"%h\""])$ /* %h, when 'x' (multiplication) is used, causes error. */
plot2d(exp(x),[x,0,20]);

And the error output:

:UTF-8 stream decoding error on
#<SB-SYS:FD-STREAM for "file /private/var/folders/.../maxplot.svg" {100290C563}>:

  the octet sequence #(215 49) cannot be decoded.

set format y "%h" will ask gnuplot to use e.g. 1x10^5 with the multiplication sign x being a special character in the svg file. set format y "%G" won't generate such special character hence there's no error.

I'm not sure if this bug should be considered on the gnuplot side or on the jupyter side. I tend to think that jupyter should be able to include/display any reasonable svg.

[yitzchak]

I haven't been able to replicate this yet, but if it is an error in maxima-jupyter then the problem probably lies in common-lisp-jupyter since that is where the file reading code is located.

Just for reference, what OS are you using and what version of maxima and gnupliot do you have?

[ymei]

I am running maxima and gnuplot on both macOS and Linux. This issue is shown on both platforms. common-lisp-jupyter comes from quicklisp.

G N U P L O T Version 5.2 patchlevel 7
Maxima 5.43.0 using Lisp SBCL 1.5.3

[yitzchak]

I was able to reproduce this by using ISO 8859-1 encoding.

set_plot_option([gnuplot_preamble,"set format y \"%h\"
set encoding iso_8859_1"])$

Can you try forcing UTF8 with the following?

set_plot_option([gnuplot_preamble,"set format y \"%h\"
set encoding utf8"])$

[ymei]

I confirm that set encoding utf8 solves the problem.

And I inspected the generated maxplot.svg a bit. The encoding= in the xml file maxplot.svg does correspond to what is specified in gnuplot correctly: <?xml version="1.0" encoding="iso-8859-1" standalone="no"?>. I would tend to think that lisp should be able to adapt.

[yitzchak]

Right now we are just assuming that the file is in UTF8. I'll have to add code to common-lisp-jupyter to inspect the XML header.

[yitzchak]

After a little more research, I am not sure that switching the encoding is even possible in Jupyter Notebook. In other words, since SVG files are embedded as text (not as base 64) in the notebook then the SVG encoding has to be the same as the overall notebook file. There doesn't appear to be a way to switch the encoding of the notebook file via the user interface.

Basically, only set encoding utf8 will work. Some systems seem to assume utf8 which is why the preamble call isn't needed.