v0.15.7

Summary of Improvements

Please welcome CSL support. This is still a bit experimental, but this begins the deprecation of the home grown bibliography styling in our bibtex package. SILE can now leverage industry standard Citation Style Language specifications, which opens up a world of existing locale and style guide support. This is thanks to long-running work from from @Omikhleia on overhauling bibliography support. His work on the math package also continues with many operator and layout fixes and a few new features continuing to round out out MathML compatibility.

For folks calling out to Lua functions or system tools in their classes, packages, or documents, those calls will now track the document locale. If you use an OS function or external tool that supports localization, you should now get results that match the locale in the document at the time it was called. Responsiveness to locale modifiers might be a little rough until we normalize our locale format to be BCP-47 compliant, but it already works for basic locales. Incidentally this also provides a utility API that can be used to set any ENV variable you want before running external tooling.

Of interest only to SILE developers, some new configure flags have been added for convenience. Any individual tool used in the development and release process can be skipped or overridden using standard --with[out]-PACKAGE=PATH flags. Additionally skipping all the tooling checks can be done with --without-developer-tools even when --with-developer-mode is used.

Finally for distribution packagers, cross compilation of the Rust binaries should be a bit easier now.

New Features

• build: Add configure flag to skip checks for all developer tools (c01c867)
• build: Enable --with[out]-EXECUTABLE=PATH configure flags for tooling dependencies (89b5836)
• core: Set Lua's internal locale so builtin functions respond to document language (a614169)
• core: Set system locale for subprocesses to match the document language (b28cafd)
• math: Add pre-defined TeX-like operator functions (cos, sin, etc.) (8d83821)
• math: Support TeX-like apostrophe and multiplication sign as primes and asterisk (b8f35ff)
• math: Support TeX-like left..right delimiter syntax (960dc3f)
• math: Support the MathML operator dictionary and many TeX-like aliases (3dd25e9)
• packages: Add lightweight CSL engine (8d3961c)
• packages: Keep track of cited bibliography entries (57b3b7c)
• packages: Use experimental CSL renderer for BibTeX (808c6bb)
• utilities: Add function to set environment variables (0f0ed02)

Bug Fixes

• build: Support cross-compilation of Rust binaries (#2178) (19c7c1d)
• math: A period must be allowed in TeX-like math syntax for numbers (56edc14)
• math: Add math.font.script.feature setting, defaulting to ssty (2adc912)
• math: Improve spacing rules on limit-like operators (781f62a)
• math: Spacing rules must distinguish binary and unary operators (81a1be5)
• math: Suppress invisible operators in MathML (#2177) (72faad5)
• math: The (escaped) percent is an ordinary atom in TeX-like syntax (4170719)
• packages: Fix bogus command in Pandoc module's definition lists leaking bold (8c9348b)

v0.15.6

Summary of Improvements

Things got away from me. This release has more features and fixes than we usually let pile up in a patch release. I’ve spent a good deal of time over the last month in the hospital with my son and PR notifications just kept pouring in!

Most of the substantive features and fixes again come from @Omikhleia. This release includes a whole bunch of his work on math feature support fleshing out the MathML compatibility and correcting various math layout issues. It also his work updating hyphenation support for a number of languages and adding support for many more, mostly adapted from TeX hyphenation libraries. He also normalized some inconsistencies in the AST generated by the SIL and XML readers and fixed line layout in the verbatim package. His work towards CSL support is still pending my bandwidth to review in the bibliography project.

Additionally this release should be a bit more flexible in working with 3rd party modules. Notably we've accommodated some Lua limitations on macOS and Homebrew installations should be more versatile now. For Docker workflows, the SILE-provided LuaRocks should be visible to the host system as well.

Behind the scenes some minor features provided by vendored Lua libraries have been ported to Rust instead. This shouldn't even be noticeable to end users at this point, but serves as a proof of concept for implementing other internals in Rust while still bridging seamlessly to Lua. Many issues with building static binaries with embedded resources were resolved and build conflicts with ICU 76.1 resolved. Passing the class options via a CLI flag is again parsed correctly. Finally a few error messages and warnings have been cleaned up to be more readable and perspicuous.

New Features

• build: Add --with-vendored-luarocks=dir for use in distros with pseudo-system path handling (e93f072)
• cli: Catch missing required arguments at parse time (84f0d78)
• cli: Mimic Lua VM's ;; to extend not replace the default package paths (bb47f21)
• core: Add px unit as 0.75pt as per CSS3 (59f4bfd)
• core: Add separate crate to enable exporting interfaces to Lua (3a95a48)
• core: Extend SU.collatedSort for complex table sorting (3854be8)
• docker: Splice SILE's vendored rocks tree into system LuaRocks (548f426)
• languages: Add Albanian (sq) basic support (a1eedfc)
• languages: Add Belarusian (be) basic support (084f81d)
• languages: Add bibliography localizations for Swedish (#2125) (17371f3)
• languages: Add bibliography localizations for Turkish (84f15e9)
• languages: Add Church Slavonic (cu) basic support (01bead5)
• languages: Add Coptic (cop) basic support (e25b806)
• languages: Add Friulan (fur) basic support (4b88d91)
• languages: Add Galician (gl) basic support (b5116ed)
• languages: Add Interlingua (ia) basic support (6f920e6)
• languages: Add Kurmanji (kmr) basic support (c700786)
• languages: Add Macedonian (mk) basic support (c4ef9e9)
• languages: Add Occitan (oc) basic support (4610122)
• languages: Add Pāli (pi) basic support (13c2531)
• languages: Add Telugu (te) basic support (237d06d)
• languages: Add Upper Sorbian (hsb) basic support (51ac4f8), closes #1994
• math: Add mathcal, mathbb and similar commands in TeX-like math (c867d60)
• math: Add more TeX-like math commands behaving as big operators (ac9337e)
• math: Support a TeX math like text command (518e497)
• math: Support MathML bevelled fractions (567ac54)
• math: Support MathML maction (basic) and mstyle (partial) (0df93b1)
• math: Support MathML movablelimits attribute (92c678a)
• math: Support MathML mpadded and refactor TeX-like phantoms (d579096)
• math: Support MathML mphantom and TeX-like phantom, hphantom, vphantom (46e5954)
• math: Support MathML mroot element (88ebff7)
• math: Support MathML mtext and ms elements (3023b93)
• math: Support mathvariant on special extra greek characters (6f379da)
• math: Support square roots (7efd24d)
• math: Support TeX-like math limits (eb402c5)
• rusile: Enable embedding of Rusile module (693d7d4)

Bug Fixes

• build: Correct regression in v0.15.5 in generating embedded files list (dae2f7f)
• build: Smooth out build on ICU 76.1 per Homebrew report (8390534), closes #2152
• cli: Correct return value handling when process() doesn't lift weight (fb6b866)
• cli: Fix handling of --option flag in Rust CLI (a77170e)
• core: Correct module load paths when building in developer mode (4fbb6c0)
• docker: Setup system locales in containers where it actually counts, not before (55a4ba3)
• inputters: SIL commands and environments must generate the same syntax tree (4854992)
• languages: Add missing hyphenation patterns in polytonic Greek (42d14dd)
• math: Add missing MathML mathvariant mappings (803d699)
• math: Avoid page breaks before display math equations (19fd61f), closes #2160
• math: Correct math greek symbols and their var-alternate (6346a2d)
• math: Correct sub/superscript position for subformulas vs. symbols (6827eb9), closes #2122
• math: Default table row/column spacing must depend...

v0.15.5

Summary of Improvements

This is a relatively minor maintenance release. Thanks again to ongoing work by @Omikhleia, support for bibtex format bibliographies is improving. Date and author parsing and rendering has been improved. Support for CSL is still brewing in the bibliography project.

Additionally the --quiet flag now correctly suppresses all parts of warning messages for cleaner output when you don't want to debug your project. Even more behind the scenes, the entire code base has also been spell-checked, with hundreds of words in comments, code, and documentation being corrected.

New Features

• packages: Support biblatex date field and improve date formatting (5169d67)

Bug Fixes

• build: Fix distribution of unit tests clean up dist file list (00abf9b)
• classes: Make document state available during class's post init (71336b9)
• cli: Suppress trace info in quiet for warnings but do show errors (e12dcc9)
• packages: Parse and split all bibtex name fields (e3a7dc1), closes #2052
• packages: Use non-breakable space in bibTeX name splitting, not tilde (f4ea4ed)

v0.15.4

Summary of Improvements

The dust finally settled on the big v0.15.0 release. This patch does include a few more build system touch ups affecting the source tarball that made in require automake when it didn't really need it. But the impetus for this release is actually new features and typesetter bug fixes.

On the language front, @jodros has jumped in with some more domain expertise and Portuguese should now fully support hyphenation of Brazilian locale specific patterns. Additionally we fixed two bugs in the localization system. The locale set for Fluent localizations now properly tracks the document language at any given moment whether set by a font or even a Latin interjection in a bibliography.

Speaking of bibliographies, @Omikhleia is in the middle of a deep overhaul of our bibliography handling. This release has a number of features and bug fixes that lay some groundwork for better parsing and handling bibtex files. More fixes and features are also in progress and even ready for testing. If you have any interest in bibliographies feel free to review ongoing issues and PRs in the bibliography cleanup project.

New Features

• build: Add a developer target to test build dist without automake (7bdaf16)
• languages: Add Brazilian Portuguese specific hyphenation points (895c575)
• packages: Biblatex data inheritance and field mapping (646e3a4)
• packages: Support @string syntax in bibTeX bibliography (63083ad), closes #2051
• packages: Support @xdata entry type and xdata field in bibTeX bibliography (ca906f0)
• packages: Support crossref field in bibTeX bibliography (c4ae919), closes #2021
• tooling: Add developer target to build Rust API docs (384fa46)

Bug Fixes

• build: Avoid building sile binaries if not using them (bbde8e6)
• build: Avoid the perceived need for an extra automake cycle in dist tarball (f634a60)
• build: Do not distribute main man page, requires Rust tooling (26a6346)
• build: Don't distribute distfiles list with configure option specific output (b9d95fd)
• build: Flag transitive dependency on Git if not using system LuaRocks (ea2d501)
• languages: Always set Fluent locale when setting document.language (283fdc3)
• packages: Correct handling of ampersands and tildes in bibtex (aaa613f), closes #2050 #1860
• packages: Ignore @preamble in bibTeX bibliography (742a0c4), closes #2051

v0.15.3

Summary of Improvements

This release fixes a regression in v0.15.0 involving 3rd party modules. We were not referencing the LUA_PATH (and LUA_CPATH) environment variables at run time, only at build time. This made it unreasonably difficult to install and use 3rd party modules to a user's $HOME directory and use them in SILE. We were finding modules installed at the system level or locally to a project, but other trees were not being seen.

With this fix, installing modules via luarocks --local install should be a viable workflow again. Before running SILE, make it aware of these modules with eval $(luarocks --local path). The same goes for any arbitrary luarocks --tree locations.

Bug Fixes

• core: Allow LUA_PATH env var to take effect at runtime (e573c2d)

v0.15.2

Summary of Improvements

The big v0.15.0 release had some rough edges. But we mentioned that in the v0.15.1 release notes already. The truth is we fumbled some of the sources in the source tarball again. Here is to being able to build cleanly from the source tarball again.

Bug Fixes

• build: Bundle all assets in source distribution (5198641)

v0.15.1

Summary of Improvements

The big v0.15.0 release had some rough edges. First the source tarball was missing a vendored source file needed to build against some Lua versions. This did not affect building from Git sources, but made downstream distro packaging difficult. Additionally the now completely removed legacy package manager was still asking for Git as a run time dependency, which is now obsolete.

Also the big changes to the way SIL sources are parsed revealed several order bugs in the book class. These were visible even in the SILE manual as a regression in the way indentation was (not) suppressed after chapters and sections. This was all cleaned up and hopefully paragraph handling is a bit more logical now. Note that per the breaking changes in v0.15.0 custom implementations of these functions will need to be adapted the same way.

Bug Fixes

• build: Distribute vendored compat-5.3.c source file (3c469d6)
• build: Remove Git from configure dependencies (352a125)
• classes: Correct order of operations when paragraphs are ended manually (08f1a7b)
• packages: Change retrograde package so 'target' version works as expected (6ad9b76)

v0.15.0

Summary of Improvements

In the works for over a year with over 500 commits and 100 issues closed, please welcome the biggest single release in SILE history. This is the big “Rewrite it in Rust”! Caveat lector. In truth this is a big release but it is not a rewrite. All the typesetting internals of SILE are still written in Lua and 100% user modifiable at runtime. From an end user or 3rd party module developer standpoint little has changed. However SILE itself is now a compiled Rust application that includes its own Lua interpreter. The build process can (optionally) embed all the Lua and other resources files that makeup SILE and its dependencies in a single binary. This opens the door for improvements such as being able to leverage Rust libraries (including exposing their functions to Lua), write some parts of core functions in Rust for performance or preference, write modules in languages other than Lua or C, package SILE for platforms where Lua is not easy to get running, and much more.

The language change mostly affects building and packaging SILE itself. Once running, relatively little has changed with the way SILE interacts with documents. That being said quite a number of default settings have been changed. See the Usage section of these release notes and the retrograde module documentation for tips on how to transition smoothly.

For a transition period, the Lua based CLI is still available as sile-lua. This may be useful for scripting environments that generate inputs and/or parse the output of SILE itself. These should be transitioned to the new Rust CLI sile which has a few minor differences in argument handling and output message formatting. The Lua CLI will only be available for a limited number of future releases. Please do report any issues using the new CLI.

Extra thanks to @Omikhleia for lots contributions and input during the development cycle; and also to @ctrlcctrlv for generous sponsorships that enabled me to commit quite a bit more time to development.

Installation: For Anyone Installing From Packages

If you install SILE from your distro's package manager or other packaging, nothing about your process needs to change. Update via your system tools and enjoy.

Installation: For Distro Packagers and Source Installations

If you install from source or package SILE, the build command sequence is the same but there are new prerequisite dependencies. It now requires Rust tooling (cargo, rustc) as well as some more utilities (jq) to be available at build time. No new dependencies are needed at run time. In fact it is now no longer necessary to use a Lua VM available at runtime for the new CLI. SILE brings its own Lua VM along. (The legacy sile-lua CLI of course still uses Lua as before.) Optionally the build process can also be setup to embed all of its runtime dependencies in a single binary. See ./configure --enable-embedded-resources and ./configure --enable-static if you want to pursue the single binary route. Most users should use the defaults that install the Lua files and other assets separately as an easy reference for tinkering with and overriding them.

Another notable change is that the default Lua VM has been switched from whatever the system supplied to to LuaJIT. This has been an option for a while, but the default has been whatever the newest PUC Lua version was on the host system. LuaJIT is roughly equivalent to Lua 5.1 and doesn't have some small niceties from 5.4, but it is much much faster. Some of the differences are papered over since SILE depends on and provides the compat53 library compatibility layer. Of course build time SILE can still be configured to used any version of Lua of your choice. This can be used to match compatibility with 3rd party modules or other system components.

Usage

A number of command behaviours and default settings have been changed. This will likely cause documents to render with a different flow. It will also break support for some 3rd party modules which will need to be updated to match. Many (but not all) of the changes can be temporarily disabled to cause as few changes when rendering old documents as possible. A new module called retrograde can be loaded at run time that will reset defaults and even revert some commands and functions to their previous behavior. A target argument can be passed for the version of SILE your document was designed for. Any default setting changes and as much other functionality as practical that may have changed since that release will be reverted.

$ sile -u 'packages.retrograde[target=v0.14.17]' <INPUTS>

Since the Lua VM version used by default is different, you may need to reinstall 3rd party modules with a matching Lua version. You can first query SILE to understand what version of Lua it is using. With that information you can specify the Lua version you want when you install modules as LuaRocks to match. As a demonstration we'll install a Markdown input module in a project-local directory where SILE will find it without extra path configuration, then use it to render a PDF file:

$ sile -q -e 'print(SILE.lua_version); os.exit()'
5.1
$ luarocks --lua-version 5.1 --tree lua_modules install markdown.sile
[…]
$ echo 'Test *Markdown* rendering.' > test.md
$ sile -u inputters.markdown test.md
[…]

Without further ado, here is the nitty–gritty.

⚠ BREAKING CHANGES

• packages: Lists now respect the input document spacing and normal settings with regard to paragraphs breaks before, after, and inside lists. This is place of overriding the paragraph skip settings to match the list item spacing setting and always forcing paragraph breaks before and after lists.

• classes: Hitherto SILE has cleared the current.parindent setting as soon as it used it at the beginning of a paragraph. With this release, the setting is not being cleared until a paragraph is explicitly ended. This will not have an affect on many documents, but could completely blow up layout code that implicitly relied on the effect. Normal paragraphs (e.g. separated by a blank line in the input) and any use cases that explicitly called far ending a paragraph (e.g. by calling \par) will be unaffected. But anywhere a paragraph break was simulated by adding vertical space, the indentation will not be applied the next start of a line. This means that calling any variant of \skip inline in a paragraph will result in content beginning on a new line without using the parindent setting.

  Fixing this change in behaviour requires either explicitly resetting the current.parindent setting after it is initially used or explicitly ending a paragraph before or after placing a vertical skip.

• inputters: Input documents using the SIL language will now retain whitespace more consistently. Whitespace following environment blocks is no longer swallowed in differently than space following command syntax. Consecutive line breaks in the input will consistently trigger new paragraphs no matter what they follow.

  Note that this change cannot be patched over via the retrograde package settings because by the time your document could specify what packages to load or settings to set, the input document has already been parsed. To achieve the same rendering results where environments could be ended leaving any amount of blank lines and still joined to the following content as part of the same paragraphs, you will need to remove the extraneous whitespace.

• core: Several top level instance creators of various names have been re-organized under SILE.types.

  Specifically SILE.color, SILE.measurement, and SILE.length have the same names, just under SILE.types.. Additionally SILE.nodefactory is now SILE.types.node and SILE.units is not SILE.types.unit.

  This brings a little bit of sanity to the naming schemes so that you can guess how to use something from the name, but it also makes room for 3rd party add ons to more easily extend or replace these functions. It also makes it easier to start substituting Rust bits where desired.

• core: Use SILE.papersize() instead of SILE.paperSizeParser()

• classes: The "center", "raggedleft" and "raggedright" environments formerly reset the margins (left or right skips), meaning they'd take the full frame width. They all cancelled the paragraph indent. The new behaviour honors the fixed part of the parent context's margins, meaning that if you have an environment playing with margins such as an epigraph or an indented quote, those margins are not lost. The raggedleft and raggedright environment also now no longer cancel the paragraph indent.

• classes: The \script function was heavily overloaded to have many different functions at once and more targeted tools were introduced in SILE v0.14.0 To load 3rd party modules designed for use with SILE, use \use[module=...] instead of \script[src=...]. To run arbitrary Lua code inline use \lua{}; Lua code may be provided inline or externally via either a require= option to load a regular (non-SILE) Lua module using the Lua module path or src= option to load a file by file path.

• core: For ... reasons ... the default width of spaces in SILE has been a highly opinionated and non-standard 1.2 spaces. While it can be argued that this makes some fonts and some documents look better, it is a very strange thing to have as a global default. Unfortunately setting it back to a more conventional 1 space is a major change and will cause many/most documents to re–flow.

  The old default can be recovered either in documents with:

  \set[parameter=shaper.spaceenlargementfactor,value=1.2,makedefault=true]
  

  ...or even from the CLI when rendering a document:

  $ sile -e 'SILE.settings:set("shaper.spaceenlargementfactor", 1.2, true)'
  `...

v0.14.17

Summary of Improvements

The "big v0.15" is still brewing, but we keep coming up with little improvements that easily fit in the v0.14 series. It turns out the special hyphenation handling we added to Polish (for explicitly hyphenated words) is used by quite a few languages. At this time we've applied it by default to Croatian, Czech, Portuguese, Slovak. and Spanish. Thanks to @Omikhleia for much of the research on this as well as @jodros, @DavidLRowe, @jakubkaczor, and @tomas-vl for domain expertise.

Additionally João contributed an option for columns command to more easily set up column frames with balancing turned on or off.

Features

• languages: Enable explicit hyphen repetition handling in Croatian (c29545d)
• languages: Enable explicit hyphen repetition handling in Czech (b05d621)
• languages: Enable explicit hyphen repetition handling in Portuguese (2a58d96)
• languages: Enable explicit hyphen repetition handling in Slovak (82640b0)
• languages: Enable explicit hyphen repetition handling in Spanish (8db7f23)
• packages: Add balancing option to makecolums command (#1950) (b5ce8e6)

v0.14.16

Summary of Improvements

If goofed up some of the merges involved in the release process for v0.14.15. This doesn't bring any user facing features or fixes, only a bit of refactoring intended to be in the last release. The main purpose is to straighten out the repository so the expected branch contains the release tags. Sorry for the noise.

Bug Fixes

• tooling: Re-merge so release tag is on master branch (e8eaf11)