v0.14.5

Summary of Improvements

We're just squishing bugs today and making the manual a little neater. Working with STDIN streams should be a bit more robust now as the content type detection isn't so picky about whitespace and isn't so prone to false positive detections. Document file restrictions have been relaxed a little too allowing comments and whitespace before the leading document tag. Package developers should have a little bit easier time with in the event their package is loaded twice. Also a number of small typesetting issues in the user manual were cleaned up by @Omikhleia.

Bug Fixes

• inputters: Correct false positive detection of STDIN as Lua content (d54946b)
• inputters: Don't duplicate passthrough content in AST (07c8e87)
• inputters: Permit content outside of the document note, e.g. comments or blanks (#1596) (f1a508a)
• inputters: Relax SIL format sniffing to allow valid syntax (43fc4bc)
• languages: Remove superfluous line (848b91f)
• languages: Tidy up variable scope in languages/unicode.lua (78b453d), closes #699
• measurements: Allow redefinition of existing units (#1608) (8d81018)
• packages: Ensure a page switch does not break boustrophedon (#1615) (64abaf9)

v0.14.4

Summary of Improvements

Today we have a small rollup of fixes and improvements. @Omikhleia overhauled the counters packages as well as fixed up bugs in several other packages. The default center of rotation has been changed to give a more expected result. Text casing functions are accessible again from the Lua side of things, changing masters during a page output doesn't break page breaking, and using the twoside package doesn't force the use of mirroring. @OlivierNicole also stepped in to fixup some math bugs. Also thanks to @ThatGeoGuy and @snan for pointing out and fixing small issues in our documentation.

Features

• packages: Add boolean noleadingzeros option to counter formatter (e4f8133)
• packages: Add new command \set-multilevel-counter (11578a8)

Bug Fixes

• classes: Always break out of hmode before processing \chapter headings (0c44d8e)
• core: A typo in a variable prevents using -u with a class (b8f5c40), closes #1569
• languages: Make 'und' an exception to language name canonicalization (52e9b79)
• math: Fix insertion order of MathML children (738e9e6)
• packages: Account for depth when calculating rotation center (289dd2a)
• packages: Avoid forcing mirrored masters in twoside package (#1562) (8cdf6ed)
• packages: Combine \unichar'ed chars with same font only (91a8d40)
• packages: Correct rotation origin calculation back to pre v0.10.0 (3521936)
• packages: Don't inhibit page breaking after switching masters mid-page (6b20f73)
• packages: Make sure PDF initialized before rotate package directly calls it (449b2a6)
• packages: Rework simple and multilevel counters (1e6e91a)
• packages: Textcase package name typo preventing using methods from code (7f68766), closes #1568

Reverts

• Revert "docs(packages): Fixup unichar documentation, work around known bug (#1549)" (03d1b11), closes #1549

v0.14.3

Summary of Improvements

This release addresses a few pain points discovered using the v0.14 series in production.

For 3rd party package developers, it's now easier to find non-Lua resources you installed along with your package. Among several contributions from @Omikhleia, the output backend now returns useful information about images it embeds directly. The same work also allows specific pages of PDFs to to be emdeded as images, not just the first page. He also helped refactor existing interfaces to make them easier to extend, such as the table of contents package. A long standing issue with Turkish hyphenation has been addressed, eliminating the need for preprocessing or other workarounds. More long standing issues with the book class and chapter opening spreads have been addressed. Some priority and interactions between new CLI flags has been worked out. A few more misc bugs were also squished and relevant tests added.

Features

• languages: Handle hyphenation of inter-word apostrophes in Turkish (50ae936)
• packages: Add \open-spread function with more features that \open-double-page (c2ba579)
• packages: Add ability to select a page in PDF images (a477d94)
• packages: Allow for customized content on otherwise blank filler pages (5ae97bf)
• packages: Provide base directory to packages (#1529) (f9ae994)
• utilities: Return image resolution with libtexpdf backend (a9c11d3)

Bug Fixes

• cli: Actually apply cli provided class options (505919e)
• cli: Allow CLI option to override document specified class (5232ce8)
• languages: Make Turkish hyphenation less bad around intraword apostrophes (008d4c4)
• nodes: Correct calculating width of postbreak discretionaries (ea7912c)
• nodes: Work around discressionaries being output when not wanted (c7dc439)
• packages: Fix over-aggressive eject in \open-double-page (5620556)
• packages: Homogenize image width and height as measurements (b91cfbb), closes #1506
• packages: Make sure PDF initialized before PDF package does anything (#1550) (ebc3748)
• packages: Resolve src= relative to document for SVG images (b55fc98), closes #1532

Reverts

• Revert "chore(build): Avoid mktemp during build, breaks opensuse packaging (#1542)" (bca007f), closes #1542
• Revert "chore(cli): Output header before doing anything that might throw warnings" (58da8ad)

v0.14.2

Summary of Improvements

It’s a slow news day, but these few little fixes touchup glitches introduced in the v0.14.0 release and generally improve the robustness of the new API interfaces. Writing (and porting) classes should go just a bit smoother and behave more as expected now.

Bug Fixes

• classes: Allow package option declarations to be reset (215e83a)
• classes: Bring back space after subsection numbering (70a3304)
• packages: Check for user supplied commands before setting noops (54b5071)

v0.14.1

Summary of Improvements

These few quick touch-ups smooth over last minute bugs and ease the transition process from v0.13.x. Besides squashing a pointless warning messages and some bug fixes, this changes the order deprecated includes are processed to facilitate the transition of old projects to new APIs by making it easier to update them one bit at a time.

Features

• inputters: Expand postamble functionality for parity with preambles (#1518) (eb09eb3)

Bug Fixes

• cli: Suppress deprecation message for internal shims (b339e27)
• cli: Swap order of new --uses and legacy --include (ef0087e)
• packages: Correct (and improve scope of) exported testcase functions (fd438e9)
• tooling: Make sure Git version detection only picks up semver tags (25d669a)

v0.14.0

Summary of Improvements

Work to refactor SILE's internals into a more regular API has been in progress for some time. This release brings a big chunk of that work to light. Many/most of the changes are under the hood and won't show up in the appearance of documents. They do affect anybody writing classes or packages for SILE, hopefully for the better.

Most notably, 3rd party modules have much more robust support. A module may be a class, a package, a typesetter, a shapper, an inputter, an outputter, etc. This makes it easy for developers to extend SILE without needing to modify the core distribution. Modules can be packaged and installed using LuaRocks. LuaRocks.org may be used to host and distribute SILE addons. Rocks may also be distrusted independently. Either way they are easy to install with luarocks install and such installs will make the modules available for loading into SILE. The old built in package manager is still functional for now but deprecated.

The ergonomics of packages has changed a little. The overloaded \script command has gained some more specialized peers. For example Lua code may be passed in \lua commands. To load a package or other module, use \use[module=package.name]. This syntax also allows passing options to modules.

The inputters have been normalized a bit, and XML, SIL, and Lua are all first class input languages. Passing some content in XML format from a SIL format document is as easy as placing it in an \xml command. The same goes for SIL format content in an XML document.

3rd party developers may add new input formats as well! All that is required is an inputter module that returns an AST. Expect to see new inputters soon. @Omikhleia has already been working on a new Markdown inputter module. Other XML based formats such Docbook have been supported for a while, but the new class and inputter APIs make them easier to add and use.

The CLI has gained several new options. As usual sile --help and man sile both have relevant information. Notably the -I/--include argument has been deprecated in favor of more explicit -u/--use, -p/--preamble, and -P/--postamble. As you might expect from those options this makes it possible to load an inputter, class, or package from the command line without having it set in the document. The preamble/postamble system has been cleaned up so content or code can easily be injected at the start or end of a document. This is a change from the previous 'preamble' system that actually became the master document and then included the original master as effectively a postamble.

Many old shims and deprecated functions have been completely removed. Most will still throw relevant warning messages with hints about how to update.

While I was busy moving cheese, several contributers were chipping in new features and fixes. @rolfmblindgren pitched in Norwegian localization and improved hyphenation. @matteodelabre contributed quite a few new features and fixes to the math package. @Omikhleia did some data mining and came up with localization strings for a whole slew of languages. He also pitched in fixes to a number of packages. He also has pending contribution for many new packages and features. Be sure to checkout the PR scene on GitHub if you are looking for new features.

⚠ BREAKING CHANGES

• packages: The primary use was probably internal to SILE, but if by chance you have bibtex databases with formatting commands in SIL markup format rather than just plain text content the markup will cease to function and will need to be converted to XML syntax instead. This enables the use of declarative markup in Fluent localizations.

• classes: Each SILE package now inherits some interfaces from a common base package. This model allows packages easier access to SILE internals while at the same time tracking what they do so it is easier to enable/disable them. The package knows which document class instance it is attached to, and the document class knows which packages are loaded at any given time. Legacy style packages will continue to work for the time being but will not be tracked in the same way.

• core: The role of document commands has always been tightly scoped to classes. For example the book class has a \footnote command while plain does not—unless you manually load the package and setup the frames. In spite of this obvious functional scope, registering commands has been a global operation that stored them in a global registry. In order to allow SILE to be used more programmatically as a library with potentially more than one document and class being processed at at once, these need to be moved out of the global scope. This will also facilitate things like being able unload packages and revert to previous functionality for anything they over-rood on load. For now the functionality is shimmed, but code using the SILE.registerCommand() function should switch to the method of the same name on the current class, i.e. class:registerCommand().

• core: Some internal files and APIs got renamed with more structured name spaces. In particular the inputter, shaper, and outputter libraries all have a common naming scheme now and sensible inheritance chains. No functionality was harmed, but if you are overriding undocumented internal Lua methods you might have to update your name spaces to match.

• cli: The -I / --include option was overloaded for more than one purpose and is now deprecated in favor of more specific replacements: -r / --require for loading code into SILE before input processing, -p / --preamble for processing content prior to a document and -P / --postamble for processing content after a document.

• packages: The original package manager POC that used Git to clone packages into the SILE installation directory has been deprecated. It will continue to function for a while, but all new 3rd party packages should use the LuaRocks based installation process. Whether or not they use luarocks as a package manager or LuaRocks.org as a distribution channel they should install themselves to any usable the system or user Lua library path under a top level "sile" namespace.

• classes: The shims allowing classes designed for SILE releases v0.12.x and prior have now been removed and documents using them will now throw errors when rendering. Only the refactored class system introduced in SILE v0.13.0 is supported going forward.

• deps: We previously deprecated all use of stdlib. This release stops providing it entirely. If you use it in your own projects you will now need to provide and require() it directly.

• build: The C modules compiled as shared libraries (.so files on Linux, .dll on Windows) are now installed to the project root shared directory instead of it's 'core' subdirectory. Distro packages that split the library into its own package will need to adjust this path. People installing from their distro packages or from source should be unaffected, but this will bring us one step closer to being able to install and use SILE as a library.

Features

• classes: Add \use command to help deconflate \script usage (eb298c3)
• classes: Track loaded packages per document class (32bd87b)
• cli: Add CLI argument -E for evaluating Lua code after input (5948aca)
• cli: Add usage hints and cleanup output of errors (cc58824)
• cli: Allow loading custom inputters from -r option (a212e83)
• cli: Allow passing options to any modules specified from --use (4cdcae7)
• cli: Change --require to --use to match declarative markup (2411328)
• core: Add ability to pass args to modules via \use and other commands that load modules (9e54bad)
• core: Add ability to pass args to modules via \use and other commands that load modules (e64ce0f)
• core: Add inline-escaping in SIL-language (f09b135)
• core: Support loading classes/packages installed with luarocks (232e72b)
• i18n: Add more Russion localizations (350cf14)
• i18n: Add support for as many languages as possible (da57577)
• i18n: Fallback to messages from 'und' language if no localized (9f47715)
• i18n: Parse XML style SILE commands in Fluent messages (989290b)
• inputters: Allow arbitrary root elements from XML input without a preamble (ad46a92)
• inputters: Allow CLI to mandate inputter used for master document ([1b9009f](https://g...

v0.13.3

Summary of Improvements

The upcoming v0.14.0 is chock full of changes (hopefully all for the better!) but several small issues have cropped up in the v0.13.x series and there doesn't seem to be any point in inconveniencing existing users with bugs we've already fixed while we work on new features. This should fix up some issues with includes not activating exported functions and getting blocked when both directories and files of the same name exist. The Nix Flake will now work with the new Nix 2.10.x releases.

Additionally @Omikhleia contributed some fixups and features to the Bibtex package to make it much more usable.

Features

• packages: Add minimal support for usual BibTeX types (bibtex) (292a2f2)

Bug Fixes

• build: Update Flake to work with Nix >= 2.10 (3d5a18c)
• core: Avoid duplicate paths blocking directory searches (7a7209f)
• core: Avoid error when outputting overflow warnings with specific measurements (cb51ed5), closes #945
• outputter: Non-RGB colors shall work with the debug outputter (#1469) (41fbdf4)
• packages: BibTeX types/tags are case-insensitive, etc (61c1fc6)
• packages: Make \script command properly initialize packages (9ded7e1)

v0.13.2

Summary of Improvements

This patch release brings both bug fixes and new goodies.

@Omikhleia contributed a new package for formatting lists. Features include bullet, numbered, and mixed list types with many styling options.

@ctrlcctrlv fixed zenkaku unit handling and helped refactor the jplain/jbook classes so we now have tate-capable but language-agnostic tplain/tbook classes in addition to the Japanese specific variants. He also contributed full support for Esperanto including hyphenation, number formatting, and localized output strings. Even if you don't have a use case for Esperanto directly the issues that rooted out with language and class handling should benefit everyone.

The papersize option provided by the base class is now case insensitive and allows dashes. If you prefer A4 to a4 or ANSI-A to ansia feel free to use them. A couple new paper size presets were added in the process.

Additionally the font fallback system got an overhaul. A change from previous behavior is that TOFU handed out by the last available fallback font will be rendered instead of skipping the characters completely. Switching back and forth between the color fonts package and the font fallback package is now possible. Finally it should stop throwing bogus warnings when it successfully shapes a segment via the last available fallback.

Other minor fixes include interactions with color plus dropcaps, some bibtex errors, and errors in the pecha class.

Features

• core: Add presets for some ANSI paper sizes and ArchE variants (0f26756)
• languages: Add full Esperanto language support (b740709)
• packages: Add 'lists' package (bullets and enumerations) (6af3c62)
• packages: Add more options for custom 'lists"' styling (3167410)
• packages: Handle font fallback when glyph named null returned on shape falure (09c0a86)
• packages: Pass through font-specific options to fallback fonts (fb29442)

Bug Fixes

• classes: Clarify the scopes of tate and jplain (db83e9e)
• classes: Fix circular reference in pecha class (4501ec0)
• classes: Fix diglot and triglot class instantiation (71af1a9)
• core: Make paper size parser case insensitive, e.g. 'a4' or 'A4' (af441c8)
• measurements: Move the zenkaku width (zw) unit into core (cfe5060)
• packages: Correct fall-back font processing (d3cc59b)
• packages: Correct package load path for colored dropcaps (41a0c17)
• packages: Fix coding errors in untested corners of bibtex package (804b1a5)
• packages: Fix loading TOC twice resetting pdf links (97797b8)
• shaper: Handle switching between color & fallback shapers in single document (04f2d5d)
• utilities: Raise Lua error instead of manually aborting if inside pcall() (6e70a17)

v0.13.1

Summary of Improvements

Hot on the heals of a major release we have small one to patch up some odds and ends. Thanks to @ctrlcctrlv for correcting my oversight in not distributing some of the new localization files. If by chance you've on a new fangled hardware platform like RISC-V things should build out of the box now. The documentation for installation on various systems has also been cleaned up. Thanks to @Omikhleia for some new rule functions and fixes to old ones plus leader alignment corrections.

Features

• build: Update libtexpdf to support new hardware platforms (da1182e)
• packages: Add hrulefill command to the "rules" package (ccd3371)
• packages: Add strikethrough command to the rules package (#1422) (f230a3a)
• packages: Use new strikethrough when rendering Panndoc's SILE writer (20d19eb)

Bug Fixes

• build: Make sure i18n/ dir is actually distributed (#1445) (61ed8e1)
• packages: Add more props to keep CJK from tipping over, per #1245 (381b9f1)
• packages: Leaders shall be an explicit (non-discardable) glue (631ba21)
• packages: The fullrule now extends over a full standalone line (8fe57c8)

v0.13.0

Summary of Improvements

In the last patch release I hinted we had bigger things coming. This release finally surfaces major internal changes I've been working on since 2019. If our releases had code names this one might be "class wars". The SILE code base has two distinct usages for 'class'. One usage is our document classes, cohesive bundles of features and styles used to render a document. The second usage is the object-oriented programming paradigm of class inheritance. In serendipitous naming ripe for conflation, the document class feature is itself written using programming classes, enabling document classes to inherit from each-other.

This release refactors the implementation of both types of classes. Since nearly its inception, SILE has depended on the stdlib project for many of its Lua abstractions. Unfortunately this library is not as standard as its name suggests, has not been developed or even well maintained, and has been a constant source of problems. It even tinkers with core Lua functionality making it very difficult to keep SILE working across interpreter versions and to debug problems. For some time we've been gradually replacing it with abstractions from the Penlight library (loosely based on Python's standard library). While not perfect, that library of functions is much more robust and predictable. This release finally removes the last of SILE's usage of stdlib. For this release cycle the core program will still provide the stdlib library to ease transition for downstream projects that currently assume it is present, but it is no longer used internally. Beginning with the next major release cycle, v0.14.0, it will not be provided at all. (You will of course be free to include it as a direct Lua dependency in your own projects.)

We've attempted to shim most of the API changes so that most old SILE documents (and in particular, custom classes) continue to work out of the box. Most simple projects should still render without intervention, but you may see many warnings about deprecated functions. A few more advanced projects may run into trouble and fail to render at all, especially if they messed around with classes (of either sort) very much. As far as possible we've tried to add warning notices with hints about how to use the new class models correctly. Please don't hesitate to open issues if you are having trouble getting anything to work. For package authors, a new hook system should make it a lot easier to write packages that do more without also having to write a dedicated class to use them.

Additionally the localization system for all language specific strings that may be rendered by SILE has been changed from the home-grown system of nested SILE commands to a more flexible i10n system based on Fluent. The Lua implementation is not at 100% feature parity with Project Fluent reference implementation, but it is 100% interopperable with other implementations for features implemented and is a much more robust localization system than just substituting strings. Besides using it to easily customize the limited set of embedded localizations (such as the Table of Contents header or chapter titles), the tooling for localizing messages and rendering them in context aware functions is exposed for package developers and document authors to use at will.

A small collection of bug fixes to the typesetter rounds out this release. Justification of lines with ligatured characters is much improved. Rules with a depth property no longer throw off baseline calculations. Empty documents now generate blank PDFs. Adding a new master frameset layout no longer destroys the current page's frameset.

⚠ BREAKING CHANGES

• settings: All the functions under SILE.settings.*() should now be called using the instance notation SILE.settings:*(). Usage should be shimmed with a warning for now.

  Changing this in your code is relatively easy with a search and replace. As an example with a project in Git, you could use perl like this:

   funcs="pushState|popState|declare|reset|toplevelState|get|set|temporarily|wrap"
   git ls-files | xargs -n1 perl -i -pne "s#(SILE\.settings)\.($funcs)#\1:\2#g"

• typesetter: Making a new instance of the typesetter should now be done by calling SILE.defaultTypesetter() instead of copying the object. It has been changed from a std.object to a Penlight class. As such the correct initialization function is also now _init() instead of init(). A shim is in place to catch legacy usage, but this will be removed in the future.

• deps: All calls to the Lua default string library have been using a version monkey-patched by stdlib. This has created all sorts of issues including not being able to properly use some of Lua's default features and conflicts with out explicit meta methods. Also we're busy dropping dependency stdlib altogether.

  If you were relying on it for any of your string operations, replace string.func() with std.string.func(). For now std is being provided by SILE, but if you use it in your projects please add it as a direct dependency yourself since that will eventually be removed as well.

  By the way in case anything ever git bisects back to here, one way to test if your problem is related to this change or not (especially if you have downstream code that might have built on the assumption SILE's Lua strings were monkey patched) is to load it manually yourself:

   $ sile -e 'require("std.string").monkey_patch()' your_file.sil

• classes: This changes the way classes are represented as Lua objects and the mechanism used for inheritance. While shims will be in place to catch most cases that use old syntax it is not possible to grantee 100% API compatibility. If you have classes that do anything remotely fancy (i.e. not just copy/paste from SILE examples) they may or may not work at all; and even if they do they should be updated to
  explicitly use the new API.

Features

• classes: Add hook system for more versatile packages (9287721)
• languages: Add \ftl command to make adding fluent localizations easy (b331456)
• languages: Add fluent() command to output translations (ad87995)
• languages: Validate languages against CLDR database (f96a331)

Bug Fixes

• backends: Add Pango shaper when selecting Cairo backend (bbc2817)
• backends: Always output pdf on finish() even if no content (3af7a94)
• backends: Correct image sizing in Cairo and Podofo backends (f2785ad)
• core: Avoid throwing deprecation errors when just inspecting SILE's internals (b303059)
• core: Justify lines with ligatures (workaround) (cf2cb3a)
• core: Patch Penlight 1.9.0 compatibility issue (092fbd3)
• languages: Correct bogus usage of resource loading / error catching (fb1fd7f)
• packages: An hrule with depth shall not affect current baseline (c759892)
• packages: Don't destroy frames when defining masters, only when switching to one (b7de7ca)
• packages: Fix autodoc parsing, typeset string not series of bytes (14f6126)

Miscellaneous Chores

• deps: Drop std.string.monkey_patch() (e8b2bdf)

Code Refactoring

• classes: Convert class inheritance from stdlib to Penlight (f7dafe0)
• settings: Change settings object to be self referential (dd97d05)
• typesetter: Change typesetter instancing to Penlight model (a9400ad)