[Epub] Recommendation on handling relative assets/css

[chrisrzhou]

Context

I am trying out and learning how to apply unified in writing an epub reader. Readers such as epub.js exist but this is possible, and potentially cool to do in unified!

Approach

• Unzip .epub contents.
• Loop through spine in content.opf and keep track of contents (html/xhtml).
• Render contents with rehype (apply whatever plugin makes sense).
• Win!

Problem

My current problem is that I do not know how to handle CSS/assets defined as relative links in the epub archive. For example, this epub has images and CSS files:

And the various HTML files reference them relatively:

Requirements

There is no intention to statically serve these files. The above problem should easily work if the files are served in the same folder and the relative links 'just work', but I want to do this without physically storing the files (e.g. a user should be able to upload an .epub file with a file input and we will use the program above to render it on the fly).

Potential Solution

• Loop through all assets and turn them into data URIs and internal CSS (for image/video assets and CSS respectively).
• Keep track of them in some local state.
• Write a hast util to replace relative links with data URIs or internal CSS.

I think this could work in theory, but there's potential duplication of CSS and assets for each HTML page (most likely CSS duplication).

Questions

• What other pros/cons exist for the solution mentioned above?
• How would we be able to share/point to the same CSS/asset resource without potential duplication (with the approach I mentioned above)?
• Any recommendation on how this problem could be approached?

[chrisrzhou]

Note: I roughly browsed through the source code of epub.js and it seems their approach is to turn assets/CSS links into resources: https://github.com/futurepress/epub.js/blob/master/src/resources.js#L106.

Would like to know and get recommendations from the community on how this could be solved/organized in the unified way.

[ChristianMurphy]

Either way could work, The pros and cons aren't necessarily unified related.

Any recommendation on how this problem could be approached?

If you want everything to load at once, inline/encode the content.
If you way to lazy-load or deliver different versions the content, externalize it.

How would we be able to share/point to the same CSS/asset resource without potential duplication (with the approach I mentioned above)?

It depends on if duplication matters in your use-case.
If you all inline content, there is a possibility of duplication, that may be fine, it reduces the number of network calls to get a page to one.
If you externalize the content you can prevent duplication, but external content will generate more requests to the server.
There's a trade off, pick what works for you.

[wooorm]

Btw, I don’t think the content.opf file matters to what you’re doing with css or images. Yes, for EPUBs all files must be referenced there, but how they’re referenced from HTML, does not take into account anything in the OPF file. I believe an URL of the form /a.jpg is resolved from the root of the archive, whereas a URL of the form a.jpg, ./a.jpg, or ../a.jpg are resolved from the path to the directory of of the HTML file (available on a file at file.dirname, made absolute by path.resolve(file.cwd, file.dirname) I think).

I’ve previously had the problem of dealing with archives from various sources, and I didn’t want to match the potentially complex, or even maybe dangerous (on certain file systems) results from unknown EPUBs. What I did there is find all srcs and hrefs, resolve them, and then have a dictionary mapping (e.g., complex/or/dagenrous.jpg to a new UUID (asdfghj.jpg)) which I’d put the static files at and updates the nodes to.

[chrisrzhou]

It seems that items in the manifest tracks each resource assignable to an id. I'm assuming that could be a natural ID to use?

I'm going to start by writing some code today and get a feel for how this will go.

Pseudocode

const files = unzip(epub);

// create an array containing `manifestId` that indicates the order of pages.
const navigation = getNavigation(files);  // e.g. ['item1', 'item2', 'item3'];

// get a mapping of `hrefs/srcs` to `manifestIds`
const manifest = getManifest(files);  // e.g. { '../href1': 'item5', '../../href1': 'item5', 'href123': 'item6', ... }

// create a mapping of `manifestId`to 'inlinable' CSS
const styles = getStyles(files);  // e.g. { 'item5': ".cover { ...}\n .background {...}\n ...", ... }

// create a mapping of `manifestId` to 'inlineable' dataURIs
const assets = getAssets(files);   // e.g. { 'item8': jpg1DataUri, 'item10': video3DataUri... }

// create a mapping of `manifestId` to HTML file
const contents = getContents(files);  // e.g. { 'item1': '<html>...</html>' }

At this point, we should have styles, assets, contents that are interlinked to each other, and these mapping objects seem to store their unique data without duplication. For rendering a page, I think we should have enough information to go from here:

const resources = { manifest, assets, styles }
function render(contents, pageNumber, resources) {
  const contentManifestId = navigation[pageNumber];
  const content = contents[contentManifestId];

  const html = unified()
    .use(rehypeParse)
    .use(resolveIds, resources.manifest) // replace/resolve `hrefs/srcs` with matching manifest Ids
    .use(attachStyles, resources.styles)  // custom plugin to add `styles` as inline CSS in `head` node
    .use(urifyAssets, resources.assets)  // custom plugin to convert all resolvable `hrefs/srcs` (now referenced with `manifestId`) to inlineable dataURIs
    .use(stringfiy);

  return html;
}

In my project's use case, I cannot persist or store the data as static files because everything has to be done in the browser (it's a little browser script that does everything on the fly without setup). Earlier, I was worried that all contents with replaced inline data URIs and inline CSS would take up a lot of memory and tons of duplicated assets. With the approach above, I think the data that I store is orthogonal and unique:

• contents
• styles
• assets

When a page is rendered, it can lookup this relevant data and generate one page for rendering. If we navigate to a new page, we can recreate it using the relevant data we need.

Anyway, just vocalizing my thought process and if there are any improvements/suggestions with the rough approach here and what I should look out for, it's greatly appreciated before I make big mistakes that could be avoided early :P

[chrisrzhou]

Update: using the quick approach above, I was able to render epub pages in a a controlled manner with a simple implementation.

https://unified-doc.netlify.app/examples/epub (source code link in example).

Keeping track of assets by ID and only generating them as inlined styles and data URLs when a specific page is rendered seems to work well. There are some obvious holes and edge cases not handled, but I'll continue to flesh it out over time in the future. One main challenge is not knowing how to replace links within Ebook pages. I'm not sure how to handle this given the approach I've taken. The only way I can think of is to turn links into click handlers that allows the appilcation sitting above the rendering code to decide which page to navigate to. It's likely not possible to simply use semantic links in the simple approach I've taken.