Service publication PDF generation hang for a particular document

[ronaldtse]

See ituob/service-publications-docs#34 for details.

1060-M.1400/T-SP-M.1400-2014-E.adoc hangs in PDF generation.

$ cd sources/1060-M.1400
$ bundle exec metanorma T-SP-M.1400-2014-E.adoc
[info]: Compiling /Users/me/src/ituob/service-publications-docs/1060-M.1400/T-SP-M.1400-2014-E.adoc ...
Metanorma XML Style Warning: (XML Line 000074): Table should have title
Metanorma XML Style Warning: (XML Line 018958): Table should have title
Metanorma XML Style Warning: (XML Line 018980): Table should have title
Style: No Summary has been provided
Style: No Keywords have been provided
Local cache version is obsolete.
Style: Annex to ITU OB E.212 does not match ITU document identifier conventions
Document Attributes: published is not a recognised status
Metanorma XML Syntax: (XML Line 018939:73): element "fn" not allowed here; expected the element end-tag, text or element "add", "bookmark", "br", "concept", "del", "em", "eref", "erefstack", "hr", "image", "index", "index-xref", "keyword", "link", "pagebreak", "ruby", "smallcap", "span", "stem", "strike", "strong", "sub", "sup", "svg", "tt", "underline" or "xref"
Metanorma XML Syntax: (XML Line 019028:16): element "itu-standard" incomplete; missing required element "bibliography"
java -Xss5m -Xmx2048m -Dapple.awt.UIElement=true -jar /Users/me/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/mn2pdf-1.62/lib/../bin/mn2pdf.jar --xml-file "/Users/me/src/ituob/service-publications-docs/_site/documents/1060-M.1400/T-SP-M.1400-2014-E.presentation.xml" --xsl-file "/Users/me/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/metanorma-itu-2.2.6/lib/isodoc/itu/itu.service-publication.xsl" --pdf-file "/Users/me/src/ituob/service-publications-docs/_site/documents/1060-M.1400/T-SP-M.1400-2014-E.pdf" --param baseassetpath="/Users/me/src/ituob/service-publications-docs/1060-M.1400" --syntax-highlight  --font-manifest "/var/folders/2t/xmdrn2sd2lv2w49dv0zw9_q00000gp/T/fontist_locations20230309-8344-zk830u.yml"

[Intelligent2013]

The issue in table auto-layout Apache IF processing.
Source XML is 1.4Mb
XSL-FO file size 18Mb.
IF file size 39Mb.

[Intelligent2013]

Another issue - email's don't fit into the cells:

The reason: link contains address in the attribute, but auto-layout algorithm analyzes the node's text content:

<link target="mailto:[email protected]"/>

To do: in the template mode td_text_with_formatting update link to: <link>[email protected]</link>

[Intelligent2013]

Issue with links fixed:

[Intelligent2013]

Current processing time is ~580sec. I'll optimize further.

[Intelligent2013]

The table auto-layout XSLT optimized in mn2pdf. Now the processing time takes 130sec (was 580sec).

But some emails don't fit into the cells:

Need update in common.xsl.

[ronaldtse]

Thank you @Intelligent2013 !

I think the PDF times could also be further optimized, maybe it's something we should undertake.

[ronaldtse]

@Intelligent2013 we can use this repo to test:

• fix: workflow update ituob/service-publications-docs#36 (comment)

[Intelligent2013]

Issues with the overflowing fixed:

[Intelligent2013]

we can use this repo to test:

@ronaldtse I've updated mn2pdf (v1.63 https://github.com/metanorma/mn2pdf/releases/tag/v1.63) and ITU XSLT for fixing issues from this ticket.

From log (PR ituob/service-publications-docs#39): https://github.com/ituob/service-publications-docs/actions/runs/4387204614/jobs/7682219643, mn2pdf-1.57:

java -Xss5m -Xmx2048m -jar /tmp/packed-mn-20230310-20[72](https://github.com/ituob/service-publications-docs/actions/runs/4387204614/jobs/7682219643#step:7:80)-1w8qcht/mn2pdf.jar --xml-file "/home/runner/work/service-publications-docs/service-publications-docs/_site/documents/1000-SR.1/T-SP-SR.1-2012-MSW-E.presentation.xml" --xsl-file "/tmp/packed-mn-20230310-2072-1w8qcht/itu.service-publication.xsl" --pdf-file "/home/runner/work/service-publications-docs/service-publications-docs/_site/documents/1000-SR.1/T-SP-SR.1-2012-MSW-E.pdf" --param baseassetpath="/home/runner/work/service-publications-docs/service-publications-docs/1000-SR.1" --syntax-highlight  --font-manifest "/tmp/fontist_locations20230310-2072-1dfhosv.yml"
/__tebako_memfs__/lib/ruby/gems/2.7.0/gems/mn2pdf-1.57/lib/mn2pdf.rb:79:in `mn2pdf'
/__tebako_memfs__/lib/ruby/gems/2.7.0/gems/mn2pdf-1.57/lib/mn2pdf.rb:54:in `block in convert'
/__tebako_memfs__/lib/ruby/gems/2.7.0/gems/mn2pdf-1.57/lib/mn2pdf.rb:100:in `block in dump_fontist_manifest_locations'

I don't understand good how test-docker is working. But it means that docker image should be updated?

[ronaldtse]

@Intelligent2013 indeed we need to update the docker image. Ping @CAMOBAP on mn2pdf version. Thanks!

[Intelligent2013]

The issue occurs again after 2 weeks. I'll investigate the reason.

[Intelligent2013]

The issue relates to the metanorma site genarate command, not metanorma-itu.

[ronaldtse]

The issue relates to the metanorma site genarate command, not metanorma-itu.

It's not, we're not using site functionality here but only for a single document.

If it is not Java then this seems to be related to the mn2pdf-ruby gem where it is not returning?

My trace:

bundle exec metanorma T-SP-M.1400-2014-E.adoc
Calling `DidYouMean::SPELL_CHECKERS.merge!(error_name => spell_checker)' has been deprecated. Please call `DidYouMean.correct_error(error_name, spell_checker)' instead.
Metanorma XML Style Warning: (XML Line 000072): Table should have title
Metanorma XML Style Warning: (XML Line 018956): Table should have title
Metanorma XML Style Warning: (XML Line 018978): Table should have title
Style: No Summary has been provided
Style: No Keywords have been provided
Style: Annex to ITU OB E.212 does not match ITU document identifier conventions
Document Attributes: published is not a recognised status
Metanorma XML Syntax: (XML Line 018937:73): element "fn" not allowed here; expected the element end-tag, text or element "add", "bookmark", "br", "concept", "del", "em", "eref", "erefstack", "hr", "image", "index", "index-xref", "keyword", "link", "pagebreak", "ruby", "smallcap", "span", "stem", "strike", "strong", "sub", "sup", "svg", "tt", "underline" or "xref"
Metanorma XML Syntax: (XML Line 019026:16): element "itu-standard" incomplete; missing required element "bibliography"
java -Xss5m -Xmx2048m -Djava.awt.headless=true -Dapple.awt.UIElement=true -Duser.home=/Users/mulgogi -jar ~/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/mn2pdf-1.66/lib/../bin/mn2pdf.jar --xml-file "~/src/ituob/service-publications-docs/1060-M.1400/T-SP-M.1400-2014-E.presentation.xml" --xsl-file "~/.asdf/installs/ruby/3.1.2/lib/ruby/gems/3.1.0/gems/metanorma-itu-2.2.7/lib/isodoc/itu/itu.service-publication.xsl" --pdf-file "~/src/ituob/service-publications-docs/1060-M.1400/T-SP-M.1400-2014-E.pdf" --syntax-highlight  --font-manifest "/var/folders/2t/xmdrn2sd2lv2w49dv0zw9_q00000gp/T/fontist_locations20230324-76082-3jpnw.yml"

[ronaldtse]

Actually it is returning, it just took a long time to complete. Thank you @Intelligent2013 .