Subreading acronym expansion does not show in main reading

[snomos]

Cf the following:

"<CD-spiller>"
	"spiller" N Sem/Obj-el Sg Nom <W:0.0> <cohort-with-dynamic-compound> @HNOUN #1->0 "cd-spiller"phon
		"CD" N Prop Sem/Org ACR Dyn Cmp/Hyph Cmp <W:0.0> #1->0 "cee dee -spiller"phon
		"CD" N Prop Sem/Org ACR Dyn Cmp/Hyph Cmp <W:0.0> #1->0 "cee dee -spiller"phon
	"spiller" N Sem/Obj-el Sg Nom <W:0.0> <cohort-with-dynamic-compound> @HNOUN #1->0 "cd-spiller"phon
		"CD" v1 N Sem/Obj-el ACR Cmp/Hyph Cmp <W:0.0> #1->0 "cee dee -spiller"phon
		"CD" v1 N Sem/Obj-el ACR Cmp/Hyph Cmp <W:0.0> #1->0 "cee dee -spiller"phon
:\n

Expected output for the main reading is: "cee dee -spiller"phon as well, just as in the subreadings.

@bbqsrc this likely affects your code too.

[flammie]

is it just that CD doesn't expand itself? Debugging says it drops at normaliser automaton:

$ echo CD-spiller | hfst-tokenise -g '/home/flammie/github/giellalt/lang-sme/tools/tts/tokeniser-tts-cggt-desc.pmhfst'  | divvun-blanktag '/home/flammie/github/giellalt/lang-sme/tools/tts/analyser-gt-whitespace.hfst'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/generated-remove-lexicalised-compounds.bin'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/valency.bin'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/mwe-dis.bin'  | cg-mwesplit  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/disambiguator.bin'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/functions.bin'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/dependency.bin'  | ./src/divvun-normaliser -a  '/home/flammie/github/giellalt/lang-sme/tools/tts/analyser-gt-norm.hfstol' -g  '/home/flammie/github/giellalt/lang-sme/tools/tts/generator-tts-gt-norm.hfstol' -n  '/home/flammie/github/giellalt/lang-sme/tools/tts/transcriptor-gt-desc.hfstol' -t ABBR -t ACR -t Arab -t Sem/Year -t Sem/Date -t Sem/Time-clock -t Symbol -v --debug

New surface form: CD-spiller
"<CD-spiller>"
Using lemma: spiller
No expansion tags in
	"spiller" N Sem/Obj-el Sg Nom <W:0.0> <cohort-with-dynamic-compound> @HNOUN #1->0
Expanding because of ACR
Using lemma: CD
1. looking up normaliser
Normaliser results empty.
		"CD" N Prop Sem/Org ACR Dyn Cmp/Hyph Cmp <W:0.0> #1->0
no usable results, printing source:
		"CD" N Prop Sem/Org ACR Dyn Cmp/Hyph Cmp <W:0.0> #1->0
Using lemma: spiller
No expansion tags in
	"spiller" N Sem/Obj-el Sg Nom <W:0.0> <cohort-with-dynamic-compound> @HNOUN #1->0
Expanding because of ACR
Using lemma: CD
1. looking up normaliser
Normaliser results empty.
		"CD" v1 N Sem/Obj-el ACR Cmp/Hyph Cmp <W:0.0> #1->0
no usable results, printing source:
		"CD" v1 N Sem/Obj-el ACR Cmp/Hyph Cmp <W:0.0> #1->0

same for just CD really:

$  echo CD | hfst-tokenise -g '/home/flammie/github/giellalt/lang-sme/tools/tts/tokeniser-tts-cggt-desc.pmhfst'  | divvun-blanktag '/home/flammie/github/giellalt/lang-sme/tools/tts/analyser-gt-whitespace.hfst'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/generated-remove-lexicalised-compounds.bin'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/valency.bin'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/mwe-dis.bin'  | cg-mwesplit  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/disambiguator.bin'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/functions.bin'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/dependency.bin'  | ./src/divvun-normaliser -a  '/home/flammie/github/giellalt/lang-sme/tools/tts/analyser-gt-norm.hfstol' -g  '/home/flammie/github/giellalt/lang-sme/tools/tts/generator-tts-gt-norm.hfstol' -n  '/home/flammie/github/giellalt/lang-sme/tools/tts/transcriptor-gt-desc.hfstol' -t ABBR -t ACR -t Arab -t Sem/Year -t Sem/Date -t Sem/Time-clock -t Symbol -v --debug

New surface form: CD
"<CD>"
Expanding because of ACR
Using lemma: CD
1. looking up normaliser
Normaliser results empty.
	"CD" v1 N Sem/Obj-el ACR Sg Nom <W:0.0> @HNOUN #1->0
no usable results, printing source:
	"CD" v1 N Sem/Obj-el ACR Sg Nom <W:0.0> @HNOUN #1->0
Expanding because of ACR
Using lemma: CD
1. looking up normaliser
Normaliser results empty.
	"CD" v2 N Sem/Obj-el ACR Sg Nom <W:0.0> @HNOUN #1->0
no usable results, printing source:
	"CD" v2 N Sem/Obj-el ACR Sg Nom <W:0.0> @HNOUN #1->0

[flammie]

nvm, I missed the phon step from previous debug. Phon does line-by-line processing, it'll only see CD with ACR once it's done with the main reading:

echo CD-spiller | hfst-tokenise -g '/home/flammie/github/giellalt/lang-sme/tools/tts/tokeniser-tts-cggt-desc.pmhfst'  | divvun-blanktag '/home/flammie/github/giellalt/lang-sme/tools/tts/analyser-gt-whitespace.hfst'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/generated-remove-lexicalised-compounds.bin'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/valency.bin'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/mwe-dis.bin'  | cg-mwesplit  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/disambiguator.bin'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/functions.bin'  | vislcg3 -g '/home/flammie/github/giellalt/lang-sme/tools/tts/dependency.bin'  | ./src/divvun-normaliser -a  '/home/flammie/github/giellalt/lang-sme/tools/tts/analyser-gt-norm.hfstol' -g  '/home/flammie/github/giellalt/lang-sme/tools/tts/generator-tts-gt-norm.hfstol' -n  '/home/flammie/github/giellalt/lang-sme/tools/tts/transcriptor-gt-desc.hfstol' -t ABBR -t ACR -t Arab -t Sem/Year -t Sem/Date -t Sem/Time-clock -t Symbol  | ./src/divvun-phon -p '/home/flammie/github/giellalt/lang-sme/tools/tts/text2phontext.hfstol' -a ACR=/home/flammie/github/giellalt/lang-sme/tools/tts/acro2text.hfstol -v --trace
Tracing (auto-verbose)...
Being verbose.
Text2ipa set to: /home/flammie/github/giellalt/lang-sme/tools/tts/text2phontext.hfstol
Reading: /home/flammie/github/giellalt/lang-sme/tools/tts/text2phontext.hfstol
Alternate text2ipa for tag 'ACR' set to /home/flammie/github/giellalt/lang-sme/tools/tts/acro2text.hfstol
Reading /home/flammie/github/giellalt/lang-sme/tools/tts/acro2text.hfstol for tag ACR
New surface form: CD-spiller
"<CD-spiller>"
Using surf: CD-spiller
looking up text2ipa: CD-spiller
	"spiller" N Sem/Obj-el Sg Nom <W:0.0> <cohort-with-dynamic-compound> @HNOUN #1->0 "cd-spiller"phon DIVVUN-PHON:TEXT2IPA
Using surf: CD-spiller
Using ACR
looking up text2ipa: CD-spiller
		"CD" N Prop Sem/Org ACR Dyn Cmp/Hyph Cmp <W:0.0> #1->0 "cee dee -spiller"phon DIVVUN-PHON:ALT:ACR
Using surf: CD-spiller
Using ACR
looking up text2ipa: CD-spiller
		"CD" N Prop Sem/Org ACR Dyn Cmp/Hyph Cmp <W:0.0> #1->0 "cee dee -spiller"phon DIVVUN-PHON:ALT:ACR
Using surf: CD-spiller
looking up text2ipa: CD-spiller
	"spiller" N Sem/Obj-el Sg Nom <W:0.0> <cohort-with-dynamic-compound> @HNOUN #1->0 "cd-spiller"phon DIVVUN-PHON:TEXT2IPA
Using surf: CD-spiller
Using ACR
looking up text2ipa: CD-spiller
		"CD" v1 N Sem/Obj-el ACR Cmp/Hyph Cmp <W:0.0> #1->0 "cee dee -spiller"phon DIVVUN-PHON:ALT:ACR
Using surf: CD-spiller
Using ACR
looking up text2ipa: CD-spiller
		"CD" v1 N Sem/Obj-el ACR Cmp/Hyph Cmp <W:0.0> #1->0 "cee dee -spiller"phon DIVVUN-PHON:ALT:ACR

[flammie]

I'm kind of thinking it should be normalisation to go from CD to seedee but maybe we went against that because it's not a dictionary word? cause seedee should also go through phonemiser too

[snomos]

You are right that CD should go through the normaliser first. That does not solve the general case, though.

[flammie]

Ah true. Probably both handlers should wait for handling before subreadings are all read and just use one processing run for whole.

[snomos]

nvm, I missed the phon step from previous debug. Phon does line-by-line processing, it'll only see CD with ACR once it's done with the main reading:

Yeah, it needs to read the whole cohort as one structure, and then process the sub-readings bottom up, so to speak, and collect all sub-reading phon strings in the top/main reading phon string.

[snomos]

Maybe @albbas could have a look? This is high priority, it blocks a number of text normalisation cases.