this context of mdast-util-from-markdown extension enter/exit functions

[vadistic]

Hello, great work, thank you for this project!

I'm migrating some of our custom syntax into mdast/micromark extensions and I cannot quite reverse engineer the api of this provided to enter/exit fns in records of MdastExtension.

There are quite a lot of utils in source (not a bad thing!) but that make is a bit hard to understand at glance and I would be really thankful for a few brief pointers :)

As far as I see those function have following interface.

function enterSomeStatement(this: MdastContext, token: Token)
function exitSomeStatement(this: MdastContext, token: Token)

where MdastContext is

interface MdastContext {
  // render node
  enter: (node: Node, token: Token) => void;
  // stop rendering node
  exit: (token: Token) => void;
  // convert micromark token to string
  sliceSerialize: (token: Token) => string;
  // ??? what it does, is the interface correct?
  getData: (key: string) => any;
  // ??? what it does, is the interface correct?
  setData: (key: string, value: any) => void;
  // ??? what it does?
  resume: () => void;
  // current ast stack
  stack: Node[];
  // current token stack
  tokenStack: Token[];
};

Q1: Could you consider adding those interface to public types? Current typing of (token: Token) => void is not quite enough.

Q2: Is my interface correct? What's the resume() does?

Q3: What the state get/set data should be used for?

Q4 (extra): is there a way to access other (child) micromark tokens inside those functions? There are no in token stack, and I cannot add them via another .enter() because those have no nodes of it's own. The reason for this is that I'm not serializing micromark tree 1:1 into unist ast - I need to sliceSerialize specific parts of it to create ast node. If you saw something like this in some existing plugins/extensions - please just link to it, no need to write-up :)

Thank you!

[wooorm]

Hi there! Jakub!

Q1: Could you consider adding those interface to public types? Current typing of (token: Token) => void is not quite enough.

Working on it. I’m in the process of updating the whole collective (unifiedjs/unified#121), and yesterday pushed types to micromark. There are a couple of other issues going into that release.

I first built CommonMark support only. Then came all the extensions. Those all affected the core too. Now I’m going through everything again, making it stable, and documenting it.

Q2: Is my interface correct? What's the resume() does?

Not completely, for one the return types are different than the code: https://github.com/syntax-tree/mdast-util-from-markdown/blob/main/lib/index.js.

Q3: What the state get/set data should be used for?

An example is how micromark-extension-directive collects attributes

Q4

Could you explain in a bit more detail why and what you are doing?

[vadistic]

Hey, thanks for the really fast reply.

Q3

Ok, I see wrong return type for resume() & buffer() is missing.

Q4

At the moment I'm writing something like tsx (typechecking via unified plugins) + mdx (components) but for the handlebars syntax in markdown (that's the requirement 🤷 ).

I've sliced source nicely with micromark lexer into tokens, but now I needed to aggregate multiple micromark token data to build one custom handlebars ast node.

I think your link to directive answer my issue. I thought that each enter/exit should yield unist ast node (which is a problem because I'm not mapping 1:1 micromark<=> unist), but I see I can just collect some data (like directive attributes). Thanks!

One last question: what buffer() does?

function buffer() {    
   this.stack.push({type: 'fragment', children: []}) 
}

We're pushing "fragment" node into the stack, why?

Thanks again!

[vadistic]

Working on it

Glad to hear it! Great!

FYI: This is unrelated but if you're fixing micromark typings right now...

Currently, the generated type for micromark extension have tiny issue.

export interface MicromarkSyntaxExtension {
  document?: Record<CodeAsKey, Construct | Construct[]>;
  ...
}

Because CodeAsKey= number | 'null' this always require null field in the hooks map, which is incorrect.

It should be like

export interface MicromarkSyntaxExtension {
  document?: Partial<Record<CodeAsKey, Construct | Construct[]>>;
  ...
}

[wooorm]

buffer and resume are explained better in micromark: https://github.com/micromark/micromark/blob/62aa005d0bb60bdd34afcbc62697d89fc28d0582/lib/compile/html.js#L164-L171.

---

Yeah, micromark’s published types have some bugs. These ones added yesterday are better, and today I’m setting strictNullChecks, which if yesterdays types didn’t solve your issue, should solve it!