You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #723 we implemented contextual keywords but at the cost of always allocating the input substring that we attempt to promote into a contextual keyword. This is a remnant of how we implemented the lexer via macros and implemented the keyword promotion using a similar mechanism.
Problem
Currently, it's impossible to only borrow the string from the same time because we mutably borrow ParserContext to advance its inner position in the lexer macros.
Possible solution
We're only collecting the identifier to see if it was fully recognized by the keyword trie (exactly matching its length), so it might be possible to just collect the length rather than allocating a string and passing it down to the macros.
Alternatively, we could rework the promotion mechanism not to use the keyword trie and combine the identifier+keyword lexer to return whether the identifier is directly promotable (contextual), reserved or a legal identifier.
The text was updated successfully, but these errors were encountered:
In #723 we implemented contextual keywords but at the cost of always allocating the input substring that we attempt to promote into a contextual keyword. This is a remnant of how we implemented the lexer via macros and implemented the keyword promotion using a similar mechanism.
Problem
Currently, it's impossible to only borrow the string from the same time because we mutably borrow
ParserContext
to advance its innerposition
in the lexer macros.Possible solution
We're only collecting the identifier to see if it was fully recognized by the keyword trie (exactly matching its length), so it might be possible to just collect the length rather than allocating a string and passing it down to the macros.
Alternatively, we could rework the promotion mechanism not to use the keyword trie and combine the identifier+keyword lexer to return whether the identifier is directly promotable (contextual), reserved or a legal identifier.
The text was updated successfully, but these errors were encountered: