Lexer options don't get passed to lexer

[alexanderweiss]

Lexer options, defined in config.rb don't seem to get passed to passed to the lexer:

activate :syntax, lexer_options: { prompt: '$,#,;' }

The above doesn't actually override the default settings of the console lexer.

I've been trying to figure out what was going on and it seems like the lexer_options need to be passed as the third argument to find_fancy, rather than to lexer.lex (in https://github.com/middleman/middleman-syntax/blob/master/lib/middleman-syntax/highlighter.rb):

def self.highlight(code, language=nil, opts={})
  highlighter_options = options.to_h.merge(opts)
  formatter = Middleman::Syntax::Formatters::HTML.new(highlighter_options)
  lexer_options = highlighter_options.delete(:lexer_options)
  
  lexer = Rouge::Lexer.find_fancy(language, code, lexer_options) || Rouge::Lexers::PlainText
  
  highlighter_options[:css_class] = [ highlighter_options[:css_class], lexer.tag ].join(' ')
  
  formatter.format(lexer.lex(code, lexer_options))
end

[markets]

Hey @alexanderweiss 👋🏼

Would you like to send a patch? Better late than never 💪🏼.

I think it makes sense what you mentioned in the comment above ☝🏼.

[markets]

Hello @alexanderweiss 👋🏼

Sorry for the ping! Just wanted to ask if you can help us to fix this one 🙏🏼 If you can send a patch, that would be amazing, otherwise if you can point me to the right docs I'll try to submit a patch (and a test) by myself. Would like to better understand what this lexer_options: { prompt: '$,#,;' } is supposed to do, I've never used raw Rouge and I'm not familiar with some concepts as the lexer.

ℹ️ I started some months ago to collaborate in the Middleman ecosystem, so I'm trying to catch up with old issues/PRs.

[alexanderweiss]

Hey! Sorry for the delay. Haven't been using middleman in a while. But rouge's lexers are basically the language parsers that turn some input into a common format. Each individual lexer/parser accepts language-specific options. See e.g. the console lexer. These are passed to the lex function