Guidance for the builder feature

[andrebraga]

I'm trying to understand how do the macros "know" what can be expanded statically and what needs interpolation. Things are working great since your recent commits with the trivial builder, but ideally there should be a way of executing the result of a builder and work on that, not unlike executing a function that returns HTMLElement and work on that, instead of assuming interpolation is always needed. I am hoping that "fixing" the latter will also fix the former; currently

let plop = {
  tr(td())
}()

#html(plop) as String

expands to

"" + String(describing: plop) + ""

but does it have to?

What's the "magic" that transforms #html(tr(td())) as String into "<tr><td></td></tr>"? You describe this as "promotion" in some places and I suspect it has to do with the initialiser that takes macro context arguments, but I haven't really figured this out. Pointers, tips, guidance and hand-holding are welcome :)

[RandomHashTags]

The "magic" that transforms the input into a compiled result is parseArguments. More specifically parseInnerHTML and parse_literal_value. There are a few other things needed to understand what is happening and why.

Swift Macros

While Swift Macros are powerful, in their current form they are very limited. When expanding a macro, you get a syntax representation of the parameters, not an exact 1-for-1 representation. You need to manually convert them into a system that best fits your usage. For my usage (Swift HTMLKit) I basically identify whether each element in the macro's innerHTML is an HTMLElement, and if so, keep parsing via recursion, otherwise it is treated as interpolation.

For implicit data we only get the declaration representation of the referenced data, which is only the name of the variable. We literally have no way in knowing what the data is, just that it is a declaration reference. A lot of tedious work would need to be created to even check if certain inputs can be promoted or not, which is why I chose to just use String(describing:) for now.

Promotion

The reason why I added a system of promoting certain interpolation to an equivalent StaticString is due to runtime interpolation costs a lot of performance. Because we don't get a 1-for-1 representation of the parameters, a lot of tedious work has to be done to check whether or not it can be promoted. It is literally impossible at the moment to check whether or not implicit data can be promoted due to the current state of Swift Macros.

A recent addition to swift-syntax is SwiftLexicalLookup, which may be utilized to be able to promote referenced data in the same scope, but it is currently experimental and not even in an official swift-syntax release yet. I do have my eye on it.

I have added an experimental lookupFiles parameter to the html macro to promote references of static content (variables, declarations, enum values, etc), but even then it won't work on macOS due to macros being sandboxed (denied system file access).

Explicit vs Implicit data

Explicit data is any data that you explicitly type. Implicit data is a reference to any data.

// Explicit
let _:String = #html("<div></div>") // no promotion necessary

// Implicit
let string:String = "<div></div>"
let _:String = #html(string) // currently impossible to promote

// Promotion
let _:String = #html("\("<div></div>")") // we can promote it because it is explicitly typed

[RandomHashTags]

I will be refactoring how the content is parsed and handled by the macro expansion, and will be utilizing SwiftLexicalLookup when it is available (a PR was opened yesterday to make it officially usable in swift-syntax). I have a few more refactors planned for the library to better improve usability (native result builders), maintainability, build performance, documentation, adoption of newer Swift Language features, and compiler warnings.

[andrebraga]

Hey, how's this going? The result builders are the most wanted feature to be honest, my homegrown version is not super amazing I think... certainly feels like I am working around the HTMLKit rather than with it. Not to mention how the source formatting tools are noticeably tuned to trailing closure syntax instead of parentheses (that are always closing 1 extra level to the right...)

[andrebraga]

(The PR was recently merged by the way.)

[RandomHashTags]

I've been working with some result building and closures to replace the existing functionality but have been running into crashes due to bad pointer dereferences, which has been blocking further development. I'll test if the main branch of swift-syntax has fixed them.

[RandomHashTags]

Latest commit adds support to create elements' inner html using a trailing closure. I planned to add support for result builders after moving the html elements to their own files, but I'll probably add them to the existing system (I'll need more time to develop the auto port of elements to their own file).

I'll try to test improvements to the library by utilizing SwiftLexicalLookup in the coming days.

[andrebraga]

I already tested the potential appearance and this will work so great with result builders! Great potential there!