Annotation processing improvements

[Twisol]

Hey all. I've been cooking up an improved architecture for the annotation processor, and I wanted to poll folks about their experience interacting with the current processor. (For those unaware, it lives under merlin.processor, with annotations in merlin.framework.annotations.)

I know a couple of you on the Aerie team (I shan't name names 😉) have delved into the processor to add new features. I'd like to learn what your experience has been!

Background

What currently exists is actually the second revision of the processor. Back when we revamped it for the first time, I went through a series of concepts for what a better architecture could look like, but floundered because the processor was too small for me to really get clear user stories from. I was also the only one who'd really spent time in the system, so I only had my own opinions to draw on. After floundering for a bit, we organized a basic data model in merlin.processor.metamodel, and built the rest of the processor ad-hoc on top.

I think it's fair to say that, these days, the processor is struggling to keep any sense of maintainability or comprehensibility. Its capabilities have expanded pretty considerably since then, and the logic for each new feature has been threaded through the existing code traversals and output templates without much cohesion. The spaghetti is finally cooked 😃

As processors in general go, I think Merlin's is not actually too complex. But it feels to me like you get mired in all kinds of fussy details which make it hard to get either a bird's eye view or a good low-level understanding. It's also incredibly hard to test the processor, since you have to feed in a whole mission model (via Gradle!!) and manually inspect the output. You can't even set breakpoints and use the debugger (unless you know how to run javac from a Java program, which I sure didn't until a month ago). I want to make these things better!

Questions

I've been living and breathing javac's annotation processing system for the last month (I even reported a bug!), and I think I have a better concept of what a good processing architecture should look like. I've been developing a prototype on this concept, and things are slowly starting to coalesce. But this time, I'm (hopefully) not the only one with opinions!

• What problems, if any, did you encounter when working on the processor?
• Was it easy/difficult to read the existing code?
• Was it clear/unclear where you needed to make changes to achieve the result you wanted?
• Was there anything in particular that got in your way? (Anything that was a particular help?)
• Was there anything missing that you wish you had? (Anything missing that you're glad wasn't there?)

Any and all related thoughts are welcome. Think of me as a product owner trying to figure out how to make a better experience for my customers (the developers of the processor, here!).

[pcrosemurgy]

May be a bit of Stockholm syndrome but as long as the processing and generation are separated I'm pretty happy :) That's a pretty low bar though. I usually don't have too much trouble figuring out what parts process/generate what I'm looking for, but I completely agree about the lack of testability – it would be especially nice to have a set of regression tests so we can track generation changes down to the char.

[mattdailis]

From the "Future Work" section of #281 :

having true regression tests for our Java codegen would be awesome.