feat: overhaul drawing

[Julusian]

Starting point for the button drawing overhaul #3293

Let me introduce the worst ui you have ever seen:

But it works, and is a starting point

cc @dnmeid

Required follow ups before this can be released:

• Deal with any TODO-layered

[Julusian]

The UI is making a bit more sense now. Leaving it at that for today

[dnmeid]

Wow, you're fast. I wanted to start that stuff for so long but always had an excuse for not doing it. I'll try to add some pieces from time to time.
I had a brief look at it, especially the interface.

First of all I think the way we are looking at the concept now, mabe we are using the term "layer" in a wrong way. Usually in a drawing app you are using objects or elements and you can arrange them in layers (and groups). If we are going to have only one element per layer, we could technically still call it a layer (like e.g. openGL does), but I'd recommend to call it an element instead so we won't have a hard naming time if we want to introduce real layers later.

I suggest to drop the "canvas" and replace it with a "matte". A matte is an element that always fills the whole area of the button (no adjustable size) and draws a color or maybe later a gradient.
All elements should be optional. Without any element you will get a transparent image. On a Streamdeck the base color will be black, so if you want to have a different background color, you need to add a matte. I envision that this graphics will also be combined later with different backgrounds, e.g. use the graphics of one control as an element in a diferent control. Or webbuttons or satellite may want to have some different overall background...

I would make the topbar a separate element. So the user is free to add it or not, or reposition it in a later version or maybe we add some styling options in the future. I think "statusbar" would be the better name then. Later we can separate the yellow border from it, for now as long as there is no position or size control all status decorations can stay in one element.

For the image element we should take care from the start that there will be more possible formats than PNG. Just for the naming.

For the UI I suggest to show the topmost layer on the top and the bottom layer at the end of the list. I'm not sure if in the definition we should have the same order, I tend to say technically it makes more sense to have the lowest element first in the array because it is drawn first. So only the presentation would change.
The add buttons should be above the list.

[Julusian]

Wow, you're fast. I wanted to start that stuff for so long but always had an excuse for not doing it.

A lot of it is very early at this point, I did a draft of the renderer using the existing Image class and didn't test it much until the ui was functional. And at this stage the functionality of what each 'layer' can do is limited to what we currently support.

And thanks for this input, this is exactly what I was hoping would happen. I find it much easier to make something quick and crude and refine it being able to talk about concrete things rather than large concepts/ideas.

First of all I think the way we are looking at the concept now, mabe we are using the term "layer" in a wrong way. Usually in a drawing app you are using objects or elements and you can arrange them in layers (and groups). If we are going to have only one element per layer, we could technically still call it a layer (like e.g. openGL does), but I'd recommend to call it an element instead so we won't have a hard naming time if we want to introduce real layers later.

Good point. Yeah I am happy to rename it all to elements. I would be tempted to refer to the 'layer group' as a layer then (once that exists)

I would make the topbar a separate element. So the user is free to add it or not, or reposition it in a later version or maybe we add some styling options in the future. I think "statusbar" would be the better name then. Later we can separate the yellow border from it, for now as long as there is no position or size control all status decorations can stay in one element.

I'm not sure about this. While I can see the appeal of what you are proposing, whether the top bar is shown affects the vertical position/height of other drawing. So this would mean we either need to drop this behaviour, or accept that any layer can affect some basic properties about the canvas, rather than just the first 'layer'.
If if its own element, then should the user be able to add it multiple times?

I suggest to drop the "canvas" and replace it with a "matte". A matte is an element that always fills the whole area of the button (no adjustable size) and draws a color or maybe later a gradient.
All elements should be optional. Without any element you will get a transparent image.

This kind of ties into my thoughts about the topbar. I am using the canvas element as a place to dump any settings that affect how this is drawn. I think that this is something that will be useful.

I am open to a matte type, it will probably make more sense once we can do gradiants, to limit where we clutter with these properties. But again it feels like one of those types where the only valid place to use it is as the second layer, so its just extra steps?
I suppose I am not thinking very forward here, but it will be much easier to split a layer out later than try to merge them.

For the image element we should take care from the start that there will be more possible formats than PNG. Just for the naming.

Yep, I made sure to name the it generically. It didn't accept a jpeg when I tried it, but I didn't look into it. Might be a stray guard somewhere checking for PNG, the actual drawing won't care since we dropped pngjs and let the canvas handle parsing image buffers.

For the UI I suggest to show the topmost layer on the top and the bottom layer at the end of the list. I'm not sure if in the definition we should have the same order, I tend to say technically it makes more sense to have the lowest element first in the array because it is drawn first. So only the presentation would change.
The add buttons should be above the list.

Yeah, this part of the ui has had no attention yet, just dumped out some html to be able to prove it worked.

[Julusian]

Looking much more like a proper ui now

[dnmeid]

I'm not sure about this. While I can see the appeal of what you are proposing, whether the top bar is shown affects the vertical position/height of other drawing. So this would mean we either need to drop this behaviour, or accept that any layer can affect some basic properties about the canvas, rather than just the first 'layer'.

At the moment there is no possibility of user positioning or sizing an element. So I guess the current system reflects the behavior of the old buttons, that means a text or image element is sized according to if the topbar is shown or not. So in this regard it wouldn't make any difference if the topbar is an elemet of its own.
At the time we introduce position/size we will have to deal with that anyway, regardless if the topbar is part of a canvas or separate.
I would drop the show/hide topbar automation and replace it by some style presets. The style preset would be a direct replacement for the selection of the button type "page up", "page down", "regular". We can offer premade sets of elements that reflect the same button look and behavior. A "page up" would add the arrow or plus as an element and a text element with the page name and a text element with the page number. Instead of one simple button we can offer a preset for a button with the topbar and a preset for a button without a topbar. The amount of clicks stays the same, no matter what you choose. Also this is compatible with current button preset definitions, where modules can set this property.
When you decide later to change the style, you manually have to adjust the elements. So changing this would be slightly more effort, but the overall process of button styling will consume more time, e.g. for adding an image you first will need to add the image element and then select the image.

Users that really want to have a show/hide topbar automation can use a custom variable and make the style dependent on the variable, e.g. text element y-position = $(custom:showtopbar) ? 16 : 0, topbar opacity = $(custom:showtopbar) ? 1.0 : 0.0

BTW: having talked about style presets. Actually this can be just a preset offered by the internal module and the "create button" dropdown can remain working as a way of assigning these internal presets to a control.

If if its own element, then should the user be able to add it multiple times?

Yes, why not. I would try to make as less exceptions as possible. It may not make sense to add five topbars, but I'd allow it. I'd say all elements are optional and all can be added as often as you like.
Maybe when we have groups/layers/containers/divs (whatever we will call 'em) a user wants to use these as different graphical states and he wants to have three groups containing a topbar and one group without a topbar.

[Julusian]

At the time we introduce position/size we will have to deal with that anyway, regardless if the topbar is part of a canvas or separate.

I intend to do this before calling this overhaul ready for general usage, so I think we should think about it as if it was already implemented.

Users that really want to have a show/hide topbar automation can use a custom variable and make the style dependent on the variable, e.g. text element y-position = $(custom:showtopbar) ? 16 : 0, topbar opacity = $(custom:showtopbar) ? 1.0 : 0.0

Does this mean that we won't support auto vertical centering the text anymore? Unless all the drawing is aware that there is a topbar, then numbers will need to be manually adjusted by a pixel amount (and we want to discourage using pixel values) using numbers that users are just expected to guess?

Sure, we can make the default button be pre-configured, but then users will need to think about this offset for any change they make. Changing the text alignment to bottom, now they need to zero that value. Or to top, and they need to put it to 16. Or to middle, and to 8.
Add a new image layer, and set the value yourself.

Having to compensate for this would be incredibly frustrating, and a step backwards

The style preset would be a direct replacement for the selection of the button type "page up", "page down", "regular".

I'm very open to this idea as a follow up, I'm not sure about whether it should for the top bar setting or not.

[dnmeid]

Does this mean that we won't support auto vertical centering the text anymore?

No, I'd like to continue that. The text element has an overall position and dimensions and the alignment of the text is done within these boundaries.

Unless all the drawing is aware that there is a topbar, then numbers will need to be manually adjusted by a pixel amount (and we want to discourage using pixel values)

No, the preset would give you a button with the correct numbers in it. Right, we should have this expressed in percent instead of pixels, but the logic is the same. The topbar fills 18%, then there is 82% remaining for the other elements.
Maybe I should add another thought I have on the group thing. It should work more like a <div> and have its own position and size. Maybe we should call it a "container"? The position/size of the elements within that container should be relative to the container. That means when our preset has a container for the part not occupied by the topbar, the user can easily add elements to it and they will all automaticall fit 100% of the container.
As far as I know typical user workflows most of them do not often change the topbar visibility. They adjust their preference and then maybe they'll have a few special buttons with different topbar setting.

using numbers that users are just expected to guess?

Well, I think that every element has some initial values when you add it. When the statusbar is a single element without other decorations, I think it will have X:0, Y:0, W:100%, H:18% by default. So you have a hint what the numbers are.

Sure, we can make the default button be pre-configured, but then users will need to think about this offset for any change they make. Changing the text alignment to bottom, now they need to zero that value. Or to top, and they need to put it to 16. Or to middle, and to 8.

I don't get this point. The text alignment is not related to the outer dimensions or position of the element.

[Julusian]

Every field can now accept expressions. I reworked the data structure to be able to represent this more cleanly.

In this fun example, the button is redrawing every second with the font size constantly changing

I think my next step is going to be refactoring the drawing and working towards the button preview in this editor being rendered client side. That way it will be easier for users to play with different aspect ratios, and for us to draw some additional lines to indicate boundaries and things

[Julusian]

Maybe I should add another thought I have on the group thing. It should work more like a

and have its own position and size. Maybe we should call it a "container"? The position/size of the elements within that container should be relative to the container.

Right yeah, that makes more sense now, that would work and be useful anyway. But I would worry that forcing the user to make sure that every element they create ends up in this container might be an annoyance.
Even if we add buttons to be able to add elements directly inside containers, it will be very easy to click the wrong one and then wonder later why the text on some buttons isn't vertically centered.

But my technical concerns would go away with containers, so this really comes down to do we want the topbar to be a fixed/rigid part of the button drawing process, or do we want it to be an element that the user is free to place and move around freely, and to almost always force use of a container.

I don't get this point. The text alignment is not related to the outer dimensions or position of the element.

With containers this is void; or I suppose a proper way of defining the bounds of some text would negate my thoughts here too

[dnmeid]

this really comes down to do we want the topbar to be a fixed/rigid part of the button drawing process, or do we want it to be an element that the user is free to place and move around

Correct, so the options are:

• leaving the topbar (or other decorations) something that the button does when switched on and elements can ever only use the remaining space
• offering always the whole button area to elements and the topbar is an user adjustable element itself

First of all I'd not say that the topbar as a separate element would force the use of a container. A container will make it easier for other elements to stay within the area not occupied by the topbar, but even without a container you can do it. Maybe you put a matte in the background, a text element on top of that and the topbar on the very top. You only have to tweak the position or size of the text to your liking and that doesn't necessarily needs calculating pixels or percent, you can just shift it until it looks good.
Also I would say that the possibilities of a container compensate for the overhead effort of using it.

I think the biggest advantage of a fixed topbar is that it remains possible to switch the topbar on and off and the remaining button just rescales automatically (if the elements used percent).

On the other hand the biggest advantage of a separate topbar is the styling flexibility. You can position the bar at the top or at the bottom or somewhere else. You can have some containers with a topbar and some without and switch between them programatically (e.g. per step). You can resize the topbar to your liking. Currently it is 18% of the height, when I think of e.g. a 640x400px display, I don't think that a 18% topbar will be what you want. Maybe you want it to be only 5% high and only 20% wide.

We could of course offer both possibilities, the fixed topbar and a statusbar element at the same time. I just think then we are carrying unnecessary code and UI stuff over, because actually only one system is needed. Maybe it's an idea to have both possibilities for a testing period and then we see what we like more or how some test users react before releasing it.

[Julusian]

First of all I'd not say that the topbar as a separate element would force the use of a container. A container will make it easier for other elements to stay within the area not occupied by the topbar, but even without a container you can do it.

Yes, that is all true, but I would probably recommend to do to to make things easier.

I am thinking that the default layers on a newly created button should be the 'canvas' layer (if that remains), a 'matte' layer (if that exists), a 'text' layer, and this status bar layer. The text layer might be in a container, to encourage users to add their stuff there for easier positioning, if not the text layer will need the position adjustments.

The probably most common change to make to this layer structure will be adding a background image. Which means adding a layer, moving it into the container or adjusting its height and moving it below the statusbar.

Maybe the 'solution' to my concern is to also create that image layer by default. Then to match the functionality of today it wont require fiddling with layer order, or positions.

You can resize the topbar to your liking. Currently it is 18% of the height, when I think of e.g. a 640x400px display, I don't think that a 18% topbar will be what you want. Maybe you want it to be only 5% high and only 20% wide.

I hadn't considered displays of this kind of size. I was expecting this to be drawn at a fixed height still (which yes, adds its own problems of mixing percent and px)

Maybe it's an idea to have both possibilities for a testing period and then we see what we like more or how some test users react before releasing it.

I can get behind this idea. Supporting the old flow is very little code, seeing as it is used for the old buttons and simply adjusts what the min/max x/y values are.
And until all this code lands in beta I will not be taking any care to not break the data structure.

Regarding the matte type, since containers, I think this is an appropriate way to go as this will start to go towards being able to draw regions in different colours.
Should it be 'matte' or 'solid' or 'rectangle'? This is the start of different drawing primitive shapes, so we should probably name it in a way that will make sense once there are other shapes

[dnmeid]

Should it be 'matte' or 'solid' or 'rectangle'?

Let's collect some ideas for elements to see how tey can build a useful set with distinct functionality.
Top priority

• Container: Can hold other elements. Has X,Y,W,H. Maybe rotation? Child elements use the extends as their 0%-100% range, but can also draw outside of the container area. E.g. container x is at 5px and child x is at -5px, actual x will be at 0.
• Matte: draws a color or a gradient (maybe gradient as a separate element?). Has color. Does always draw to the extends of the parent element
• Box: Has XYWH, rotation, fill color/gradient, border width, border color, border alignment, corner radius?. Maybe we make the box the container? Easier sometimes but more unused options most often
• Text: Has text, font size, font, horizontal alignment, vertical alignment, color, style, line height. Maybe later animation like scroll? Maybe enable interpretion dropdown for plain|md|html, maybe formatting option like in excel.
• Image: Has XYWH, rotation, image, alignment horizontal, alignment vertical, crop/fit. Can use different bitmap and vector image formats. Maybe gif support? (usually super ugly but highly requested by users). Maybe Images are taken as a reference from a image storage. Drag and drop from the OS should be possible either to the image option of an existing element or directly in the layer stack.
• Statusfield: Has XYWH, rotation, color, border color. Inner elements will adopt to size/aspect of element.

Medium priority

• Line: Has X1, Y1, length, angle, width, color, style, cap start, cap end
• Gauge: Has XYWH, rotation, value, style (horizontal, vertical, circular...), inactive color, active color, indicator width, indicator color, border width, border color
• Circle: Has XY, diameter, angle start, angle end, fill color/gradient, border width, border color, border alignment
• Icon: Has XYWH, rotation, icon from a super exhaustive set of vector icons, overlay color

Low priority

• Gauge: add possibility to use custom style for power users or to export from presets
• Videostream: like image, but can show a videostream (e.g. CITP, NDI, RTMP)
• Web: Has XYWH, rotation, scale, refresh. renders html from URL (probably with CSS but without JS). This may add limited value to a tiny button but will be very useful at larger areas
• Reference: Has XYWH, rotation, source, element, alignment horizontal, alignment vertical, crop/fit. Can reuse other containers or elements of the same graphics or from other buttons or complete other buttons

Any more?

All elements should have a Name/ID, Show/hide toggle, opacity and draw mode.
When you select an element, an outline should be shown in preview.

[Julusian]

Re that list, I'm not sure I see the value in matte, it looks like it is a subset of box, so may as well just use box?
I think that everything should have XYWH, including text. Because it will act as the bounding box, for text that means reducing the region for autosize to try filling.

When you select an element, an outline should be shown in preview.

yeah this is why I wanted to figure out client side rendering, I'm not sure what you mean by outline, but at the very least we should show XYWH bounding lines for the selected layer, hopefully drawn a few pixels outside the canvas so that they are always identifiable

---

Now playing around with client side rendering, which is working, and has some 'fun' results due to the fixed number calcs

[dnmeid]

I'm not sure I see the value in matte, it looks like it is a subset of box, so may as well just use box?

Yep, matte is a simplified box. My thought is that people would use a box with 100% size and without a border a lot as it translates to the old background. So maybe it is worth having this as an element that doesn't eat up much screen real estate in the list as a full featured box.
We can start with implementing the full featured box and see how it feels though, maybe I'm wrong and it's better to have only one element of this kind.

I think that everything should have XYWH, including text.

You're absolutely right with the text. I guess I missed it there.
But do you mean also the circle or the line with everything? I could go with the circle. It then would be more like the ellipse tool of Powerpoint. I prefer the XY for the center though.
For the line I don't see XYWH. It would actually translate to X1, Y1, X2-offset, Y2-offset. So a line from 20,40 to 50,25 would have to have 20,40,30,-25. I think that feels awkward. I think a lengh/direction approach would be best for the typical usecases I can think of when it comes to expressions. If you want to move the line, you need to adjust only one point and rotation is much easier with lenght/direction.

I'm not sure what you mean by outline, but at the very least we should show XYWH bounding lines for the selected layer, hopefully drawn a few pixels outside the canvas so that they are always identifiable

Exactly that

[Julusian]

But do you mean also the circle or the line with everything? I could go with the circle. It then would be more like the ellipse tool of Powerpoint. I prefer the XY for the center though.

no youre right, not everything, but many/most things. I suppose I meant that everything should in some way define its bounds/size (in the case of a line, that will be the two end points), nothing should only be able to fill the parent

---

But ok, I think this is getting along well as a POC. It is in no way polished or complete, but enough is done to be able to prove the concept and flow.

The preview drawing looks reasonably sane, but not 100% correct (there are some hacks in there, such as skewing the font size), and something is off in the scaled topbar maths.
It now also has fonts client side (the emoji one is playing up, or that could just be firefox)
And it is using 'live' values for any expressions (it is setup so that the client can subscribe to any expression and be notified when it changes)

I think the next step is getting feedbacks integrated into this structure. Based on my notes in the discussion, that means we need local variables to be able to handle boolean feedbacks, with advanced feedbacks being TBD.
I am starting to wonder about when would be best to merge this PR into develop, it is already pretty large in LOC

[Julusian]

I've had an idea regarding satellite and surfaces which can only do a 'simple mode' color and text, that I want to write down but may not want any discussion yet.
While we can try to pick the best element to use for this, we will often be 'wrong', so the user either needs to be able to specify which it will use, or needs a way to override it.
So perhaps on the canvas layer is some properties used for just this purpose. These won't be drawn in the normal way, they will just be data that we pipe out for the surfaces. Default should be automatic logic to pick the top/bottom elements, but with this opt-in override.

I see this as being needed for:

• some satellite devices
• xkeys (they can only do solid/flashing colours)
• blackmagic atem micro panel (only solid/flashing colours)
• some buttons on the loupedeck/streamdeck (only solid colours)

The other option for users would be to learn which elements the logic will pick, and to create some new elements which are positioned outside the canvas just to feed these surface types. Doable, but annoying and error prone. And as it is not uncommon for surfaces to have non lcd buttons, we should really cater to it nicely

Again, I am in no rush to do this, this is very much a polishing stage thing to do, currently all these protocols will pretend that these new buttons are blank. And I intend to keep the canvas element around, as I'm confident there will be something to go on it later

[dnmeid]

While we can try to pick the best element to use for this, we will often be 'wrong', so the user either needs to be able to specify which it will use, or needs a way to override it.

Yes, I've been thinking about that too and come to the same conclusion. And it is the same with the conversion of old buttons to new buttons or the mapping of the existing API or existing styling actions.
My thought is to do all of this in the same way. I'd go with some special element names. E.g. when the user calls a text element Text, we know that a set text action should set the text of that element. For surfaces we can publish what the keywords are, e.g. LED for a LED background.
When we don't find an element with the special word, we still can do the best effort guess.
We should give guidlines to the surface module developers so that there is some consistency in the names and also in the interpretation. E.g. if I have a 7-segment display, even if the element to use is clear, there should be a comon standard in how to interpret the data then.

[dnmeid]

we need local variables to be able to handle boolean feedbacks

I just had this coming to my mind.
When a feedback stores its result in a local variable and the options of a feedback can be variables (or expressions), wouldn't it be possible to write a feedback loop with potential hazardous results?

[Julusian]

My thought is to do all of this in the same way. I'd go with some special element names.

Rather than using the element names, I would be tempted to give them a 'user id' field. For the same reason I am resisting doing page based things by name; I don't want to tie a field that intended for the user to describe things to themselves into technical operation.
Especially here, having to call a field 'text' so that it gets used by satellite when its one of 3 text fields which I want to name based on their position/usage.

Instead I would prefer a 'user id' field, which could by default be automatically derived from the name (strip out some characters/spaces) with a checkbox to toggle it to manual mode.
I wouldn't be opposed to exposing this to the api/actions (if we want to keep those around and not deprecate them)
If we don't want to expose this, then maybe instead of 'user id' it should be a 'usage' dropdown, with the hardcoded usages that we know about.

But, this doesn't address my other concern. needing these special layers will be very common. xkeys,loupedeck,sds,sd-neo and more will all make use of these special values for some of their buttons/controls.
I still have the concern about needing to add some of these specially named elements, just to satisfy these special devices.
Perhaps you have a button with a gradiant background, and want an xkeys to use the midpoint of that gradiant, that would mean add a new layer to be this 'LED' colour.
Maybe this kind of reuse isn't actually that common, so this won't be that bad? And what you are proposing does fit nicely into not breaking the apis for setting properties..

I suppose that users wouldn't need to change the position of these hidden special elements, but instead simply change the opacity of them or a parent group to 0. Opacity probably shouldn't be factored into how satellite/surfaces read these values.

I don't have a pitch for how the SDS led ring should operate in this model, it probably should be done somehow in this system, but also feels a bit too specific (depending on how we want to expose it. limit it to % and a color/gradient?)

So maybe you are right, but with the added separation of name and 'id'

[dnmeid]

I can live with a separate ID field. Maybe we reuse the heading logic of actions for a name and add a separate ID.

needing these special layers will be very common. xkeys,loupedeck,sds,sd-neo and more will all make use of these special values for some of their buttons/controls.

My assumption is the opposite of yours in regard of how often these special elements will be used.
The reason for combining this into one field was that I don't expect this field to be used very often for the purpose of an ID and then we would have an ID field in every element that stays empty at 95%.
The main workflow should be that the surface module makes an automatic assumption on how to show the elements on a physical control and you should only need to override that in some cases. Most users will style the control for one specific physical element, e.g. for a square LCD button or for the rotary of SDS or for X-Key. They will learn and adopt quite fast how to use the elements so that the automatics can do its magic and for the users that have different surfaces at the same time or are working with the same config on different surfaces, the automatic should also almost every time give a sufficient pleasing result on a physical control that is was not made for. E.g. when you show a control made for a rotary on a LCD-Button.
So I want this override-method to be as unobtrusive as possible, but if it needs a field of its own, then ok.

I don't have a pitch for how the SDS led ring should operate in this model

My idea is to use a gauge element for that. In my elements layout there is a gauge element with XYWH, rotation, value, style (horizontal, vertical, circular...), inactive color, active color, indicator width, indicator color, border width, border color
The surface module could just use these values to draw the LED ring using inactive color, active color, indicator color, indicator width and value. That is only a rough idea. Maybe we need to tweak it a little bit, e.g. use the border width to cut off a part of the full 360° circle or add some limits fields to the gauge in general.

[Julusian]

The reason for combining this into one field was that I don't expect this field to be used very often for the purpose of an ID and then we would have an ID field in every element that stays empty at 95%.
So I want this override-method to be as unobtrusive as possible, but if it needs a field of its own, then ok.

Maybe we can hide the field when in automatic mode?
It might be a bit confusing to have a button after the name field in the element editor seeing as every other field has a toggle to an expression, but a button there could enable/disable the automatic id and whether the id field is visible?
Or perhaps there is a checkbox on the canvas element (or somewhere else which is configuration for the drawing) to enable/disable this behaviour for the whole button?
I'm making guesses here though, trying to find a way to make it work

My assumption is the opposite of yours in regard of how often these special elements will be used.
Most users will style the control for one specific physical element,

Yeah, this is what I realised half way through my last message. If a button only needs to work on one surface, then the auto naming and naive value extraction approach would be fine.

I don't have a pitch for how the SDS led ring should operate in this model

My idea is to use a gauge element for that

Makes sense. I've not begun to think about new element types yet.
What you have there makes sense.

---

The bit below may want to be ignored, I might have misinterpreted what you meant, but I stand by my reasoning for that discussion (which is a follow up to this task, if anything is done)

The main workflow should be that the surface module makes an automatic assumption on how to show the elements on a physical control and you should only need to override that in some cases.

So I think this might be where we arent in sync. I know you have proposed somewhere that surfaces should handle the drawing to a bitmap but I really disagree.
The challenge as I see it is if the surface plugins are aware of the structure of our elements/layers, then any change to that could be a breaking change to the plugins and to the satellite api.
I really want to be able to use the same plugins in satellite (so that we can stop having to write them all twice) so we would either need to push this structure over satellite, or satellite and the surface plugins need to be using a simplified form of the data.

I am thinking that actually we need the list of values that a surface can pull out of the structure for simple drawing needs to be a well defined list, to be able to work nicely in satellite. Another approach could be for some schema defining what a satellite surface needs, but thats more complexity for questionable gain.

[dnmeid]

Maybe we can hide the field when in automatic mode?

I woudn't call it a "mode". My idea is more that the renderer or whatever is responsible for showing the elements on the physical control first looks for some hints how to do it and for any needed information where no hint can be found it just uses the best guess. E.g. with the SDS LED ring, it would look for a gauge with a specific ID or name or tbd. marking. If that is not found it would use the first gauge. If there is no gauge it would use the color of a box with a special id (formely background). If there is no box with that ID it would use the color of the first box. If there is no box if would use a downsampled image and so on.

but a button there could enable/disable the automatic id and whether the id field is visible?

I see two options: either we combine name and ID and look for some hints in this combined field or we separate them in two fields. If it is two fields I'd just show both of them all the time.

The challenge as I see it is if the surface plugins are aware of the structure of our elements/layers, then any change to that could be a breaking change to the plugins and to the satellite api.

Yea, I see your point here, but I think that is unavoidable. If we make the surfaces plugins, then we need to define an API and this API will change at some times and depending on how clever we are, it will have more or less often breaking changes.
Considering the Satellite software I think that at the moment we can go on with the current Satellite API and just do the best effort mapping of elements to the established additional values we transfer with Satellite API. As soon as we start having surface modules, we definitely should use these modules in Satellite too and then I would use a new API for that, basically the internal data interface for drawing to the controls and getting inputs with some additions for surface management. Maybe this can be the groundwork for the deamon idea too.
The then legacy Satellite API can remain active for third party users (I'm using it myself in a few projects).