UI layout tree optimizer

[ickshonpe]

What problem does this solve or what need does it fill?

It's possible to construct UI layouts that look and behave identically but which have vastly different performance characteristics. Often the most natural approach to constructing a UI is going to be the least efficient design.

Consider this simple inventory UI with some item buttons and labels :

Assume the labels are all from the same font, and the images are all from the same texture atlas.

A natural way to construct this UI is to have a row of nodes, where each node in the row contains an item button and its label arranged in a column (the red lines denote the boundaries of the column nodes):

With this design, the items will be rendered in seven batches:

1. inventory panel, border, inventory label, potion border
2. potion sprite
3. potion label, boots border
4. boots sprite
5. boots label, gem border
6. gem sprite
7. gem label

A different approach would be to place all the labels first and then arrange the items in a row inside a node placed above the inventory panel:

Now only two batches are generated:

1. inventory panel, border, inventory label, potion label, boots label, gem label, potion border
2. potion, boots border, boots, gem border, gem

Unfortunately though desirable, the second design is much more difficult to construct. The user has to work out a way to align the item labels with the item buttons even though they are in separate branches of the layout tree.

What solution would you like?

A tool, maybe it would be a feature built into the editor eventually, that can analyse a UI layout tree like the one in the example above and transform it into a more efficient form, without changing its appearance or behaviour.

Maybe there are some simple transformations that could be applied in ui_layout_system at run time, I'm not sure.

It could also be a nice project for someone who wants to make a third-party crate.

Additional information

Some relevant PRs: #9471, #9212, #8793

[alice-i-cecile]

Speaking as the maintainer of taffy, I'd really like to see this implemented there, and then used by Bevy. This is a neat (if very complex) idea that would be broadly useful for all users of taffy.

Of course, that only really works if we can do this in an automated non-graphical way: tools that require judgement calls on "close but not exact" layout swaps would need a GUI and wouldn't be a good candidate for upstreaming.

[nicoburns]

Why does the UI layout affect batching? I feel like the rendering system ought to be smart enough to batch things in a non order-dependent way and taking into account that non-overlapping children of a parent can be combined into single batch.

Perhaps making overflow default to Clip would help with this optimisation?