Store font texture atlas handles in a separate list in TextLayoutInfo

[ickshonpe]

Objective

Most displayed text only uses one TextureAtlas asset but:

• TextLayoutInfo stores a texture atlas handle per individual glyph.
• The text extraction functions retrieve the texture atlas from assets per glyph.

fixes #9278

Solution

Remove the texture atlas asset handle from PositionedGlyph.
Add a new field batches: Vec<PostionedGlyphBatch> to TextureLayoutInfo.
Each PositionedGlyphBatch consists of a texture atlas handle and a range. These ranges partition the PositionedGlyph list into contiguous sections sharing the same texture atlas.

Before leaving GlyphBrush::process_glyphs we sort the PositionedGlyphBatch vector by texture atlas handle ensuring that glyphs sharing the same texture are batched together during rendering.

cargo run --profile stress-test --features trace_tracy --example many_glyphs

The Bevy stress test examples don't capture the batching change improvements. This example reports 25% better fps on the textlayout-refactor branch compared to main on my computer:

use bevy::{
    diagnostic::{FrameTimeDiagnosticsPlugin, LogDiagnosticsPlugin},
    prelude::*,
    text::BreakLineOn,
    window::{PresentMode, WindowPlugin},
};

fn main() {
    let mut app = App::new();
    app.add_plugins((
        DefaultPlugins.set(WindowPlugin {
            primary_window: Some(Window {
                present_mode: PresentMode::AutoNoVsync,
                ..default()
            }),
            ..default()
        }),
        FrameTimeDiagnosticsPlugin,
        LogDiagnosticsPlugin::default(),
    ))
    .add_systems(Startup, setup);
    app.run();
}

fn setup(mut commands: Commands) {
    commands.spawn(Camera2dBundle::default());
    let sections = 
        (0..1_000).map(|n| {
            TextSection {
                value: "0123456789".to_string(),
                style: TextStyle {
                    font_size: 10. * (1 + n % 5) as f32,
                    color: Color::WHITE,
                    ..default()
                },
            }
        }).collect();

    let text = Text {
        sections,
        alignment: TextAlignment::Left,
        linebreak_behavior: BreakLineOn::AnyCharacter,
    };

    commands
        .spawn(NodeBundle {
            style: Style {
                flex_basis: Val::Percent(100.),
                align_items: AlignItems::Center,
                justify_content: JustifyContent::Center,
                ..default()
            },
            ..default()
        })
        .with_children(|commands| {
            commands.spawn(TextBundle {
                text: text.clone(),
                style: Style {
                    width: Val::Px(1000.),
                    ..Default::default()
                },
                ..Default::default()
            });
        });
}

---

Changelog

TextLayoutInfo

• Added a field atlases: Vec<(Handle<TextureAtlas>, usize)> to hold the atlas handles for the fonts instead of individual PositionedGlyphs.

PositionedGlyph:

• Removed atlas_info field.
• Added glyph_index: usize field.

extract_text2d_sprite & extract_text_uinodes:

• Added an outer loop to text extraction that iterates through TextLayoutInfo's atlas handle list.

GlyphBrush::process_glyphs:

• Generates and returns a second vector holding the list of texture atlas handles.

Migration Guide

TextLayoutInfo no longer stores texture atlas handles per glyph but instead in a separate vector atlases.

[rparrett]

Would it be possible/make sense to do something similar, but based on section_index rather than glyph_index?

[github-actions]

Example alien_cake_addict failed to run, please try running it locally and check the result.

[ickshonpe]

Would it be possible to do something similar, but based on section_index rather than glyph_index?

not sure The code would be simpler if it just stored an atlas handle per section index but we'd end up storing a lot of duplicate handles for stuff like multi-colored text.

[rparrett]

Are glyphs already stored/processed in increasing order of section_index? Could you use the same strategy of storing the section_index range endpoint with the handles?

Either way, maybe (Vec<(Handle<TextureAtlas>, usize)>, Vec<PositionedGlyph>) could use a custom type because this code does feel pretty mysterious.

Sorry for the somewhat lazy reviews. Not enough contiguous free time to actually understand what's going on in these pipelines.

[ickshonpe]

Are glyphs already stored/processed in increasing order of section_index? Could you use the same strategy of storing the section_index range endpoint with the handles?

Either way, maybe (Vec<(Handle<TextureAtlas>, usize)>, Vec<PositionedGlyph>) could use a custom type because this code does feel pretty mysterious.

Sorry for the somewhat lazy reviews. Not enough contiguous free time to actually understand what's going on in these pipelines.

You've given me a really good idea actually, I can clean this up and fix #9278 at the same time.

[github-actions]

Example alien_cake_addict failed to run, please try running it locally and check the result.

[rparrett]

Alright, getting up to speed here. I hadn't noticed that process_glyphs is involved in building the texture atlases.

This is interesting. So this is doing "local batching" of for each entity. Text across multiple entities with the same font is not batched.

It also looks like a scenario similar to your new example like:

sections: [font_a, font_b, font_a]

would result in at least 3 batches, right? More if the particular glyphs in a section lie in a different atlas of the FontAtlasSet?

It seems like a fairly niche optimization. Have we attempted to / is there a plan to eventually do "global batching" of glyphs?

[ickshonpe]

Alright, getting up to speed here. I hadn't noticed that process_glyphs is involved in building the texture atlases.

This is interesting. So this is doing "local batching" of for each entity. Text across multiple entities with the same font is not batched.

Text from different entities can get batched together. It depends on how elaborate the UI is and how the node tree is organised. #8793 added in 0.11 improved things a lot. The many_buttons example which just consists of labelled button rects is rendered entirely in one batch now compared to ~24,000 batches in 0.10.

Also, if you are conscious of how things work you can rearrange a UI layout to reduce the number of batches significantly.
Consider this simple inventory UI with some item buttons and labels:

Assume the labels are all from the same font, and the images are all from the same texture atlas.

A natural way to construct this UI is to have a row of nodes, where each node in the row contains an item button and its label arranged in a column (the red lines denote the boundaries of the column nodes):

With this design, the items will be rendered in seven batches:

1. inventory panel, border, inventory label, potion border
2. potion sprite
3. potion label, boots border
4. boots sprite
5. boots label, gem border
6. Gem sprite.
7. Gem label.

A different approach would be to place all the labels first and then arrange the items in a row inside a node placed above the inventory panel:

Just two batches would be generated for this design:

1. inventory panel, border, inventory label, potion label, boots label, gem label, potion border
2. potion, boots border, boots, gem border, gem

It also looks like a scenario similar to your new example like:

sections: [font_a, font_b, font_a]

would result in at least 3 batches, right? More if the particular glyphs in a section lie in a different atlas of the FontAtlasSet?

Yep that would result in three batches in main, and two with this PR (assuming we're not over the texture atlas limit for fonts).
Another advantage is that because we do this sort in process_glyphs it's only performed on changes to the text, not every frame in the prepare stage like for everything else.

If someone is displaying large pages of text that use many different fonts these changes will have a large impact but otherwise, it's not going to help that much.

It seems like a fairly niche optimization. Have we attempted to / is there a plan to eventually do "global batching" of glyphs?

I've got a second PR #9212 that uses a similar "local batching" strategy to greatly improve overall UI extraction performance. I don't know that much about rendering, so I've been concentrating on improvements like this which reduce the amount of data and arrange and manage it more efficiently etc. Text rendering is quite difficult to optimise I think as the glyphs have transparency and we can't just sort everything by depth. The UI layout tree z-order rules are very complicated as well. Unfortunately we can't just walk the UI layout tree and assume that every element deeper in the tree will be drawn above every element before it.

[rparrett]

Ah, right. The UI stack stuff is a bit of an obstacle.

Yep that would result in three batches in main, and two with this PR

I was actually confused there, but I see now that the existing UI batching gets that down to two.

I appreciate the thorough explanation.

[ickshonpe]

Ah, right. The UI stack stuff is a bit of an obstacle.

Yep that would result in three batches in main, and two with this PR

I was actually confused there, but I see now that the existing UI batching gets that down to two.

I appreciate the thorough explanation.

Yep, another change I'd like to make is to remove ZIndex::Global. I'm not sure that it offers any tangible advantages over PositionType::Absolute and very rarely ever see anyone using it, but it's expensive and adds a lot of extra complexity.