Replies: 2 comments 3 replies
-
Do we need the additional camera? I recon in the extract logic we can just stop iterating the children when we encounter a UiTexture? |
Beta Was this translation helpful? Give feedback.
3 replies
-
My use case was to make miniatures that can be animated(scaled) back into full menus - this would be perfect! |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
This is a proposal for a
UiTexture
component that causes UI sub-trees to be rendered on textures using customUiMaterials
(or a default material with basic properties like color tint).Objectives
Background
Currently, if you want a fade-in/fade-out effect on UI then you need to define a custom render target and stick a UI tree on a camea with
TargetCamera
. This works well, but requires some boilerplate and technical knowledge to set up, and does not extend to the advanced case of applying effects in the middle of a UI tree on nodes that participate in layout. It also means world-space UI is clunky.See:
Summary
The new
UiTexture<M>
component sets up a UI node so all children of the node are rendered to aUiMaterial
stored on that node.An entity with
UiTexture<M>
requires the following:Node
: We are usingUiMaterial
, so we also needNode
(and, transitively,Style
/etc.).Handle<M: UiTextureMaterial>
: Stores a handle to theUiMaterial
that will present our texture. If no handle is provided by the user, then a new default material will be inserted.UiTextureImage
: Stores a handle to theImage
asset of our texture. If no handle is provided by the user, then a new default image asset will be inserted. Note that the image will be auto-resized every tick.After a
UiTexture<M>
is added, we perform the following steps:Image
asset if necessary.M
material if necessary, and set its texture from the image asset (withUiTextureMaterial::set_texture
).Camera
and insert it to the entity in aUiTextureCamera
component. This camera is hidden, we will update it manually.Then, every tick, we:
UiTextureCamera
to propagate the camera entity to children in aUiRenderCamera
component. The 'render camera' will receive nodes during UI extraction, while theTargetCamera
andDefaultUiCamera
will continue to be used for layout calculations (and as fallbacks in case there's noUiRenderCamera
on a node).World-space 2D UI
Currently, any entity tree that starts with a
Node
will be rendered with the UI rendering pipeline. If a root node hasUiTexture
, then theUiMaterial
on that node will be rendered to a normal UI camera. However, for world-space UI we want a UI tree to render to a texture but for that texture to not be rendered in a UI camera (and instead get used by some mesh in the world).To support this, we can use the
WorldUiRoot
marker component to 'disable' nodes that haveUiTexture
so it collects children on itsUiMaterial
without the material being rendered by the UI pipeline.WorldUiRoot
then a layout computation will start at theWorldUiRoot
component (enabling you to stick UI directly on an entity with a mesh for world-space UI (pending analysis of compatibility withGlobalTransform
propagation)).Node
then a warning is emitted.WorldUiRoot
then it will not be rendered.TargetCamera
will propagate fromWorldUiRoot
entities, which is useful for layout.Note that this is not a complete solution to world-space UI, since we don't have a way to incorporate world coordinates automatically. But it should be relatively easy to expand on this concept in follow-up PRs.
API
Implementation
Comments
UiRenderCameraOverride
is a 'bonus' feature that is not required for an MVP and can be implemented in a follow-up PR.UiTexture
node is not rendered on theUiTexture
because otherwise the node would contain content rendered on two separate cameras (i.e. the UI components on the sub-camera, and theUiTextureMaterial
on a parent camera).Changes to existing code
content_size
field toNode
usingtaffy::Layout::content_size
. Seeupdate_uinode_geometry_recursive
.UiPlugin
, replacecheck_visibility<WithNode>
withcheck_visibility<(WithNode, Without<Changed<UiRenderCamera>>, Without<UiRenderOverrideTarget>)>
.UiUpdate::PostLayout
to run betweenVisibilityPropagate
andCheckVisibility
.extract_ui_material_nodes
to incorporate an optionalUiMaterialLayout
component on nodes.TargetCamera
propagation to start from both root nodes andWorldUiRoot
nodes.Tricks
UiRenderCamera
components every tick, and then only access them for rendering if they were changed this tick. This avoids complex hierarchy-management code. The perf cost is partially amortized by shifting somecheck_visibility
work into this traversal. See Add PropagateOpacity for controlling the alpha of entire UI node trees #15206 for the origin of this trick.UiRenderCamera
propagation, we can avoid duplicate traversals by checking if a given 'starter' node withUiTextureCamera
has aUiRenderCamera
already (which was presumably inserted by an ancestor withUiTextureCamera
in a previous tick). If it does, then cache the camera entity for a second-pass traversal once all entities withUiTextureCamera
and withoutUiRenderCamera
have been traversed (and remove the entity from the cache if we encounter it during traversal down from an ancestor).Scheduling and logic
To make everything fit together, we require precise organization of logic in the schedule.
UiTexture
components to their UI nodes.UiTextureMaterial
assets to create effects.Added<UiTexture<M>>
(system inUiTextureMaterialPlugin<M>
)Handle<M>
is default, add a new material asset.UiTextureImage
is default, add a new image asset.UiTextureCamera
with the material's texture as render target.UiTextureCamera::dont_render
field (system inUiTextureMaterialPlugin<M>
).ui_layout_system
WorldUiRoot
markers.TargetCamera
for layout.update_uinode_geometry_recursive
extract thetaffy::Layout::content_size
value and save it inNode
.(content_size - node_size)
as an offset from the upper left node corner (for offsetting the camera) and(content_size - node_size)*2 + node_size
as total size.UiTextureCamera
(need equivalent ofcamera_system
)camera.computed.target_info
from the new texture size.camera.computed.clip_from_view
from the projection.UiAntiAlias
and possibly other attributes need to be inferred from the destination camera, which need to be transitively inferred all the way up to the rootTargetCamera
/DefaultUiCamera
.UiTextureImage
asset's size.UiMaterialLayout
component.UiRenderCamera
to children ofUiTexture
(including but stopping at children who also haveUiTexture
).UiTexture::camera
: Ancestor withUiTexture
.UiTexture::should_render
: All nested render targets are not transparent (use cumulativeUiTextureCamera::dont_render
values), taking into accountUiRenderCameraOverride
.UiTexture::render_offset
: Offset between render camera origin and target camera origin (taking into account scale factors), taking into accountUiRenderCameraOverride
.VisibleEntities
for eachUiTextureCamera
(equivalent ofcheck_visibility
).UiRenderCameraOverride
.CheckVisibility
UiRenderCameraOverride
, push entities into theVisibleEntities
component on the correct override camera.!UiRenderCamera::should_render && !Has<UiRenderCameraOverride>
.WorldUiRoot
.clip
values need to be offset correctly so the clip rectangle lines up on the render target.UiMaterialLayout
to correctly position and size the material.extract_cameras
, use collectedVisibleEntities
fromUiSystem::PostLayout
.Testing
UiScale
UiTexture::scale
(potentially nested)Transform
scaling (does this affect UI?)UiTexture
:Outline
, then usingUiTexture::margin
should show/hide the outline at the edges.CalculatedClip
.Questions
UiBatch
andUiTextureSlicerBatch
? They don't seem to be accessed anywhere. If the entity is needed then it's an open question what to do, since the render camera may point to aUiTextureCamera
rather thanCamera
.UiTextureCamera
needs to function properly (i.e. seemlessly, as if it didn't exist), nor the exact setup code to ensure scaling behaves correctly.#[requires(Handle<M>, ...)
valid syntax?UiTexture::scale
is not 1.0?DefaultUiTextureMaterial
?UiMaterials
drawn in front of or behind the outline and border of a node?TransparentUi::sort_key
is tied to render-world entity index. Sinceextract_ui_material_nodes
is inRenderUiSystem::ExtractBackgrounds
which is afterRenderUiSystem::ExtractBorders
, the borders will presumably be drawn on top. It maybe necessary to add a separate step for extractingUiTexture
materials...Image
assets from the world, update them, and reinsert to get new asset ids and update cameras/materials with the new ids.GlobalTransform
would be affected.UiRenderCamera::transform_offset
to force-correct the transform in render extraction. Alternatively, we could directly mutateGlobalTransform
in theUiTexture
hierarchy traversal.Material
onDefaultUiMaterial
? The use-case is world-space UI.Future work
WorldUiNode
to be an enum that controlsLayoutContext
for taffy and also orientation/occlusion:WorldUiNode::Billboard
: Uses default camera or TargetCamera; render on that camera but useGlobalTransform
from a scene hierarchy to position it; 100% dimension = 100% of viewport. Replacesbevy_mod_billboard
without depth culling.WorldUiNode::BillboardObstructed
: Uses default camera or TargetCamera; render on that camera but useGlobalTransform
from a scene hierarchy to position it; 100% dimension = 100% of viewport. Replacesbevy_mod_billboard
with depth culling.WorldUiNode::Sized(fixed size)
: Assume there is a mesh on the node where the material will be rendered. 100% dimension = 100% of specified size.Beta Was this translation helpful? Give feedback.
All reactions