RUMM-2429 [SR] Capture UIViews and UILabels semantics

[ncreated]

What and why?

📦 This PR enhances Session Replay with recognising certain node semantics for UIViews and UILabels. For both, the recorder now captures more information on the native view and puts it into the JSON prepared for SR player.

The Recorder now captures:

• appearance information on each UIView:
  • background color
  • border style (color + width)
  • corner radius
  • alpha
• text information on each UILabel:
  • the text displayed
  • the actual frame of the test (including paddings)
  • text color
  • font

How?

Proposed implementation introduces a small framework prepared for capturing semantics of other nodes in extensible way promoting OCP (UITextField, UIImageView, UISwitch will come in next PR).

Each view traversed by ViewTreeSnapshotBuilder gets queried by a sequence of NodeRecorders:

nodeRecorders: [
    UIViewRecorder().eraseToAnyNodeRecorder,
    UILabelRecorder().eraseToAnyNodeRecorder,
    UIImageViewRecorder().eraseToAnyNodeRecorder, // coming in next PR
    UITextFieldRecorder().eraseToAnyNodeRecorder, // coming in next PR
    UISwitchRecorder().eraseToAnyNodeRecorder,    // coming in next PR
]

Each recorder gets asked for a semantics of given view:

internal protocol NodeRecorder {
    associatedtype View: UIView

    func semantics(of view: View, with attributes: ViewAttributes, in context: ViewTreeSnapshotBuilder.Context) -> NodeSemantics?
}

To avoid querying all recorders for each view, the iteration stops as soon as best semantics is found. Pessimistically, this gives O(N * M) time complexity on the main thread, where N is number of views and M is number of node recorders. Given that M is constant and some recorders (e.g. UISwitchRecorder) will not require inspecting sub-tree, the effective estimate will be O(N).

To enable different optimisations and stay open for future extensibility of Recorder (to support more types in ambiguous view trees), the NodeSemantics is defined as a broad interface:

internal protocol NodeSemantics {
    /// The severity of this semantic.
    ///
    /// While querying certain `view` with an array of supported `NodeRecorders` each recorder can spot different semantics of
    /// the same view. In that case, the semantics with higher `importance` takes precedence.
    static var importance: Int { get }

    /// A type defining how to build SR wireframes for the UI element this semantic was recorded for.
    var wireframesBuilder: NodeWireframesBuilder? { get }
}

with 4 semantics introduced in this PR:

/// Semantics of an UI element that is of unknown kind. Receiving this semantics in `Processor` could indicate an error
/// in view-tree traversal performed in `Recorder` (e.g. working on assumption that is not met).
internal struct UnknownElement: NodeSemantics { /**/ }

/// A semantics of an UI element that is either `UIView` or one of its known subclasses. This semantics mean that the element
/// has no visual appearance that can be presented in SR (e.g. a `UILabel` with no text, no border and fully transparent color).
/// Nodes with this semantics can be safely ignored in `Recorder` or in `Processor`.
internal struct InvisibleElement: NodeSemantics { /**/ }

/// A semantics of an UI element that is of `UIView` type. This semantics mean that the element has visual appearance in SR, but
/// it will only utilize its base `UIView` attributes. The full identity of the node will remain ambiguous if not overwritten with `SpecificElement`.
internal struct AmbiguousElement: NodeSemantics { /**/ }

/// A semantics of an UI element that is one of `UIView` subclasses. This semantics mean that we know its full identity along with set of
/// subclass-specific attributes that will be used to render it in SR (e.g. all base `UIView` attributes plus the text in `UILabel` or the
/// "on" / "off" state of `UISwitch` control).
internal struct SpecificElement: NodeSemantics { /**/ }

These 4 semantics are enough to handle basics, incl.: UIViewRecorder, UILabelRecorder, UIImageViewRecorder, UITextFieldRecorder and UISwitchRecorder. We may want to introduce more semantics when dealing with elements that require very much different rendering (like alerts or popovers displayed on top of other views). I haven't yet looked into this.

Review checklist

• Feature or bugfix MUST have appropriate tests (unit, integration)
• Make sure each commit and the PR mention the Issue number or JIRA reference
• Add CHANGELOG entry for user facing changes

Custom CI job configuration (optional)

• Run unit tests
• Run integration tests
• Run smoke tests

[ncreated]

There are yet no unit tests for NodesFlattener as it is still not in its final shape 🚧. The RUMM-2461 will focus solely on flattening algorithm and then tests will become mandatory.

[ncreated]

Because of lack of NodeWireframesBuilder in "unknown" and "invisible" semantics, earlier I tried to model NodeSemantics as enum:

enum NodeSemantics {
   case unknownElement
   case invisibleElement
   case ambiguousElement(wireframesBuilder: NodeWireframesBuilder)
   case specificElement(wireframesBuilder: NodeWireframesBuilder)
}

While it looked better here in declaration, it was very cumbersome for use both in Recorder and Processor (flattening part). It required guard let case pattern matching and was forcing numerous code branches.

I found the "protocol + optionals" approach to be more efficient in this kind of problem.

[maxep]

Great job here!! it reads super well and the separation of concerns is perfect.

I'm wondering if the genericity of NodeRecorder has reached its purpose. Since it only supports subclasses of UIView, and the type-check is made at runtime. I think we could benefit from removing the associatedtype and check the view type in its concrete implementations. LMKWYT!

[maxep]

It can be checked at compile time instead.

Suggested change

	init<NR: NodeRecorder>(_ concreteRecorder: NR) {
	self.erasedMethod = { view, node, context in
	guard let concreteView = view as? NR.View else {
	// Means that the `UIView` instance passed to erased `NodeRecorder` was not of the type
	// that this recorder supports. In that case, semantics cannot be determined.
	return nil
	}
	return concreteRecorder.semantics(of: concreteView, with: node, in: context)
	}
	}
	init<NR: NodeRecorder>(_ concreteRecorder: NR) where NR.View == View {
	self.erasedMethod = { view, node, context in
	return concreteRecorder.semantics(of: concreteView, with: node, in: context)
	}
	}

[maxep]

Ok, so actually no, it can't.

After reading the usage of NodeRecorder and its type-erasure, I wonder if the associatedtype is actually necessary. The type check is made at runtime so it can jump on the next recorder, the generic constraint is not applied here. I think this should be made explicit by using this interface:

internal protocol NodeRecorder {
    func semantics(of view: UIView, with attributes: ViewAttributes, in context: ViewTreeSnapshotBuilder.Context) -> NodeSemantics?
}

The concrete implementation of a recorder can return nil if the type is not supported.

internal struct UILabelRecorder: NodeRecorder {
    func semantics(of view: UIView, with attributes: ViewAttributes, in context: ViewTreeSnapshotBuilder.Context) -> NodeSemantics? {
        guard let label = view as? UILabel else {
            return nil
        }
        ...
}

[ncreated]

Yeah, the main purpose of erasure was to avoid repeating guard let in all implementations. But your point is correct - all views are instance of UIView and derived type, so it could be avoided. I'm OK to remove erasures for the sake of simplicity 👍 - less constraints could be even better as I can't fully foresee what other challenges are there in UIKit / SwiftUI hierarchies.

[ncreated]

I went with guard let ... else { nil } way - it's simpler and not that tedious anyway 👍.

[ncreated]

I'm wondering if the genericity of NodeRecorder has reached its purpose. Since it only supports subclasses of UIView, and the type-check is made at runtime. I think we could benefit from removing the associatedtype and check the view type in its concrete implementations. LMKWYT!

Agree - done it and it feels better now 👍 - there was not much gain from associatedtype 🙂.

[maxep]

Awesome!! looks really good and the documentation is on point.
Thank you for the PR description, it really helped!