Expose video playback events on amp-video

[lexandera]

We are looking into supporting video playback in amp ads and have discovered that there are some missing features which would allow us to support some common patterns which appear in video ads:

• Custom "play" buttons (or other interactive elements) which are hidden on playback start and cannot be "baked" into the video poster image for various reasons
• Product shot/details appearing when the video ends
• Sometimes contextual information appears at a certain point in the video

The first pattern is doable if you also add several "hide" actions to the button which initiates playback, but there is no way of doing the second and the third one. All would be made possible if amp-video exposed a couple of playback related events that could be used for binding actions to.

Based on what we've been doing in ads so far, the most useful events to have would be:

• "video ended" event - the most useful by far - for restoring custom playback controls, or for switching to an interactive product view after the video ends.
• "video paused" and "video playing" event - mostly useful for implementing custom controls.
• "video cue point" event - for reacting to video playback reaching a certain point in time in order to display some contextual information relevant to that part of the video (for example, a description of currently visible product). Not used as often, but can give nice results when used correctly.

[aghassemi]

@lexandera
As you mentioned (1) is already possible. We actually have a sample for it on AmpByExample as well.

As for exposing events, there are not technical hurdles in doing so the issue we need to deal with is potential abuse. AMP does not expose events that do not come from direct user actions. Video events like end, cue points are too separated from the initial play user action and may not even have a user action associated with them (in the case of autoplay). Couple of options here:

1- Start allowing A4A format to have more flexibility when it comes to non-user-initiated events as any UX abuse will be contained within the Ad box. Not sure if we have other AMP features that are that are A4A only. @jasti @lannka @rudygalfi

2- Built these features into amp-video itself as suggested by @ericlindley-g: e.g.

<amp-video>
  <div overlay="start"></div>
  <div overlay="cue" start="3" end="5">Overlay between 3-5 seconds timeline</div>
  <div controls>
    <button play></button>
  </div>
</amp-video>

This approach is less flexible but more aligned with AMP principles, It will also turn amp-video into more complicated video-player rather than simple replacement for HTML5 video

[aghassemi]

@rudygalfi @jasti to you guys for prioritization.

[lexandera]

@aghassemi What about using an approach similar to what mobile browsers do where they allow certain actions from user initiated events only (and calls cannot be deferred), while allowing others in all cases? A "video ended" event in that case would allow things like showing/hiding/animating of existing objects, but not also automatic replaying of videos, or page redirects.

Once amp-animation becomes available it's most likely going to be possible to hide a video element and replace it with a product shot at the end of the video by creating a long "empty" animation which performs the switch after (let's say) 30 seconds, and then starting both the scene and the video at the same time from a user triggered event; but it won't be possible to account for the video loading time, leading to premature switches on slower connections, and delays on faster ones. Making it possible to toggle visibility from a "video ended" event instead of using such hacks would just make it more predictable.

[aghassemi]

@lexandera even show, hide are important to be tied to direct user actions or they can introduce page jumps which AMP has been trying very hard to avoid. We have realized CSS and AMP animation can be used to get around it and actually have discussed approaches to prevent it (it is a fairly hard problem as you would imagine). However we still don't want to have "official" ways to allow abuses that are against AMP UX principles.

That being said, as per my #1 above, the context difference between an AMP Page and an AMP Ad is significant enough (e.g. sandboxed container, no scrolling, etc..) to justify different rules. show, hide is a good example of this. Within an Ad, it can not cause page jumps, so IMO it should be supported.

@jasti @ericlindley-g thoughts? (In short, I am suggesting we enable certain features only in A4A because A4A can not abuse them but an AMP page can)

[jasti]

@aghassemi +1 for supporting custom events based on timeline because of ad size restrictions.
Also like the idea of only allowing certain actions depending on the type of event. e.g. videoEnded --> Show controls.

[jasti]

CC @ampproject/a4a as heads-up if anyone has any concerns.

[ericlindley-g]

+1 To this idea of whitelisting certain behaviors only for A4A — it seems like a low-risk way to enable a set of features that we just couldn't do in AMP. Would be good to review this with a larger group to get consensus. One tradeoff is the possible loss of semantics—do we care if developers use A4A not for ads at all, but only for some set of features that can only happen in the A4A context? Is it called something else at that point?

[aghassemi]

@jpettitt brought up a good point in the design meeting today: This feature is useful outside of AMP in-a-box case and if we go with approach 1 (#8971 (comment)) then top-level AMP pages will either miss out on this feature or they will be forced to unnecessarily iframe another AMP page just to get this feature.

At this point I am recommending to implement this feature on amp-video (approach 2), the proposal to allow in-a-box to have more power when it comes to running actions not tied to a direct user-interaction is still a valid feature to consider for other use-cases.
/cc @ericlindley-g

[dreamofabear]

Approach 1 may be feasible if we implement a tiered trust levels for events and actions as suggested in #8901 (comment):

• Add new video play/pause/end events under new "LOW" trust level (current actions have a higher trust)
• Add support for "overlay" elements for children of AMP components with "LOW" trust actions show, hide, toggle

[rudygalfi]

@aghassemi @jasti Is there any updated status on this?

[aghassemi]

• flexible custom controls coming later Support custom controls for amp-video #10457 implementation in progress Custom video controls (phase 1), minimized video controls and video docking bug fixes #10865
• start overlay already possible: https://ampbyexample.com/advanced/click-to-play_overlay_for_amp-video/
• cue / end overlays can be implemented with amp-animation by syncing timelines with amp-video (see description of amp-position-observer and redesign of scrollbound/visibility-based behaviour  #10818 )

[aghassemi]

We have exposed a few (firstPlay and timeUpdate) already. https://amp.dev/documentation/guides-and-tutorials/learn/amp-actions-and-events#amp-video,-amp-youtube

Note these will stay low-trust events.

Please file issues for others as needed.