Add declareInitSegment method to the SegmentBuffer abstraction

This commit adds the `declareInitSegment` and `freeInitSegment` methods
to the RxPlayer's SegmentBuffer abstractions (which is the part of the
code handling the operations on SourceBuffers, such as pushing media and
init segments).

The short term idea is to improve the handling of initialization
segments in the SegmentBuffer.
Until now, each pushed media segment lead to a check that the
initialization segment it relies on is the same than the last one
pushed. To be able to perform that check, a caller need to communicate
again the initialization segment's data each time a chunk is pushed to
the buffer.

This check can be performed efficiently in most cases because we first
check init segment's data equality by reference, which, by pure luck,
should be equal in most cases in the current code. In cases where it
isn't the same reference however, it can lead to a byte-per-byte check,
which should not be an issue in terms of performance in most cases, but
is still an ugly specificity which could be handled in a more optimal
and understandable way.

This commit now allows the definition of a `initSegmentUniqueId`, an
identifier for initialization segments, on SegmentBuffers.
Any pushed segments can then refer to its associated init segment by
indicating which `initSegmentUniqueId` it is linked to.

The SegmentBuffer will ensure behind the hood that the right
initialization segment is pushed before pushing media segments, like
before, excepted that this can now be done just by comparing this
`initSegmentUniqueId` - it also means that the caller is no more
required to keep in memory the data of the loaded initialization
segment, the `SegmentBuffer` is already doing that.
Previously, the initialization segment's data was kept by the
`RepresentationStream`, the abstraction choosing which segments
to load (which is part of the reasons why the reference mostly never
changed).

The declaration and "freeing" of init segment is done through a
`declareInitSegment`/`freeInitSegment` pair of methods on a
`SegmentBuffer`. This sadly means that memory freeing for the
initialization segment is now manual, whereas we just relied on garbage
collection when the initialization segment was directly used.

---

Though mostly, the long term benefit is to implement the hybrid-worker
mode that we plan to have in the future, where buffering is performed in
a WebWorker (thus improving concurrence with an application, with the
goal of preventing both UI stuttering due to heavy player tasks and
rebuffering due to heavy UI tasks).

In the currently-planned long term worker features we would have thus
the following modes:

  - full worker: where both the rebuffering logic and MSE API are called
    in a WebWorker, allowing to avoid UI and media playback blocking
    each other to some extent

    This however requires the
    [MSE-in-Worker](w3c/media-source#175)
    feature to be available in the browser AND it also implies a more
    complex API, notably some callbacks (`manifestLoader`,
    `segmentLoader` and `representationFilter`) which will have to be
    updated.

  - hybrid mode: The buffering logic is mainly performed in a WebWorker
    but MSE API are still in the main thread. This allows e.g. to not
    fight for CPU with the UI to know which segments to download and to
    avoid blocking the UI when the Manifest is being parsed.

    Though the UI blocking could still mean that a loaded segment is
    waiting to be pushed in that mode.

    Because here MSE APIs may have to be called through `postMessage`-style
    message passing, the previous logic of communicating each time the
    same initialization segment each time a segment was pushed, with no
    mean to just move that data (in JavaScript linguo, to "transfer" it)
    was considerably worst than before.
    Relying on a short identifier instead seems a better solution here.

  - normal mode: The current mode where everything stays in main thread.

However it should be noted that all of this long term objective is still
in an higly experimental phase, and the gains are only theoretical for
now.

src/core/fetchers/cdn_prioritizer.ts

@@ -16,19 +16,22 @@
 
 import config from "../../config";
 import { ICdnMetadata } from "../../parsers/manifest";
-import { IPlayerError } from "../../public_types";
 import arrayFindIndex from "../../utils/array_find_index";
 import EventEmitter from "../../utils/event_emitter";
 import { CancellationSignal } from "../../utils/task_canceller";
 
 /**
- * Class signaling the priority between multiple CDN available for any given
- * resource.
+ * Class storing and signaling the priority between multiple CDN available for
+ * any given resource.
  *
- * This class might perform requests and schedule timeouts by itself to keep its
- * internal list of CDN priority up-to-date.
- * When it is not needed anymore, you should call the `dispose` method to clear
- * all resources.
+ * This class was first created to implement the complexities behind
+ * Content Steering features, though its handling hasn't been added yet as we
+ * wait for its specification to be both standardized and relied on in the wild.
+ * In the meantime, it acts as an abstraction for the simple concept of
+ * avoiding to request a CDN for any segment when an issue is encountered with
+ * one (e.g. HTTP 500 statuses) and several CDN exist for a given resource. It
+ * should be noted that this is also one of the planified features of the
+ * Content Steering specification.
  *
  * @class CdnPrioritizer
  */
@@ -41,7 +44,7 @@ export default class CdnPrioritizer extends EventEmitter<ICdnPrioritizerEvents>
   private _downgradedCdnList : {
     /**
      * Metadata of downgraded CDN, sorted by the time at which they have
-     * been downgraded.
+     * been downgraded ascending.
      */
     metadata : ICdnMetadata[];
     /**
@@ -174,8 +177,14 @@ export default class CdnPrioritizer extends EventEmitter<ICdnPrioritizerEvents>
   }
 }
 
+/** Events sent by a `CdnPrioritizer` */
 export interface ICdnPrioritizerEvents {
-  warnings : IPlayerError[];
+  /**
+   * The priority of one or several CDN changed.
+   *
+   * You might want to re-check if a CDN should still be used when this event
+   * is triggered.
+   */
   priorityChange : null;
 }
 

src/core/fetchers/segment/segment_fetcher.ts

@@ -64,10 +64,22 @@ const generateRequestID = idGenerator();
  * An `ISegmentFetcher` also implements a retry mechanism, based on the given
  * `options` argument, which may retry a segment request when it fails.
  *
- * @param {string} bufferType
- * @param {Object} pipeline
- * @param {Object} lifecycleCallbacks
- * @param {Object} options
+ * @param {string} bufferType - Type of  buffer concerned (e.g. `"audio"`,
+ * `"video"`, `"text" etc.)
+ * @param {Object} pipeline - The transport-specific logic allowing to load
+ * segments of the given buffer type and transport protocol (e.g. DASH).
+ * @param {Object|null} cdnPrioritizer - Abstraction allowing to synchronize,
+ * update and keep track of the priorization of the CDN to use to load any given
+ * segment, in cases where multiple ones are available.
+ *
+ * Can be set to `null` in which case a minimal priorization logic will be used
+ * instead.
+ * @param {Object} lifecycleCallbacks - Callbacks that can be registered to be
+ * informed when new requests are made, ended, new metrics are available etc.
+ * This should be mainly useful to implement an adaptive logic relying on those
+ * metrics and events.
+ * @param {Object} options - Various tweaking options allowing to configure the
+ * behavior of the returned `ISegmentFetcher`.
  * @returns {Function}
  */
 export default function createSegmentFetcher<TLoadedFormat, TSegmentDataType>(
@@ -365,11 +377,16 @@ export interface ISegmentFetcherCallbacks<TSegmentDataType> {
   /**
    * Callback called when all decodable chunks of the loaded segment have been
    * communicated through the `onChunk` callback.
+   *
+   * This call back is called before the corresponding `ISegmentFetcher`'s
+   * returned Promise is resolved.
    */
   onAllChunksReceived() : void;
 
   /**
-   * Callback called when the segment request has to restart from scratch. */
+   * Callback called when the segment request has to restart from scratch, e.g.
+   * due to a request error.
+   */
   onRetry(error : IPlayerError) : void;
 }
 

src/core/segment_buffers/implementations/audio_video/audio_video_segment_buffer.ts

@@ -21,10 +21,7 @@ import {
 import config from "../../../../config";
 import log from "../../../../log";
 import { getLoggableSegmentId } from "../../../../manifest";
-import areArraysOfNumbersEqual from "../../../../utils/are_arrays_of_numbers_equal";
 import assertUnreachable from "../../../../utils/assert_unreachable";
-import { toUint8Array } from "../../../../utils/byte_parsing";
-import hashBuffer from "../../../../utils/hash_buffer";
 import noop from "../../../../utils/noop";
 import objectAssign from "../../../../utils/object_assign";
 import TaskCanceller, {
@@ -138,20 +135,18 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
   private _pendingTask : IAVSBPendingTask | null;
 
   /**
-   * Keep track of the of the latest init segment pushed in the linked
-   * SourceBuffer.
+   * Keep track of the of the `uniqueId` of the latest init segment pushed to
+   * the linked SourceBuffer.
    *
    * This allows to be sure the right initialization segment is pushed before
    * any chunk is.
    *
    * `null` if no initialization segment have been pushed to the
    * `AudioVideoSegmentBuffer` yet.
    */
-  private _lastInitSegment : { /** The init segment itself. */
-                               data : Uint8Array;
-                               /** Hash of the initSegment for fast comparison */
-                               hash : number; } |
-                             null;
+  private _lastInitSegmentUniqueId : string | null;
+
+  private _initSegmentsMap : Map<string, unknown>;
 
   /**
    * @constructor
@@ -173,8 +168,9 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
     this._sourceBuffer = sourceBuffer;
     this._queue = [];
     this._pendingTask = null;
-    this._lastInitSegment = null;
+    this._lastInitSegmentUniqueId = null;
     this.codec = codec;
+    this._initSegmentsMap = new Map();
 
     const onError = this._onPendingTaskError.bind(this);
     const reCheck = this._flush.bind(this);
@@ -197,6 +193,19 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
     });
   }
 
+  public declareInitSegment(
+    uniqueId : string,
+    initSegmentData : unknown
+  ) : void {
+    this._initSegmentsMap.set(uniqueId, initSegmentData);
+  }
+
+  public freeInitSegment(
+    uniqueId : string
+  ) : void {
+    this._initSegmentsMap.delete(uniqueId);
+  }
+
   /**
    * Push a chunk of the media segment given to the attached SourceBuffer, in a
    * FIFO queue.
@@ -228,12 +237,12 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
     infos : IPushChunkInfos<unknown>,
     cancellationSignal : CancellationSignal
   ) : Promise<void> {
-    assertPushedDataIsBufferSource(infos);
+    assertDataIsBufferSource(infos.data.chunk);
     log.debug("AVSB: receiving order to push data to the SourceBuffer",
               this.bufferType,
               getLoggableSegmentId(infos.inventoryInfos));
     return this._addToQueue({ type: SegmentBufferOperation.Push,
-                              value: infos },
+                              value: infos as IPushChunkInfos<BufferSource> },
                             cancellationSignal);
   }
 
@@ -349,7 +358,7 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
    * @param {Event} err
    */
   private _onPendingTaskError(err : unknown) : void {
-    this._lastInitSegment = null; // initialize init segment as a security
+    this._lastInitSegmentUniqueId = null; // initialize init segment as a security
     if (this._pendingTask !== null) {
       const error = err instanceof Error ?
                       err :
@@ -462,7 +471,7 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
           const error = e instanceof Error ?
             e :
             new Error("An unknown error occured when preparing a push operation");
-          this._lastInitSegment = null; // initialize init segment as a security
+          this._lastInitSegmentUniqueId = null; // initialize init segment as a security
           nextItem.reject(error);
           return;
         }
@@ -572,15 +581,18 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
       this._sourceBuffer.appendWindowEnd = appendWindow[1];
     }
 
-    if (data.initSegment !== null &&
-        (hasUpdatedSourceBufferType || !this._isLastInitSegment(data.initSegment)))
+    if (data.initSegmentUniqueId !== null &&
+        (hasUpdatedSourceBufferType ||
+         !this._isLastInitSegment(data.initSegmentUniqueId)))
     {
       // Push initialization segment before the media segment
-      const segmentData = data.initSegment;
+      const segmentData = this._initSegmentsMap.get(data.initSegmentUniqueId);
+      if (segmentData === undefined) {
+        throw new Error("Invalid initialization segment uniqueId");
+      }
+      assertDataIsBufferSource(segmentData);
       dataToPush.push(segmentData);
-      const initU8 = toUint8Array(segmentData);
-      this._lastInitSegment = { data: initU8,
-                                hash: hashBuffer(initU8) };
+      this._lastInitSegmentUniqueId = data.initSegmentUniqueId;
     }
 
     if (data.chunk !== null) {
@@ -591,56 +603,37 @@ export default class AudioVideoSegmentBuffer extends SegmentBuffer {
   }
 
  /**
-  * Return `true` if the given `segmentData` is the same segment than the last
+  * Return `true` if the given `uniqueId` is the identifier of the last
   * initialization segment pushed to the `AudioVideoSegmentBuffer`.
-  * @param {BufferSource} segmentData
+  * @param {string} uniqueId
   * @returns {boolean}
   */
-  private _isLastInitSegment(segmentData : BufferSource) : boolean {
-    if (this._lastInitSegment === null) {
+  private _isLastInitSegment(uniqueId : string) : boolean {
+    if (this._lastInitSegmentUniqueId === null) {
       return false;
     }
-    if (this._lastInitSegment.data === segmentData) {
-      return true;
-    }
-    const oldInit = this._lastInitSegment.data;
-    if (oldInit.byteLength === segmentData.byteLength) {
-      const newInitU8 = toUint8Array(segmentData);
-      if (hashBuffer(newInitU8) === this._lastInitSegment.hash &&
-          areArraysOfNumbersEqual(oldInit, newInitU8))
-      {
-        return true;
-      }
-    }
-    return false;
+    return this._lastInitSegmentUniqueId === uniqueId;
   }
 }
 
 /**
  * Throw if the given input is not in the expected format.
  * Allows to enforce runtime type-checking as compile-time type-checking here is
  * difficult to enforce.
- * @param {Object} pushedData
+ * @param {Object} data
  */
-function assertPushedDataIsBufferSource(
-  pushedData : IPushChunkInfos<unknown>
-) : asserts pushedData is IPushChunkInfos<BufferSource> {
+function assertDataIsBufferSource(
+  data : unknown
+) : asserts data is BufferSource {
   if (__ENVIRONMENT__.CURRENT_ENV === __ENVIRONMENT__.PRODUCTION as number) {
     return;
   }
-  const { chunk, initSegment } = pushedData.data;
   if (
-    typeof chunk !== "object" ||
-    typeof initSegment !== "object" ||
-    (
-      chunk !== null &&
-      !(chunk instanceof ArrayBuffer) &&
-      !((chunk as ArrayBufferView).buffer instanceof ArrayBuffer)
-    ) ||
+    typeof data !== "object" ||
     (
-      initSegment !== null &&
-      !(initSegment instanceof ArrayBuffer) &&
-      !((initSegment as ArrayBufferView).buffer instanceof ArrayBuffer)
+      data !== null &&
+      !(data instanceof ArrayBuffer) &&
+      !((data as ArrayBufferView).buffer instanceof ArrayBuffer)
     )
   ) {
     throw new Error("Invalid data given to the AudioVideoSegmentBuffer");

src/core/segment_buffers/implementations/image/image_segment_buffer.ts

@@ -38,6 +38,22 @@ export default class ImageSegmentBuffer extends SegmentBuffer {
     this._buffered = new ManualTimeRanges();
   }
 
+  /**
+   * @param {string} uniqueId
+   */
+  public declareInitSegment(uniqueId : string): void {
+    log.warn("ISB: Declaring initialization segment for image SegmentBuffer",
+             uniqueId);
+  }
+
+  /**
+   * @param {string} uniqueId
+   */
+  public freeInitSegment(uniqueId : string): void {
+    log.warn("ISB: Freeing initialization segment for image SegmentBuffer",
+             uniqueId);
+  }
+
   /**
    * @param {Object} data
    * @returns {Promise}

src/core/segment_buffers/implementations/text/html/html_text_segment_buffer.ts

@@ -137,6 +137,22 @@ export default class HTMLTextSegmentBuffer extends SegmentBuffer {
     this.autoRefreshSubtitles(this._canceller.signal);
   }
 
+  /**
+   * @param {string} uniqueId
+   */
+  public declareInitSegment(uniqueId : string): void {
+    log.warn("ISB: Declaring initialization segment for image SegmentBuffer",
+             uniqueId);
+  }
+
+  /**
+   * @param {string} uniqueId
+   */
+  public freeInitSegment(uniqueId : string): void {
+    log.warn("ISB: Freeing initialization segment for image SegmentBuffer",
+             uniqueId);
+  }
+
   /**
    * Push text segment to the HTMLTextSegmentBuffer.
    * @param {Object} infos

src/core/segment_buffers/implementations/text/native/native_text_segment_buffer.ts

@@ -66,6 +66,22 @@ export default class NativeTextSegmentBuffer extends SegmentBuffer {
     this._trackElement = trackElement;
   }
 
+  /**
+   * @param {string} uniqueId
+   */
+  public declareInitSegment(uniqueId : string): void {
+    log.warn("ISB: Declaring initialization segment for image SegmentBuffer",
+             uniqueId);
+  }
+
+  /**
+   * @param {string} uniqueId
+   */
+  public freeInitSegment(uniqueId : string): void {
+    log.warn("ISB: Freeing initialization segment for image SegmentBuffer",
+             uniqueId);
+  }
+
   /**
    * @param {Object} infos
    * @returns {Promise}