From 201420c0b35bbc805bc436cda330a09cca012b72 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Piotr=20Krzemi=C5=84ski?= Date: Fri, 22 Mar 2024 08:47:06 +0100 Subject: [PATCH] [Spec] Clarify specification for StridedSlice (#23039) ### Details: - Add notes with descriptions of: Out of Bounds, Indexing in Reverse, Negative Indices - Clarified length of masks - Clarified the definition of `-1` value - Described in detail the behavior of masks, aligned with Reference Implementation - Added more latex-like style, add the examples for the missing masks. ### Tickets: - 90128 --- .../movement/strided-slice-1.rst | 324 ++++++++++++++++-- 1 file changed, 293 insertions(+), 31 deletions(-) diff --git a/docs/articles_en/documentation/openvino-ir-format/operation-sets/operation-specs/movement/strided-slice-1.rst b/docs/articles_en/documentation/openvino-ir-format/operation-sets/operation-specs/movement/strided-slice-1.rst index a4025de9a9f924..242e5a58160c75 100644 --- a/docs/articles_en/documentation/openvino-ir-format/operation-sets/operation-specs/movement/strided-slice-1.rst +++ b/docs/articles_en/documentation/openvino-ir-format/operation-sets/operation-specs/movement/strided-slice-1.rst @@ -14,6 +14,73 @@ StridedSlice **Short description**: *StridedSlice* extracts a strided slice of a tensor. +**Detailed description**: The *StridedSlice* operation extracts a slice from a given tensor based on computed indices from the inputs: begin (inclusive of the element at the given index), end (exclusive of the element at the given index), and stride, for each dimension. + +The operation takes inputs with the following properties: + +* :math:`input` tensor to slice, with N dimensions. +* :math:`begin, end, stride` - 1D lists of integers of the same length M. **Stride input cannot contain any zeros.** +* :math:`begin\_mask, end\_mask, new\_axis\_mask, shrink\_axis\_mask, ellipsis\_mask` - bitmasks, 1D lists of integers (0 or 1). :math:`ellipsis\_mask` can have up to one occurrence of the value 1. **Each mask can have a unique length. The length of the masks can differ from the rank of the input shape.** +* :math:`new\_axis\_mask, shrink\_axis\_mask, ellipsis\_mask` modify the output dimensionality of the data. If they are unused, :math:`N == M`. Otherwise, N does not necessarily equal M. + +.. note:: Negative Values in Begin and End (Negative Values Adjusting) + + Negative values present in :math:`begin` or :math:`end` represent indices starting from the back, i.e., the value of -1 represents the last element of the input dimension. In practice, negative values are automatically incremented by the size of the dimension. For example, if :math:`data = [0, 1, 2, 3]`, :math:`size(data) = 4`, :math:`begin(i) = -1` for some i, this value will be modified to be :math:`begin(i) = -1 + 4 = 3`. Note that if :math:`begin(i) = -5` for some i, this value will be adjusted as follows: :math:`begin(i) -5 + 4 = -1`, which will trigger value clamping. + +The basic slicing operation accumulates output elements as follows: + +* The operation iterates over the values of begin, end, and stride. At every step, the operation uses the i-th element of begin, end, and stride to perform the slicing at the corresponding dimension. +* Let :math:`slicing\_index = begin[i]`. This value determines the first index to start slicing. This sliced element is added to the output. +* If :math:`begin[i] == end[i]`, only a single element from the corresponding dimension is added to the output. The corresponding output dimension is then equal to 1 (in other words, the dimension is kept). +* At each step, the :math:`slicing\_index` is incremented by the value of :math:`stride[i]`. As long as the :math:`slicing\_index < end[i]`, the element corresponding to the :math:`slicing\_index` is added to the output. +* Whenever :math:`slicing\_index >= end[i]`, the slicing stops, and the corresponding element is not added to the output. + +Notice that the basic slicing operation assumes :math:`N == M` (that is, i-th slicing step corresponds to i-th dimension), as no masks are used. + +For the purposes of this specification, assume that :math:`dim` is the dimension corresponding to the i-th slicing step. + +.. note:: Indexing in Reverse (Slicing in Reverse) + + If :math:`stride[i] < 0`, the indexing will happen in reverse. At each step, the value of :math:`stride[i]` will be subtracted from the :math:`slicing\_index`. As long as the :math:`slicing\_index > end[i]`, the corresponding element is added to the output. Whenever :math:`slicing\_index <= end[i]`, the slicing stops. + +.. note:: Value Out-of-Bounds (Silent Clamping) + + If a value in begin or end is out of bounds for the corresponding dimension, it is silently clamped. In other words: + + * If :math:`begin[i] >= size(dim)`, then :math:`begin[i] = size(dim)`. If :math:`begin[i] < 0` (after Negative Values Adjusting), then :math:`begin[i] = 0`. + * If :math:`end[i] >= size(dim)`, then :math:`end[i] = size(dim)`. If :math:`end[i] < 0` (after Negative Values Adjusting), then :math:`end[i] = 0`. + + If slicing in reverse, the clamping behavior changes to the following: + + * If :math:`begin[i] >= size(dim)`, then :math:`begin[i] = size(dim) - 1`. If :math:`begin[i] < 0` (after Negative Values Adjusting), then :math:`begin[i] = 0`. + * If :math:`end[i] >= size(dim)`, then :math:`end[i] = size(dim)`. If :math:`end[i] < 0` (after Negative Values Adjusting), then :math:`end[i] = -1`. + +The operation accepts multiple bitmasks in the form of integer arrays to modify the above behavior. **If the length of the bitmask is less than the length of the corresponding input, it is assumed that the bitmask is extended (padded at the end) with zeros. If the length of the bitmask is greater than necessary, the remaining values are ignored.** + +For examples of usage of each mask, please refer to the examples provided at the end of the document. + +During the i-th slicing step: + +* If the :math:`begin\_mask[i]` is set to one, the value of :math:`begin[i]` is set to :math:`0`` (:math:`size(dim) - 1` if slicing in reverse). Equivalent of swapping left handside of Python slicing operation :math:`array[0:10]` with :math:`array[:10]` (slice from the start). +* If the :math:`end\_mask[i]` is set to one, the value of :math:`end[i]` is set to :math:`size(dim)` (:math:`0` if slicing in reverse - note that this does not allow slicing inclusively with the first value). Equivalent of swapping right handside of Python slicing operation :math:`array[0:10]` (assume :math:`len(array) = 10`) with :math:`array[0:]` (slice till the end, inclusive). +* If the :math:`new\_axis\_mask[i]` is set to one, the values of :math:`begin[i]`, :math:`end[i]`, and :math:`stride[i]` **ARE IGNORED**, and a new dimension with size 1 appears in the output. No slicing occurs at this step. Equivalent of inserting a new dimension into a matrix using numpy :math:`array[..., np.newaxis, ...]`: :math:`shape(array) = [..., 1, ...]`. +* If the :math:`shrink\_axis\_mask[i]` is set to one, the value of :math:`begin[i]` **MUST EQUAL** :math:`end[i]` (Note that this would normally result in a size 1 dimension), and the :math:`stride[i]` value **IS IGNORED**. The corresponding dimension is removed, with only a single element from that dimension remaining. Equivalent of selecting only a given element without preserving dimension (numpy equivalent of keepdims=False) :math:`array[..., 0, ...] -> array[..., ...]` (one less dimension). +* If the :math:`ellipsis\_mask[i]` is set to one, the values of :math:`begin[i], end[i],` and :math:`stride[i]` **ARE IGNORED**, and a number of dimensions is skipped. The exact number of dimensions skipped in the original input is :math:`rank(input) - (M - sum(new\_axis\_mask) - 1)`. The corresponding dimension is treated as an ellipsis ('...'), or in other words, it is treated as multiple, sequential, and unaffected by slicing dimensions, that match the rest of the slicing operation. This allows for a concise and flexible way to perform slicing operations, effectively condensing the slicing parameters for dimensions marked with ellipsis into a single slice notation. For example, given a 10D input, and tasked to select the first element from the 1st and last dimension, normally one would have to write :math:`[0, :, :, :, :, :, :, :, :, :, 0]`, but with ellipsis, it is only necessary to write :math:`[0, ..., 0]`. Equivalent of Equivalent of using the '...' (ellipsis) opeartion in numpy :math:`array[0, ..., 0] (rank(array) = 10)` is the same as writing :math:`array[0, :, :, :, :, :, :, :, :, 0]`. + +.. note:: The i-th Slicing Step and Dimension Modification + + The i-th slicing step does not necessarily correspond to the i-th dimension modification. Let i be the index of the slicing step and j be index of the corresponding processed dimension. + For trivial cases: + + * Every time all of the masks are not set (set to 0), j is incremented by one. + * Every time :math:`begin\_mask[i]` or :math:`end\_mask[i]` is set to one, j is incremented by one. + * Every time :math:`shrink\_axis\_mask[i]` is set to one, j is incremented by one. + + However: + + * Every time :math:`new\_axis\_mask[i]` is set to one, j is not incremented. + * When the value of one occurs at :math:`ellipsis\_mask[i]`, j is incremented by :math:`rank(input) - (M - sum(new\_axis\_mask) - 1)`. + **Attributes** * *begin_mask* @@ -58,15 +125,14 @@ StridedSlice **Inputs**: -* **1**: ``data`` - input tensor to be sliced of type *T* and arbitrary shape. **Required.** - -* **2**: ``begin`` - 1D tensor of type *T_IND* with begin indexes for input tensor slicing. **Required.** - Out-of-bounds values are silently clamped. If ``begin_mask[i]`` is ``1`` , the value of ``begin[i]`` is ignored and the range of the appropriate dimension starts from ``0``. Negative values mean indexing starts from the end. For example, if ``data=[1,2,3]``, ``begin[0]=-1`` means ``begin[0]=3``. +* **1**: ``data`` - input tensor to be sliced of type *T* and arbitrary shape. **Required.** +* **2**: ``begin`` - 1D tensor of type *T_IND* with begin indexes for input tensor slicing. Out-of-bounds values are silently clamped. If ``begin_mask[i]`` is ``1`` , the value of ``begin[i]`` is ignored and the range of the appropriate dimension starts from ``0``. Negative values mean indexing starts from the end. **Required.** +* **3**: ``end`` - 1D tensor of type *T_IND* with end indexes for input tensor slicing. Out-of-bounds values will be silently clamped. If ``end_mask[i]`` is ``1``, the value of ``end[i]`` is ignored and the full range of the appropriate dimension is used instead. Negative values mean indexing starts from the end. **Required.** +* **4**: ``stride`` - 1D tensor of type *T_IND* with strides. If not provided, stride is assumed to be equal to 1. **Optional.** -* **3**: ``end`` - 1D tensor of type *T_IND* with end indexes for input tensor slicing. **Required.** - Out-of-bounds values will be silently clamped. If ``end_mask[i]`` is ``1``, the value of ``end[i]`` is ignored and the full range of the appropriate dimension is used instead. Negative values mean indexing starts from the end. For example, if ``data=[1,2,3]``, ``end[0]=-1`` means ``end[0]=3``. +**Outputs**: -* **4**: ``stride`` - 1D tensor of type *T_IND* with strides. **Optional.** +* **1**: A tensor of type *T* with values selected by the slicing operation according to the rules specified above. **Types** @@ -74,46 +140,82 @@ StridedSlice * *T_IND*: any supported integer type. **Example** -Example of ``begin_mask`` & ``end_mask`` usage. + +Basic example with different strides, standard slicing and in reverse. Equivalent of performing :math:`array[0:4, 1:4, 0:4:2, 1:4:2, 3:0:-1, 3:0:-2]` on a 6D array. .. code-block:: xml :force: - + - 2 - 3 + 4 + 4 + 4 + 4 + 4 4 - 2 + 6 - 2 + 6 - 2 + 6 - 1 - 3 - 2 + 4 + 3 + 2 + 2 + 4 + 2 +Example of clamping in standard and reverse slicing. Equivalent of performing :math:`array[2:3, 2:1:-1]` on a 2D array. -Example of ``new_axis_mask`` usage. +.. code-block:: xml + :force: + + + + + + 2 + 2 + + + 2 + + + 2 + + + 2 + + + + + 1 + 1 + + + + +Example of negative slicing. Equivalent of performing array[0:2, 0:2, 0:-1] on a 3D array. .. code-block:: xml :force: - + 2 @@ -121,58 +223,218 @@ Example of ``new_axis_mask`` usage. 4 - 2 + 3 - 2 + 3 - 2 + 3 - 1 + 2 + 2 + 3 + + + + +Example of ``begin_mask`` & ``end_mask`` usage. Equivalent of performing :math:`array[1:, :, ::-1]` on a 3D array. + +.. code-block:: xml + :force: + + + + + 2 3 4 + + 3 + + + 3 + + + 3 + + + + + 1 + 3 + 3 + -Example of ``shrink_axis_mask`` usage. +Example of ``new_axis_mask`` usage. Equivalent of performing :math:`array[np.newaxis, 0:2, np.newaxis, 0:4]` on a 2D array. .. code-block:: xml :force: - + - 1 2 + 4 + + + 4 + + + 4 + + + 4 + + + + + 1 + 2 + 1 + 4 + + + + +Example of ``shrink_axis_mask`` usage. Equivalent of performing :math:`array[0:1, 0, 0:384, 0:640, 0:8]` on a 5D array. + +.. code-block:: xml + :force: + + + + + + 1 + 2 384 640 8 - 5 + 5 - 5 + 5 - 5 + 5 - 1 - 384 + 1 + 384 640 8 +Example of ``ellipsis_mask`` usage. Equivalent of performing :math:`array[0:4, ..., 0:5]` on a 10D array. + +.. code-block:: xml + :force: + + + + + + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + + + 3 + + + 3 + + + 3 + + + + + 4 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 5 + + + + +Example of ``ellipsis_mask`` usage with other masks of unequal length. Equivalent of performing :math:`array[2:, ..., np.newaxis, :10]` on a 10D array. + +.. code-block:: xml + :force: + + + + + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + + + 3 + + + 3 + + + 3 + + + + + 8 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 10 + 1 + 5 + + +