Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Spec][Internal][Op] Specification for RMS internal op #24564

Merged
merged 5 commits into from
May 21, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
.. {#openvino_docs_ops_internal_RMS}

RMS
===


.. meta::
:description: Learn about RMS a normalization operation.

**Versioned name**: *RMS*

**Category**: *Normalization*

**Short description**: Calculates Root Mean Square (RMS) normalization of the input tensor.

**Detailed description**

*RMS* operation performs Root Mean Square (RMS) normalization on a given input ``data`` along the last dimension of the input.
`Reference <https://arxiv.org/abs/1910.07467>`__.


.. code-block:: py

(x / Sqrt(ReduceMean(x^2, -1) + eps)) * scale


**Attributes**

* *epsilon*

* **Description**: A very small value added to the variance for numerical stability. Ensures that division by zero does not occur for any normalized element.
* **Range of values**: a positive floating-point number
* **Type**: ``float``
* **Required**: *yes*

* *output_type*

* **Description**: The precision for output type conversion, after scaling. It's used for output type compression to f16.
* **Range of values**: Supported floating point type: "f16", "undefined"
* **Type**: ``string``
* **Default value**: "undefined" (means that output type is set to the same as the input type)
* **Required**: *no*


**Inputs**

* **1**: ``data`` - Input data to be normalized. A tensor of type *T* and arbitrary shape. **Required.**

* **2**: ``scale`` - A tensor of type *T* containing the scale values for . The shape should be broadcastable to the shape of ``data`` tensor. **Required.**


**Outputs**

* **1**: Output tensor of the same shape as the ``data`` input tensor and type specified by *output_type* attribute.

**Types**

* *T*: any floating point type.

**Example**

.. code-block:: xml
:force:

<layer ... type="RMS"> <!-- normalization always over the last dimension [-1] -->
<data eps="1e-6"/>
<input>
<port id="0">
<dim>12</dim>
<dim>25</dim>
<dim>512</dim>
</port>
<port id="1">
<dim>512</dim>
</port>
</input>
<output>
<port id="2">
<dim>12</dim>
<dim>25</dim>
<dim>512</dim>
</port>
</output>
</layer>
Loading