From 639789dc57a61900872bcc1f718643a6e80ea000 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sun, 22 Sep 2019 16:04:25 -0700 Subject: [PATCH 1/7] Initial draft of KHR_quantized_geometry --- .../Khronos/KHR_quantized_geometry/README.md | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) create mode 100644 extensions/2.0/Khronos/KHR_quantized_geometry/README.md diff --git a/extensions/2.0/Khronos/KHR_quantized_geometry/README.md b/extensions/2.0/Khronos/KHR_quantized_geometry/README.md new file mode 100644 index 0000000000..aad08a54dc --- /dev/null +++ b/extensions/2.0/Khronos/KHR_quantized_geometry/README.md @@ -0,0 +1,82 @@ +# KHR\_quantized\_geometry + +## Contributors + +* Arseny Kapoulkine, [@zeuxcg](https://twitter.com/zeuxcg) + +## Status + +Draft (not ratified yet) + +## Dependencies + +Written against the glTF 2.0 spec. Depends on `KHR_texture_transform` for texture coordinate dequantization. + +## Overview + +Vertex attributes are usually stored using `FLOAT` component type. However, this can result in excess precision and increased memory consumption and transmission size, as well as reduced rendering performance. + +This extension expands the set of allowed component types for geometry storage to provide a memory/precision tradeoff - depending on the application needs, 16-bit or 8-bit storage can be sufficient. + +Using 16-bit or 8-bit storage typically requires transforming the original floating point values to fit a uniform 3D or 2D grid; the process is commonly known as quantization. + +To simplify implementation requirements, the extension relies on existing ways to specify geometry transformation instead of adding special dequantization transforms to the schema. + +As an example, a static PBR-ready mesh typically requires `POSITION` (12 bytes), `TEXCOORD` (8 bytes), `NORMAL` (12 bytes) and `TANGENT` (16 bytes) for each vertex, for a total of 48 bytes. With this extension, it is possible to use `SHORT` to store position and texture coordinate data (8 and 4 bytes, respectively) and `BYTE` to store normal and tangent data (4 bytes each), for a total of 20 bytes per vertex with often negligible quality impact. + +Because the extension does not provide a way to specify both `FLOAT` and quantized versions of the data, files that use the extension should specify it in `extensionsRequired` list - the extension is not optional. + +## Extending Mesh Attributes + +When `KHR_quantized_geometry` extension is supported, the set of types used for storing geometry attributes is expanded according to the table below. + +|Name|Accessor Type(s)|Component Type(s)|Description| +|----|----------------|-----------------|-----------| +|`POSITION`|`"VEC3"`|`5126` (FLOAT)
`5120` (BYTE)
`5120` (BYTE) normalized
`5121` (UNSIGNED_BYTE)
`5121` (UNSIGNED_BYTE) normalized
`5122` (SHORT)
`5122` (SHORT) normalized
`5123` (UNSIGNED_SHORT)
`5123` (UNSIGNED_SHORT) normalized|XYZ vertex positions| +|`NORMAL`|`"VEC3"`|`5126` (FLOAT)
`5120` (BYTE) normalized
`5122` (SHORT) normalized|Normalized XYZ vertex normals| +|`TANGENT`|`"VEC4"`|`5126` (FLOAT)
`5120` (BYTE) normalized
`5122` (SHORT) normalized|XYZW vertex tangents where the *w* component is a sign value (-1 or +1) indicating handedness of the tangent basis| +|`TEXCOORD_n`|`"VEC2"`|`5126` (FLOAT)
`5120` (BYTE)
`5120` (BYTE) normalized
`5121` (UNSIGNED_BYTE)
`5121` (UNSIGNED_BYTE) normalized
`5122` (SHORT)
`5122` (SHORT) normalized
`5123` (UNSIGNED_SHORT)
`5123` (UNSIGNED_SHORT) normalized|UV texture coordinates for set #n| + +Note that to comply with alignment rules for accessors, `"VEC3"` accessors need to be aligned to 4-byte boundaries; for example, a `BYTE` normal is expected to have a stride of 4, not 3. + +For `POSITION` and `TEXCOORD` attributes, the application is free to choose normalized or unnormalized storage, as well as signed or unsigned. When normalized storage is used, often the data doesn't have to be dequantized (which eliminates the need for a dequantization transform); however, if the data is not in `[0..1]` or `[-1..1]` range, using integer storage can reduce precision loss as standard glTF normalization factors such as `1/255` and `1/65535` are not repersentable exactly as floating-point numbers. + +> **Implementation Note:** As quantization may introduce a non-negligible error, quantized normal and tangent vectors are typically not exactly unit length. Applications are expected to normalize the vectors before using them in lighting equations; this typically can be done after transforming them using the normal matrix. Even if quantization is not used, normal matrix can contain scale/skew so normalization is typically required anyway. + +## Extending Morph Target Attributes + +When `KHR_quantized_geometry` extension is supported, the set of types used for storing morph target attributes is expanded according to the table below. + +|Name|Accessor Type(s)|Component Type(s)|Description| +|----|----------------|-----------------|-----------| +|`POSITION`|`"VEC3"`|`5126` (FLOAT)
`5120` (BYTE)
`5120` (BYTE) normalized
`5122` (SHORT)
`5122` (SHORT) normalized|XYZ vertex position displacements| +|`NORMAL`|`"VEC3"`|`5126` (FLOAT)
`5120` (BYTE) normalized
`5122` (SHORT) normalized|XYZ vertex normal displacements| +|`TANGENT`|`"VEC3"`|`5126` (FLOAT)
`5120` (BYTE) normalized
`5122` (SHORT) normalized|XYZ vertex tangent displacements| + +Note that to comply with alignment rules for accessors, `"VEC3"` accessors need to be aligned to 4-byte boundaries; for example, a `BYTE` normal is expected to have a stride of 4, not 3. + +For simplicity the specification assumes that morph target displacements are signed; for `NORMAL` and `TANGENT`, models with very large displacements that do not fit into `[-1..1]` range need to use `FLOAT` storage. + +## Decoding Quantized Data + +Depending on the attribute values and component types used, the attributes may need to be transformed into their original range for display. Instead of relying on separate dequantization transform, this extension relies on existing methods to specify transformation. + +For `POSITION` attribute, the transform can be specified in one of two ways: + +- For non-skinned meshes, the dequantization transform (which typically consists of scale and offset) can be encoded into `node` transformation - this can require adding an extra leaf `node` so that animations that affect the mesh transformation are not disturbed. + +- For skinned meshes, the `node` hierarchy does not affect the transform of the `mesh` directly - however, dequantization transform can be encoded into `inverseBindMatrices` for the `skin` used for the mesh. + +In both cases, to preserve the direction of normal/tangent vectors, it is recommended that the quantization scale specified in the transform is uniform across X/Y/Z axes. + +For `TEXCOORD` attribute, the transform can be specified using the `offset`/`scale` supplied by `KHR_texture_transform` extension. + +For morph target data, it's expected that deltas are quantized using the same transform as the base positions. Extra care must be taken to ensure that the quantization accomodates for large displacements, if present. + +## Encoding Quantized Data + +It is up to the encoder to determine the optimal range and format, along with the quantization transform, to maximize the output quality. If necessary, different meshes can use different component types for different attributes - this extension does not require that a single component type is used consistently for all attributes of the same type. + +> **Implementation Note:** It is possible to determine the range of the attribute for each mesh individually, and pick the 8-bit or 16-bit storage format accordingly; this minimizes the error for each individual mesh, but may prevent sharing for some scene elements (such as reusing a material across multiple meshes with different texture coordinate ranges) and may result in the same vertex being quantized differently, causing visible gaps. + +> **Implementation Note:** It is also possible to determine the range of all attributes of all meshes in the scene and use a shared dequantization transform; this can eliminate the gaps between different meshes but can increase the quantization error. \ No newline at end of file From e8d0c8aaac38c3ca1175d0882efa212b00ce7ba3 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 23 Sep 2019 19:50:35 -0700 Subject: [PATCH 2/7] Add float/int conversion details --- .../2.0/Khronos/KHR_quantized_geometry/README.md | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-) diff --git a/extensions/2.0/Khronos/KHR_quantized_geometry/README.md b/extensions/2.0/Khronos/KHR_quantized_geometry/README.md index aad08a54dc..6d73aca29c 100644 --- a/extensions/2.0/Khronos/KHR_quantized_geometry/README.md +++ b/extensions/2.0/Khronos/KHR_quantized_geometry/README.md @@ -79,4 +79,15 @@ It is up to the encoder to determine the optimal range and format, along with th > **Implementation Note:** It is possible to determine the range of the attribute for each mesh individually, and pick the 8-bit or 16-bit storage format accordingly; this minimizes the error for each individual mesh, but may prevent sharing for some scene elements (such as reusing a material across multiple meshes with different texture coordinate ranges) and may result in the same vertex being quantized differently, causing visible gaps. -> **Implementation Note:** It is also possible to determine the range of all attributes of all meshes in the scene and use a shared dequantization transform; this can eliminate the gaps between different meshes but can increase the quantization error. \ No newline at end of file +> **Implementation Note:** It is also possible to determine the range of all attributes of all meshes in the scene and use a shared dequantization transform; this can eliminate the gaps between different meshes but can increase the quantization error. + +Implementations should assume following equations are used to get corresponding floating-point value `f` from a normalized integer `c` and should use the specified equations to encode floating-point values to integers after range normalization: + +|`accessor.componentType`|int-to-float|float-to-int| +|-----------------------------|--------|----------------| +| `5120` (BYTE) |`f = max(c / 127.0, -1.0)`|`c = round(f * 127.0)`| +| `5121` (UNSIGNED_BYTE) |`f = c / 255.0`|`c = round(f * 255.0)`| +| `5122` (SHORT) |`f = max(c / 32767.0, -1.0)`|`c = round(f * 32767.0)`| +| `5123` (UNSIGNED_SHORT)|`f = c / 65535.0`|`c = round(f * 65535.0)`| + +> **Implementation Note:** Due to OpenGL ES 2.0 / WebGL 1.0 restrictions, some implementations may decode signed normalized integers to floating-point values differently. While the difference is unlikely to be significant for normal/tangent data, implementations may want to use unsigned normalized or signed non-normalized formats to avoid the discrepancy in position data. \ No newline at end of file From 3f5c782c230652b89c73a20785fabc664a990be7 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sun, 29 Sep 2019 18:27:49 -0700 Subject: [PATCH 3/7] Minor tweaks: - Added @lexaknyazev to contributor list following the text fixes suggested in PR discussion - Fix a couple of spelling errors - Clarified wording on data alignment - BYTE TEXCOORDs need to be aligned to 4 bytes as well so there's no need to highlight VEC3. --- extensions/2.0/Khronos/KHR_quantized_geometry/README.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_quantized_geometry/README.md b/extensions/2.0/Khronos/KHR_quantized_geometry/README.md index 6d73aca29c..a717e086a4 100644 --- a/extensions/2.0/Khronos/KHR_quantized_geometry/README.md +++ b/extensions/2.0/Khronos/KHR_quantized_geometry/README.md @@ -3,6 +3,7 @@ ## Contributors * Arseny Kapoulkine, [@zeuxcg](https://twitter.com/zeuxcg) +* Alexey Knyazev, [@lexaknyazev](https://github.com/lexaknyazev) ## Status @@ -37,9 +38,9 @@ When `KHR_quantized_geometry` extension is supported, the set of types used for |`TANGENT`|`"VEC4"`|`5126` (FLOAT)
`5120` (BYTE) normalized
`5122` (SHORT) normalized|XYZW vertex tangents where the *w* component is a sign value (-1 or +1) indicating handedness of the tangent basis| |`TEXCOORD_n`|`"VEC2"`|`5126` (FLOAT)
`5120` (BYTE)
`5120` (BYTE) normalized
`5121` (UNSIGNED_BYTE)
`5121` (UNSIGNED_BYTE) normalized
`5122` (SHORT)
`5122` (SHORT) normalized
`5123` (UNSIGNED_SHORT)
`5123` (UNSIGNED_SHORT) normalized|UV texture coordinates for set #n| -Note that to comply with alignment rules for accessors, `"VEC3"` accessors need to be aligned to 4-byte boundaries; for example, a `BYTE` normal is expected to have a stride of 4, not 3. +Note that to comply with alignment rules for accessors, each element needs to be aligned to 4-byte boundaries; for example, a `BYTE` normal is expected to have a stride of 4, not 3. -For `POSITION` and `TEXCOORD` attributes, the application is free to choose normalized or unnormalized storage, as well as signed or unsigned. When normalized storage is used, often the data doesn't have to be dequantized (which eliminates the need for a dequantization transform); however, if the data is not in `[0..1]` or `[-1..1]` range, using integer storage can reduce precision loss as standard glTF normalization factors such as `1/255` and `1/65535` are not repersentable exactly as floating-point numbers. +For `POSITION` and `TEXCOORD` attributes, the application is free to choose normalized or unnormalized storage, as well as signed or unsigned. When normalized storage is used, often the data doesn't have to be dequantized (which eliminates the need for a dequantization transform); however, if the data is not in `[0..1]` or `[-1..1]` range, using integer storage can reduce precision loss as standard glTF normalization factors such as `1/255` and `1/65535` are not representable exactly as floating-point numbers. > **Implementation Note:** As quantization may introduce a non-negligible error, quantized normal and tangent vectors are typically not exactly unit length. Applications are expected to normalize the vectors before using them in lighting equations; this typically can be done after transforming them using the normal matrix. Even if quantization is not used, normal matrix can contain scale/skew so normalization is typically required anyway. @@ -71,7 +72,7 @@ In both cases, to preserve the direction of normal/tangent vectors, it is recomm For `TEXCOORD` attribute, the transform can be specified using the `offset`/`scale` supplied by `KHR_texture_transform` extension. -For morph target data, it's expected that deltas are quantized using the same transform as the base positions. Extra care must be taken to ensure that the quantization accomodates for large displacements, if present. +For morph target data, it's expected that deltas are quantized using the same transform as the base positions. Extra care must be taken to ensure that the quantization accommodates for large displacements, if present. ## Encoding Quantized Data @@ -90,4 +91,4 @@ Implementations should assume following equations are used to get corresponding | `5122` (SHORT) |`f = max(c / 32767.0, -1.0)`|`c = round(f * 32767.0)`| | `5123` (UNSIGNED_SHORT)|`f = c / 65535.0`|`c = round(f * 65535.0)`| -> **Implementation Note:** Due to OpenGL ES 2.0 / WebGL 1.0 restrictions, some implementations may decode signed normalized integers to floating-point values differently. While the difference is unlikely to be significant for normal/tangent data, implementations may want to use unsigned normalized or signed non-normalized formats to avoid the discrepancy in position data. \ No newline at end of file +> **Implementation Note:** Due to OpenGL ES 2.0 / WebGL 1.0 restrictions, some implementations may decode signed normalized integers to floating-point values differently. While the difference is unlikely to be significant for normal/tangent data, implementations may want to use unsigned normalized or signed non-normalized formats to avoid the discrepancy in position data. From 7303b3ec941d062da3f13ff9dc7764d3b2c692ee Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sun, 10 Nov 2019 16:20:00 -0800 Subject: [PATCH 4/7] Tweak wording: - should => must - WebGL restrictions => platform differences --- extensions/2.0/Khronos/KHR_quantized_geometry/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_quantized_geometry/README.md b/extensions/2.0/Khronos/KHR_quantized_geometry/README.md index a717e086a4..70767997ea 100644 --- a/extensions/2.0/Khronos/KHR_quantized_geometry/README.md +++ b/extensions/2.0/Khronos/KHR_quantized_geometry/README.md @@ -25,7 +25,7 @@ To simplify implementation requirements, the extension relies on existing ways t As an example, a static PBR-ready mesh typically requires `POSITION` (12 bytes), `TEXCOORD` (8 bytes), `NORMAL` (12 bytes) and `TANGENT` (16 bytes) for each vertex, for a total of 48 bytes. With this extension, it is possible to use `SHORT` to store position and texture coordinate data (8 and 4 bytes, respectively) and `BYTE` to store normal and tangent data (4 bytes each), for a total of 20 bytes per vertex with often negligible quality impact. -Because the extension does not provide a way to specify both `FLOAT` and quantized versions of the data, files that use the extension should specify it in `extensionsRequired` list - the extension is not optional. +Because the extension does not provide a way to specify both `FLOAT` and quantized versions of the data, files that use the extension must specify it in `extensionsRequired` array - the extension is not optional. ## Extending Mesh Attributes @@ -91,4 +91,4 @@ Implementations should assume following equations are used to get corresponding | `5122` (SHORT) |`f = max(c / 32767.0, -1.0)`|`c = round(f * 32767.0)`| | `5123` (UNSIGNED_SHORT)|`f = c / 65535.0`|`c = round(f * 65535.0)`| -> **Implementation Note:** Due to OpenGL ES 2.0 / WebGL 1.0 restrictions, some implementations may decode signed normalized integers to floating-point values differently. While the difference is unlikely to be significant for normal/tangent data, implementations may want to use unsigned normalized or signed non-normalized formats to avoid the discrepancy in position data. +> **Implementation Note:** Due to OpenGL ES 2.0 / WebGL 1.0 platform differences, some implementations may decode signed normalized integers to floating-point values differently. While the difference is unlikely to be significant for normal/tangent data, implementations may want to use unsigned normalized or signed non-normalized formats to avoid the discrepancy in position data. From 3ba27de21ee045de2badbedcdb0dab521ed254a4 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sun, 10 Nov 2019 16:24:09 -0800 Subject: [PATCH 5/7] Add a note re: texture transform offset/scale. --- extensions/2.0/Khronos/KHR_quantized_geometry/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/extensions/2.0/Khronos/KHR_quantized_geometry/README.md b/extensions/2.0/Khronos/KHR_quantized_geometry/README.md index 70767997ea..1451e40f65 100644 --- a/extensions/2.0/Khronos/KHR_quantized_geometry/README.md +++ b/extensions/2.0/Khronos/KHR_quantized_geometry/README.md @@ -70,7 +70,7 @@ For `POSITION` attribute, the transform can be specified in one of two ways: In both cases, to preserve the direction of normal/tangent vectors, it is recommended that the quantization scale specified in the transform is uniform across X/Y/Z axes. -For `TEXCOORD` attribute, the transform can be specified using the `offset`/`scale` supplied by `KHR_texture_transform` extension. +For `TEXCOORD` attribute, the transform can be specified using the `offset`/`scale` supplied by `KHR_texture_transform` extension. Note that this requires merging existing `offset`/`scale` values, if present, with the dequantization transform. For morph target data, it's expected that deltas are quantized using the same transform as the base positions. Extra care must be taken to ensure that the quantization accommodates for large displacements, if present. From 652533306d7ac128418f98456d598d5081482328 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 13 Nov 2019 19:44:27 -0800 Subject: [PATCH 6/7] Rename KHR_quantized_geometry to KHR_mesh_quantization --- .../README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) rename extensions/2.0/Khronos/{KHR_quantized_geometry => KHR_mesh_quantization}/README.md (95%) diff --git a/extensions/2.0/Khronos/KHR_quantized_geometry/README.md b/extensions/2.0/Khronos/KHR_mesh_quantization/README.md similarity index 95% rename from extensions/2.0/Khronos/KHR_quantized_geometry/README.md rename to extensions/2.0/Khronos/KHR_mesh_quantization/README.md index 1451e40f65..1299c63129 100644 --- a/extensions/2.0/Khronos/KHR_quantized_geometry/README.md +++ b/extensions/2.0/Khronos/KHR_mesh_quantization/README.md @@ -1,4 +1,4 @@ -# KHR\_quantized\_geometry +# KHR\_mesh\_quantization ## Contributors @@ -29,7 +29,7 @@ Because the extension does not provide a way to specify both `FLOAT` and quantiz ## Extending Mesh Attributes -When `KHR_quantized_geometry` extension is supported, the set of types used for storing geometry attributes is expanded according to the table below. +When `KHR_mesh_quantization` extension is supported, the set of types used for storing geometry attributes is expanded according to the table below. |Name|Accessor Type(s)|Component Type(s)|Description| |----|----------------|-----------------|-----------| @@ -46,7 +46,7 @@ For `POSITION` and `TEXCOORD` attributes, the application is free to choose norm ## Extending Morph Target Attributes -When `KHR_quantized_geometry` extension is supported, the set of types used for storing morph target attributes is expanded according to the table below. +When `KHR_mesh_quantization` extension is supported, the set of types used for storing morph target attributes is expanded according to the table below. |Name|Accessor Type(s)|Component Type(s)|Description| |----|----------------|-----------------|-----------| From 39bece7df9cf0b4a800afecfe78f179968ab87d3 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Wed, 13 Nov 2019 19:46:08 -0800 Subject: [PATCH 7/7] Replace "geometry" with "mesh" in a couple of places --- extensions/2.0/Khronos/KHR_mesh_quantization/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/extensions/2.0/Khronos/KHR_mesh_quantization/README.md b/extensions/2.0/Khronos/KHR_mesh_quantization/README.md index 1299c63129..f8f7f3be4a 100644 --- a/extensions/2.0/Khronos/KHR_mesh_quantization/README.md +++ b/extensions/2.0/Khronos/KHR_mesh_quantization/README.md @@ -17,7 +17,7 @@ Written against the glTF 2.0 spec. Depends on `KHR_texture_transform` for textur Vertex attributes are usually stored using `FLOAT` component type. However, this can result in excess precision and increased memory consumption and transmission size, as well as reduced rendering performance. -This extension expands the set of allowed component types for geometry storage to provide a memory/precision tradeoff - depending on the application needs, 16-bit or 8-bit storage can be sufficient. +This extension expands the set of allowed component types for mesh attribute storage to provide a memory/precision tradeoff - depending on the application needs, 16-bit or 8-bit storage can be sufficient. Using 16-bit or 8-bit storage typically requires transforming the original floating point values to fit a uniform 3D or 2D grid; the process is commonly known as quantization. @@ -29,7 +29,7 @@ Because the extension does not provide a way to specify both `FLOAT` and quantiz ## Extending Mesh Attributes -When `KHR_mesh_quantization` extension is supported, the set of types used for storing geometry attributes is expanded according to the table below. +When `KHR_mesh_quantization` extension is supported, the set of types used for storing mesh attributes is expanded according to the table below. |Name|Accessor Type(s)|Component Type(s)|Description| |----|----------------|-----------------|-----------|