[js/common] allows using Uint16Array as data for float16 tensor

[fs-eire]

Description

Resolve #23817

Motivation and Context

[fdwr]

Thanks Yulong.

[fs-eire]

@Honry @xenova Could you please help to check if this PR fixes the problem? It should work but I don't have a chance to test it E2E.

Artifacts can be downloaded at https://dev.azure.com/onnxruntime/onnxruntime/_build/results?buildId=1629021&view=artifacts&pathAsName=false&type=publishedArtifacts

[Honry]

@fs-eire, verified, thanks for the quick fix.

[xenova]

Thanks for the fix! For transformers.js, we'll also default to Float16Array if available: #23817

[Honry]

@fs-eire, hold on, this still breaks demos that read the outputs as Uint16Array. Now you always save the output to Float16Array if Float16Array is available, however existing demos still treat it as Uint16Array.

[fdwr]

@fs-eire, hold on, this still breaks demos that read the outputs as Uint16Array. Now you always save the output to Float16Array if Float16Array is available, however existing demos still treat it as Uint16Array.

Ooh, that's more complicated. How does ORT know what the output type should be? It could look at the input types to deduce the output types (e.g. if there were uint16-as-float16 input tensors, then presumably any output float16 tensors should be uint16 too), but it wouldn't be fully robust (e.g. say you had a model that took int32 inputs and float16-as-uint16 outputs, which there is no input hint to know the output type). Well, it might be a good enough transient heuristic for now.

[Honry]

Ooh, that's more complicated. How does ORT know what the output type should be? It could look at the input types to deduce the output types (e.g. if there were uint16-as-float16 input tensors, then presumably any output float16 tensors should be uint16 too), but it wouldn't be fully robust (e.g. say you had a model that took int32 inputs and float16-as-uint16 outputs, which there is no input hint to know the output type). Well, it might be a good enough transient heuristic for now.

Exactly, it’s difficult to know what the output type should be. Users have to add API check to determine if Float16Array should be used.

[fs-eire]

Ooh, that's more complicated. How does ORT know what the output type should be? It could look at the input types to deduce the output types (e.g. if there were uint16-as-float16 input tensors, then presumably any output float16 tensors should be uint16 too), but it wouldn't be fully robust (e.g. say you had a model that took int32 inputs and float16-as-uint16 outputs, which there is no input hint to know the output type). Well, it might be a good enough transient heuristic for now.

Exactly, it’s difficult to know what the output type should be. Users have to add API check to determine if Float16Array should be used.

as explained by @fdwr, I think there is no way to do this inside ORT

to fix this problem, you need to modify the application code. a simple way is that for all float16 output, you always create a Uint16Array wrapper:

if (myTensor.type === 'float16') {
   myData = new Uint16Array(myTensor.data.buffer, myTensor.data.byteOffset, myTensor.data.length);
}

[fs-eire]

@Honry Please let me know if you are OK with this change or we need further discussion

[Honry]

@Honry Please let me know if you are OK with this change or we need further discussion

Make sense, and we should remind end users to change their application code if need.

[fdwr]

@Honry Please let me know if you are OK with this change or we need further discussion

Make sense, and we should remind end users to change their application code if need.

Sounds worth adding to the breaking changes section: https://github.com/microsoft/webnn-developer-preview?tab=readme-ov-file#breaking-changes

[Honry]

👍