Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C++] Cast to/from halffloat not implemented #20213

Closed
asfimport opened this issue Nov 15, 2018 · 7 comments · Fixed by #40067
Closed

[C++] Cast to/from halffloat not implemented #20213

asfimport opened this issue Nov 15, 2018 · 7 comments · Fixed by #40067
Assignees
@ClifHouck
Labels
Component: C++ good-second-issue Type: enhancement
Milestone
16.0.0

Comments

@asfimport
Copy link
Collaborator

asfimport commented Nov 15, 2018 •
edited
Loading 

library(reticulate)
pa <- import("pyarrow")
pa$array(c(1,2,3))$cast(pa$float16())
#> Error in py_call_impl(callable, dots$args, dots$keywords): ArrowNotImplementedError: No cast implemented from double to halffloat
#> 
#> Detailed traceback: 
#> File "pyarrow/array.pxi", line 277, in pyarrow.lib.Array.cast (/Users/travis/build/BryanCutler/arrow-dist/arrow/python/build/temp.macosx-10.6-intel-3.6/lib.cxx:30459)
#> File "pyarrow/error.pxi", line 85, in pyarrow.lib.check_status (/Users/travis/build/BryanCutler/arrow-dist/arrow/python/build/temp.macosx-10.6-intel-3.6/lib.cxx:8570)

Reporter: Romain Francois / @romainfrancois
Watchers: Rok Mihevc / @rok

Related issues:

Note: This issue was originally created as ARROW-3802. Please see the migration documentation for further details.
@asfimport
Copy link
Collaborator Author

Antoine Pitrou / @pitrou:
We lack an internal facility to interpret half-floats. Currently we merely see them as opaque 16-bit ints.

@asfimport
Copy link
Collaborator Author

Wes McKinney / @wesm:
TensorFlow uses the Eigen::half type to deal with float16. I presume that NumPy has its own implementations of things, since float16 is not implemented in hardware?

@asfimport
Copy link
Collaborator Author

Antoine Pitrou / @pitrou:
Yes, Numpy has dedicated routines.

This was referenced Jan 11, 2023
Closed
Open
Open
Closed
@rok rok mentioned this issue Feb 27, 2023
Closed
@jorisvandenbossche jorisvandenbossche mentioned this issue Jul 18, 2023
Open
@anjakefala anjakefala mentioned this issue Oct 27, 2023
Merged
pitrou pushed a commit that referenced this issue Nov 16, 2023
@anjakefala
MINOR: [Docs] document Float16 support in C++ (#38494)
62cf42b 
### Rationale for this change

There is, at least, partial halffloat support in Arrow C++. A reason to not merge this is that there is still quite a bit casting work to be completed:
* #20213
* #32802

Authored-by: anjakefala <anja@voltrondata.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@pitrou
Copy link
Member

pitrou commented Jan 26, 2024

We should revive this now that we do have a half-float library available.

@github-actions github-actions bot mentioned this issue Feb 13, 2024
Merged
7 tasks
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
@anjakefala @dgreiss
MINOR: [Docs] document Float16 support in C++ (apache#38494)
bbcbcc6 
### Rationale for this change

There is, at least, partial halffloat support in Arrow C++. A reason to not merge this is that there is still quite a bit casting work to be completed:
* apache#20213
* apache#32802

Authored-by: anjakefala <anja@voltrondata.com>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@ianmcook ianmcook added this to the 16.0.0 milestone Apr 2, 2024
@kou
Copy link
Member

kou commented Apr 4, 2024

@ClifHouck Could you add "take" only comment here to assign this issue to you?

kou pushed a commit that referenced this issue Apr 4, 2024
@ClifHouck
GH-20213: [C++] Implement cast to/from halffloat (#40067)
72d20ad 
### Rationale for this change

### What changes are included in this PR?

This PR implements casting to and from float16 types using the vendored float16 library included in arrow at `cpp/arrrow/util/float16.*`.

### Are these changes tested?

Unit tests are included in this PR.

### Are there any user-facing changes?

In that casts to and from float16 will now work, yes.

* Closes: #20213

### TODO

- [x] Add casts to/from float64.
- [x] String <-> float16 casts.
- [x] Integer <-> float16 casts.
- [x] Tests.
- [x] Update https://github.com/apache/arrow/blob/main/docs/source/status.rst about half float.
- [x] Rebase.
- [x] Run clang format over this PR.
* GitHub Issue: #20213

Authored-by: Clif Houck <me@clifhouck.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@kou kou closed this as completed in #40067 Apr 4, 2024
@ClifHouck
Copy link
Contributor

take

@github-actions github-actions bot mentioned this issue Apr 8, 2024
Merged
tolleybot pushed a commit to tmct/arrow that referenced this issue May 2, 2024
@ClifHouck @tolleybot
apacheGH-20213: [C++] Implement cast to/from halffloat (apache#40067)
4c6c4aa 
### Rationale for this change

### What changes are included in this PR?

This PR implements casting to and from float16 types using the vendored float16 library included in arrow at `cpp/arrrow/util/float16.*`.

### Are these changes tested?

Unit tests are included in this PR.

### Are there any user-facing changes?

In that casts to and from float16 will now work, yes.

* Closes: apache#20213

### TODO

- [x] Add casts to/from float64.
- [x] String <-> float16 casts.
- [x] Integer <-> float16 casts.
- [x] Tests.
- [x] Update https://github.com/apache/arrow/blob/main/docs/source/status.rst about half float.
- [x] Rebase.
- [x] Run clang format over this PR.
* GitHub Issue: apache#20213

Authored-by: Clif Houck <me@clifhouck.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
tolleybot pushed a commit to tmct/arrow that referenced this issue May 4, 2024
@ClifHouck @tolleybot
apacheGH-20213: [C++] Implement cast to/from halffloat (apache#40067)
9a35c56 
### Rationale for this change

### What changes are included in this PR?

This PR implements casting to and from float16 types using the vendored float16 library included in arrow at `cpp/arrrow/util/float16.*`.

### Are these changes tested?

Unit tests are included in this PR.

### Are there any user-facing changes?

In that casts to and from float16 will now work, yes.

* Closes: apache#20213

### TODO

- [x] Add casts to/from float64.
- [x] String <-> float16 casts.
- [x] Integer <-> float16 casts.
- [x] Tests.
- [x] Update https://github.com/apache/arrow/blob/main/docs/source/status.rst about half float.
- [x] Rebase.
- [x] Run clang format over this PR.
* GitHub Issue: apache#20213

Authored-by: Clif Houck <me@clifhouck.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
rok pushed a commit to tmct/arrow that referenced this issue May 8, 2024
@ClifHouck @rok
apacheGH-20213: [C++] Implement cast to/from halffloat (apache#40067)
06ce746 
### Rationale for this change

### What changes are included in this PR?

This PR implements casting to and from float16 types using the vendored float16 library included in arrow at `cpp/arrrow/util/float16.*`.

### Are these changes tested?

Unit tests are included in this PR.

### Are there any user-facing changes?

In that casts to and from float16 will now work, yes.

* Closes: apache#20213

### TODO

- [x] Add casts to/from float64.
- [x] String <-> float16 casts.
- [x] Integer <-> float16 casts.
- [x] Tests.
- [x] Update https://github.com/apache/arrow/blob/main/docs/source/status.rst about half float.
- [x] Rebase.
- [x] Run clang format over this PR.
* GitHub Issue: apache#20213

Authored-by: Clif Houck <me@clifhouck.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
rok pushed a commit to tmct/arrow that referenced this issue May 8, 2024
@ClifHouck @rok
apacheGH-20213: [C++] Implement cast to/from halffloat (apache#40067)
dcbc2d1 
### Rationale for this change

### What changes are included in this PR?

This PR implements casting to and from float16 types using the vendored float16 library included in arrow at `cpp/arrrow/util/float16.*`.

### Are these changes tested?

Unit tests are included in this PR.

### Are there any user-facing changes?

In that casts to and from float16 will now work, yes.

* Closes: apache#20213

### TODO

- [x] Add casts to/from float64.
- [x] String <-> float16 casts.
- [x] Integer <-> float16 casts.
- [x] Tests.
- [x] Update https://github.com/apache/arrow/blob/main/docs/source/status.rst about half float.
- [x] Rebase.
- [x] Run clang format over this PR.
* GitHub Issue: apache#20213

Authored-by: Clif Houck <me@clifhouck.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
vibhatha pushed a commit to vibhatha/arrow that referenced this issue May 25, 2024
@ClifHouck @vibhatha
apacheGH-20213: [C++] Implement cast to/from halffloat (apache#40067)
2e2259c 
### Rationale for this change

### What changes are included in this PR?

This PR implements casting to and from float16 types using the vendored float16 library included in arrow at `cpp/arrrow/util/float16.*`.

### Are these changes tested?

Unit tests are included in this PR.

### Are there any user-facing changes?

In that casts to and from float16 will now work, yes.

* Closes: apache#20213

### TODO

- [x] Add casts to/from float64.
- [x] String <-> float16 casts.
- [x] Integer <-> float16 casts.
- [x] Tests.
- [x] Update https://github.com/apache/arrow/blob/main/docs/source/status.rst about half float.
- [x] Rebase.
- [x] Run clang format over this PR.
* GitHub Issue: apache#20213

Authored-by: Clif Houck <me@clifhouck.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@felipecrv
Copy link
Contributor

Some casts (e.g. half-float -> boolean) were not implemented (#43017). I opened a PR (#43018) to fix these.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ClifHouck ClifHouck

Labels
Component: C++ good-second-issue Type: enhancement
Projects
None yet
Milestone
16.0.0
Development

Successfully merging a pull request may close this issue.

GH-20213: [C++] Implement cast to/from halffloat ClifHouck/arrow
6 participants
@kou @felipecrv @ianmcook @pitrou @ClifHouck @asfimport