Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add f16 formatting and parsing #127013

Draft
wants to merge 4 commits into
base: master
Choose a base branch
from
Draft

Add f16 formatting and parsing #127013

wants to merge 4 commits into from
+598 −95

Conversation

tgross35
Copy link
Contributor

WIP

r? @ghost

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Jun 27, 2024
@rust-log-analyzer

This comment has been minimized.

Sign in to view
@tgross35
Copy link
Contributor Author

@rustbot label +rla-silenced

@rustbot rustbot added the rla-silenced Silences rust-log-analyzer postings to the PR it's added on. label Jun 27, 2024
@tgross35
Copy link
Contributor Author

@rustbot label +F-f16_and_f128

@rustbot rustbot added the F-f16_and_f128 `#![feature(f16)]`, `#![feature(f128)]` label Jun 27, 2024
@tgross35
Copy link
Contributor Author

tgross35 commented Jul 9, 2024

This will need #127510

@bors
Copy link
Contributor

bors commented Jul 15, 2024

☔ The latest upstream changes (presumably #127020) made this pull request unmergeable. Please resolve the merge conflicts.

@tgross35 tgross35 mentioned this pull request Jul 18, 2024
Open
86 tasks
@tgross35 tgross35 force-pushed the f16-format-parse branch 2 times, most recently from 7c3f9c1 to f3ebeb3 Compare July 23, 2024 09:52
@tgross35
Copy link
Contributor Author

Update: I'm really just waiting on #128083 to bump stage0, managing #[cfg(not(bootstrap))] and always building stage1 got too painful.

@bors
Copy link
Contributor

bors commented Jul 30, 2024

☔ The latest upstream changes (presumably #128360) made this pull request unmergeable. Please resolve the merge conflicts.

@Dylan-DPC Dylan-DPC added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 20, 2024
@tgross35 tgross35 force-pushed the f16-format-parse branch 6 times, most recently from 21ffabc to 2098f01 Compare August 25, 2024 07:53
@rustbot rustbot added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Aug 25, 2024
@tgross35 tgross35 force-pushed the f16-format-parse branch 5 times, most recently from 404089f to 3636530 Compare August 30, 2024 09:04
jieyouxu
jieyouxu reviewed Mar 1, 2025
View reviewed changes
src/etc/test-float-parse/src/lib.rs
@@ -1,3 +1,5 @@
#![feature(f16)]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding this will require changing

rust/src/bootstrap/src/core/build_steps/tool.rs

Line 1243 in 8c39296

allow_features: "",

to allow f16 btw.

@rustbot rustbot added the A-testsuite Area: The testsuite used to check the correctness of rustc label Mar 2, 2025
@tgross35
Copy link
Contributor Author

tgross35 commented Mar 2, 2025

@speedy-lex I got this rebased on top of #134063 which should hopefully merge reasonably soon.

Any chance you are interested in taking this over? I'm happy to keep going of course, but it seems like you may have already done some of the work :)

@speedy-lex
Copy link
Contributor

I think I will rewrite some of the tests in ./library/coretests/tests/num/flt2dec/mod.rs for f16. Also if you want to use this, I made a calculator for f16 (which is useful for the exact_sanity_test): https://www.desmos.com/calculator/ikaghp8on9
This is another useful tool: https://evanw.github.io/float-toy/

@tgross35
Copy link
Contributor Author

tgross35 commented Mar 2, 2025

I think I will rewrite some of the tests in ./library/coretests/tests/num/flt2dec/mod.rs for f16.

That would be great; I'm also happy to add any patches here that you work on.

I think I will rewrite some of the tests in ./library/coretests/tests/num/flt2dec/mod.rs for f16. Also if you want to use this, I made a calculator for f16 (which is useful for the exact_sanity_test): https://www.desmos.com/calculator/ikaghp8on9
This is another useful tool: https://evanw.github.io/float-toy/

Awesome, more float tools! My usuals are https://float.exposed/0x0001 and https://weitz.de/ieee/ :)

(Am I supposed to see something on the desmos plot?)

@speedy-lex
Copy link
Contributor

No, the desmos plot should be empty, the float is just the result of that summation.

@speedy-lex
Copy link
Contributor

speedy-lex commented Mar 2, 2025 •
edited
Loading

@tgross35, how do you want me to contribute my changes? Should I make a PR on your repo?

Edit: I'm working on the flt2dec tests

@tgross35
Copy link
Contributor Author

tgross35 commented Mar 2, 2025

Post links to the relevant commits / branch here and I’ll pick them up 👍 or feel free to pick up my commits and open a new PR here

@speedy-lex
Copy link
Contributor

speedy-lex commented Mar 3, 2025 •
edited
Loading

@tgross35, I've ported the tests for flt2dec over to f16. This is the commit: speedy-lex@09643f9

Edit: Could you try add test-float-parse to x.py suggest?

@speedy-lex
Copy link
Contributor

PR for fixing off by one error in test-float-parse #137948

speedy-lex
speedy-lex reviewed Mar 3, 2025
View reviewed changes
src/etc/test-float-parse/src/gen/subnorm.rs Outdated
@@ -83,7 +82,13 @@ where
}

fn new() -> Self {
Self { iter: F::Int::ZERO..=min(F::Int::ONE << 22, F::MAN_BITS.try_into().unwrap()) }
let upper_lim = if F::MAN_BITS < 22 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure this check is correct?
I have a feeling it might need to be F::MAN_BITS >= 22

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might also be why https://github.com/rust-lang/rust/actions/runs/13614968407/job/38056857277?pr=127013 is failing (F::MAN_BITS is 10 so F::Int::ONE << 22 runs, but 1u16 << 22 will overflow. This aligns with the error of overflow on shl)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure this check is correct?

Nope :) I think I ran this all once or twice then got distracted and never came back. Thanks for pointing it out, I'll double check it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks wrong but works for some reason (except for in the CI job)

@tgross35
Copy link
Contributor Author

tgross35 commented Mar 4, 2025

Edit: Could you try add test-float-parse to x.py suggest?

I don't think I've ever used this and I have no idea how it works 😆 any improvements to running test-float-parse via bootstrap are welcome though, I wired it up just enough to run tests in CI.

@speedy-lex
Copy link
Contributor

I'm not very familiar with the CI system; can you link me to where you wired it up?
I've added it to suggest with speedy-lex@e0ca633 but I'm not sure how to get suggest to run ./test-float-parse --skip-huge

@jieyouxu
Copy link
Member

jieyouxu commented Mar 4, 2025

./x suggest is mostly stale as I don't think anyone actually uses it.

@speedy-lex
Copy link
Contributor

Doesn't CI use it?

@jieyouxu
Copy link
Member

jieyouxu commented Mar 4, 2025 •
edited
Loading

It does not AFAIK. It was mostly added to help local developers. #109933

I don't think it's been worked on for a while.

@speedy-lex
Copy link
Contributor

speedy-lex commented Mar 4, 2025 •
edited
Loading

What's the system for registering CI commands?
Where is it? I've searched through the many.sh files and I can't find anything.

@jieyouxu
Copy link
Member

jieyouxu commented Mar 4, 2025 •
edited
Loading

The CI goes through some shell wrappers x.ps1 or x.sh to invoke the x.py python wrapper1 which in turn invokes the bootstrap python wrapper bootstrap.py to build the bootstrap crate (incl. the bootstrap binary) itself, which is the build system.

What do you want to do with "registering CI commands"? CI also goes through bootstrap.

Footnotes

  1. they are to facade away platform shell/python differences. ↩

@tgross35
Copy link
Contributor Author

tgross35 commented Mar 7, 2025

Thanks @speedy-lex for the tests, almost everything is passing (including test-float-parse!). I have a few todos to address before the final cleanup.

What's the system for registering CI commands? Where is it? I've searched through the many.sh files and I can't find anything.

Bootstrap can be set up to build a path as a crate, ./x t --stage src/etc/test-float-parse and (I think) ./x t --stage test-float-parse should work. If you search for test-float-parse within src/bootstrap it should point you to the right place, something like impl Step for TestFloatParse. These specific CI invocation is here

rust/src/bootstrap/mk/Makefile.in

Lines 51 to 55 in 91a0e16

$(Q)$(BOOTSTRAP) test --stage 2 \
src/tools/cargo \
src/tools/cargotest \
src/etc/test-float-parse \
$(BOOTSTRAP_ARGS)

@speedy-lex
Copy link
Contributor

@tgross35, I've figured out CI on my own already but thanks for commenting.

Also, what else needs to be done before this is finished?

@tgross35
Copy link
Contributor Author

tgross35 commented Mar 7, 2025

There are a few small places where I added tests for consistency but didn’t yet update them, labeled todo:. I’ll get to them tonight, but feel free to beat me to it :)

After that this should be review-ready

tgross35
tgross35 commented Mar 8, 2025
View reviewed changes
library/coretests/tests/num/flt2dec/mod.rs Outdated
check_exact!(f(f16::MIN_POSITIVE) => b"6103515625 ", -4);
check_exact!(f(minf16) => b"59604644775390625", -7);

// todo: add tables from f32 and f64
Copy link
Contributor Author

@tgross35 tgross35 Mar 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the last thing remaining to do, I think it might take a bit of work to apply the algorithms from the Paxson paper to binary16. @speedy-lex have you touched this at all?

(I'm working on it, thinking the described algorithm would be good in test-float-parse)

tgross35 and others added 3 commits March 8, 2025 03:39
@tgross35
float: Add f16 parsing and printing
d12f15b 
Use the existing Lemire (decimal -> float) and Dragon / Grisu algorithms
(float -> decimal) to add support for `f16`. This allows updating the
implementation for `Display` to the expected behavior for `Display`
(currently it prints the a hex bitwise representation) and adds a
`FromStr` implementation.
@tgross35 @speedy-lex
float: Add tests for f16 conversions to and from decimal
f55bff2 
Extend the existing tests for `f32` and `f64` with versions that include
`f16`'s new printing and parsing implementations.

Co-authored-by: Speedy_Lex <alex.ciocildau@gmail.com>
@tgross35
float: Add f16 to test-float-parse
3bb0af3 
This requires a fix to the subnormal test to cap the maximum allowed
value within the maximum mantissa.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@fbstj fbstj fbstj left review comments

@jieyouxu jieyouxu jieyouxu left review comments

@speedy-lex speedy-lex speedy-lex requested changes

Assignees
No one assigned
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc F-f16_and_f128 `#![feature(f16)]`, `#![feature(f128)]` rla-silenced Silences rust-log-analyzer postings to the PR it's added on. S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@tgross35 @rust-log-analyzer @bors @speedy-lex @jieyouxu @fbstj @rustbot @Dylan-DPC