Enable Non-determinism of float operations in Miri and change std tests #138062

LorrensP-2158466 · 2025-03-05T14:37:39Z

Links to #4208 and #3555 in Miri.

Non-determinism of floating point operations was disabled in #137594 because it breaks the tests and doc-tests in core/coretests and std.

This PR:

enables the float non-determinism but with a lower relative error of 4ULP instead of 16ULP
changes tests that made incorrect assumptions about the operations not to make that assumption anymore (from assert_eq! to assert_approx_eq!.
changes the assert_approx_eq! macro to allow up to 1e-4 to make the tests pass

TODO:

I didn't touch the doc tests because I do not know nearly enough to come near them :)
probably change the assert_approx_eq to use the same technique as Miri (i.e., using ULP instead of EPSILON)

try-job: x86_64-gnu-aux

rustbot · 2025-03-05T14:37:44Z

r? @tgross35

rustbot has assigned @tgross35.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

rustbot · 2025-03-05T14:37:46Z

The Miri subtree was changed

cc @rust-lang/miri

tgross35 · 2025-03-05T22:55:41Z

The library changes lgtm, for the rest

r? @RalfJung

src/tools/miri/tests/pass/float.rs

LorrensP-2158466 · 2025-03-06T11:20:06Z

I didn't touch the doc tests because I do not know nearly enough to come near them :)

I should have clarified: the doc tests fail when running ./x miri --doc --no-fail-fast core coretests std -- f64 f32 because of the extra 4ULP error.

src/tools/miri/src/intrinsics/mod.rs

RalfJung · 2025-03-09T19:19:46Z

src/tools/miri/src/intrinsics/mod.rs

+                        2, // log2(4)
+                    );
+
+                    // Clamp values to the output range defined in IEEE 754 9.1.


We said we're going to follow the C standard, which is more permissive than the IEEE spec. Is there a reference for this in the C standard?

Yes and no...

For some operations it specifies it like this:

The atan functions return arctan x in the interval [−π/2, +π/2] radians

But for sin and cos like this:

The sin functions return sin x

If I read "returns sin x", I understand it as "the output is [-1, +1]", but maybe that's just me.

src/tools/miri/src/intrinsics/mod.rs

RalfJung · 2025-03-09T19:20:50Z

src/tools/miri/src/intrinsics/mod.rs

+                let fixed_res = match (f1.category(), f2.category()) {
+                    // 1^y = 1 for any y even a NaN.
+                    // TODO: C Standard says any NaN, IEEE says not a Signaling NaN
+                    (Category::Normal, _) if f1 == 1.0f32.to_soft() => Some(1.0f32.to_soft()),


Can you avoid using soft floats here?

In the miri issue you said this:

except that the logic for fixed_res and clamp has to be done with soft-floats.

Did I misunderstand what you meant?

Argh sorry, I meant "avoid using hard floats" :)

RalfJung · 2025-03-09T19:22:11Z

src/tools/miri/src/intrinsics/mod.rs

+                let fixed_res = match (f1.category(), f2.category()) {
+                    // 1^y = 1 for any y even a NaN.
+                    // TODO: C says any NaN, IEEE says no a Sign NaN
+                    (Category::Normal, _) if f1 == 1.0f64.to_soft() => Some(1.0f64.to_soft()),


Please do not duplicate the same logic 4 times. We were a bit lazy here so far as the boilerplate for making the code generic was bigger than the code, but with all this special-case handling that is no longer the case.

src/tools/miri/src/intrinsics/mod.rs

RalfJung · 2025-03-09T19:23:05Z

src/tools/miri/tests/pass/float.rs

-        // accept up to 64ULP (16ULP for host floats and 16ULP for miri artificial error and 32 for any rounding errors)
-        assert_approx_eq!($a, $b, 64);
+        // accept up to 52ULP (16ULP for host floats, 4ULP for miri artificial error and 32 for any rounding errors)
+        assert_approx_eq!($a, $b, 52);


Do we really still need 52 ULP? I would have hoped that 32 works now.

Indeed 32 works, don't know why I didn't tried it.

I tried lower but got some ULP differences of 27.

RalfJung · 2025-03-09T19:23:41Z

src/tools/miri/tests/pass/float.rs

+    // TODO: How to test NaN inputs? f*::NAN is not guaranteed
+    // to be any specific bit pattern (in std).


It is guaranteed to be a NaN though. Why do you want a specific bit pattern?

IEEE is not as permissive as C, so it doesn't matter much now, just one case: C standard says for pown(powi) the following:

pown(x, 0) returns 1 for all x not a signaling NaN

And since std doesn't guarantee the specific bit pattern of NaN, I'm not quite sure how to test it.

Oh, since NAN might be signaling? That'd be quite bad.^^

Please rely on it being not signalling, I'll see if we can fix the docs.

src/tools/miri/tests/pass/float.rs

RalfJung · 2025-03-09T19:25:42Z

I should have clarified: the doc tests fail when running ./x miri --doc --no-fail-fast core coretests std -- f64 f32 because of the extra 4ULP error.

it is failing the tests that use f32::EPSILON, right? Can you change that to 4.0 * f32::EPSILON (or maybe more if needed)? Once we know the change that makes the tests pass, we can ask the libs folks if that is reasonable.

RalfJung · 2025-03-09T19:27:12Z

library/std/tests/floats/lib.rs

 macro_rules! assert_approx_eq {
-    ($a:expr, $b:expr) => {{ assert_approx_eq!($a, $b, 1.0e-6) }};
+    ($a:expr, $b:expr) => {{ assert_approx_eq!($a, $b, 1.0e-4) }};


I think I'd rather not change this for all tests, and in particular not for f64.

An alternative would be to just set this precision with the affected tests in library/std/tests/floats/f32.rs.

Maybe the precision should be only loosened only with cfg!(miri) as well. I think it is beneficial to learn which platforms, if any, have worse precision than we expect and address that on a case-by-case basis.

LorrensP-2158466 · 2025-03-10T19:23:37Z

So I:

only used 1e-4 on failed tests in std, so 1e-6 is back.
increased epsilon threshold in the failed doctests to 4.0 or 8.0
used a macro instead of the duplicate logic in the pow intrinsic. I didn't find another way to cleanly do this + I don't know where I should put this macro.

RalfJung · 2025-03-10T19:33:35Z

No, a macro is not right. The way to avoid this duplication is to add a function that is generic over the apfloat float type.

I'd say just remove pow from the PR for now, let's use sin/cos to test out the waters. Already there the code duplication bothers me but it's less bad.

LorrensP-2158466 · 2025-03-10T19:41:50Z

Yeah Sorry, I was confused with your comment: avoid using soft floats. It should be easier with soft floats. I can try deduplicating sin/cos and pow, and if it isn't what you're looking for, I'll remove them.

RalfJung · 2025-03-10T19:47:23Z

I was confused with your comment: avoid using soft floats.

sorry for that :(

LorrensP-2158466 · 2025-03-10T19:56:20Z

No problems! This float stuff it getting to me too :)

LorrensP-2158466 · 2025-03-11T19:10:58Z

So I extended the fixed_float_value to accept powf, but this required accepting a slice of arguments and matching on the size. This pattern matching is getting extensive, but I don't find it confusing atm. powi accepts i32, so I made a separate function to handle that one.

There is still some repetitiveness, like applying the error or adjust_nan, but that's not the focus of this pr.

RalfJung · 2025-03-19T14:53:32Z

src/tools/miri/src/intrinsics/mod.rs

@@ -522,3 +572,121 @@ fn apply_random_float_error_to_imm<'tcx>(

    interp_ok(ImmTy::from_scalar_int(res, val.layout))
 }
+
+// TODO(lorrens): This can be moved to `helpers` when we implement the other intrinsics.


No please only but things in helpers that are widely useful across Miri. Specific functionality should remain in its specific file.

Alright, I thought that this would be the case when implementing the foreign_itmes, since they probably will share this functionality, like asin and such.

Ah yeah... we'll figure that out when we get there.

RalfJung · 2025-03-19T14:54:46Z

src/tools/miri/src/intrinsics/mod.rs

+        // x^(±0) = 1 for any x, even a NaN
+        ("powf32" | "powf64", [_, exp]) if exp.is_zero() => Some(one),
+
+        // C standard doesn't specify or invalid combination


This is not a sentence...?

Yeah, now that I am reading that again... I'll update it, excuse me for my bad English :)

RalfJung · 2025-03-19T14:55:16Z

src/tools/miri/src/intrinsics/mod.rs

+    // TODO: not sure about this pattern matching stuff. It's definitly cleaner than if-else chains
+    // Error code 0158 explains this: https://doc.rust-lang.org/stable/error_codes/E0158.html
+    // The only reason I did this is to use the same function for powf as for sin/cos/exp/log


I don't understand the comment. The code looks nice though :)

Okay, I should have explained why the error code applies here.
Consider this arm:

// sin(+- 0) = +- 0. ("sinf32" | "sinf64", [input]) if input.is_zero() => Some(*input),

I still have to put an if-guard on it to check that input is Zero, but in my opinion, it would be a lot nicer if I could do the following:

// sin(+- 0) = +- 0. ("sinf32" | "sinf64", [IeeeFloat::<S>::Zero]) => Some(*input),

If I then were able to make a constant for 1 and maybe other values, I wouldn't need those if-guards. But unfortunately, the compiler doesn't try to see if this can work; it just assumes this to be non-exhaustive, regardless of a _ => ... arm.

Luckily, you like the code :)

You can't use consts that depend on generics in patterns like that, that is expected.

src/tools/miri/src/intrinsics/mod.rs

RalfJung · 2025-03-19T14:56:32Z

src/tools/miri/tests/pass/float.rs

+    // TODO: How to test NaN inputs? f*::NAN is not guaranteed
+    // to be any specific bit pattern (in std).


Oh, since NAN might be signaling? That'd be quite bad.^^

Please rely on it being not signalling, I'll see if we can fix the docs.

RalfJung · 2025-03-19T14:58:17Z

Regarding the library tests, I suggested a plan above for the doc tests. Have you tried that?

EDIT: Ah, yes seems you did. :) I assume all tests pass now in the current state of the PR?

Also:

probably change the assert_approx_eq to use the same technique as Miri (i.e., using ULP instead of EPSILON)

I am not convinced we want to do that, so unless @tgross35 asks for it I'd say please stick to the current approach.

RalfJung · 2025-03-19T15:00:28Z

@tgross35 could you have a look at the library diff again?
If you prefer, instead of hard-coding 1e-4 I guess we could also have a constant like

// Miri adds some extra error to float functions, make sure the tests still pass.
const APPROX_DETLA: f32 = if cfg!(miri) { 1e-4 } else { 1e-6 };

LorrensP-2158466 · 2025-03-19T19:15:49Z

EDIT: Ah, yes seems you did. :) I assume all tests pass now in the current state of the PR?

Yes, they indeed pass now.

On the topic of doc and doctests, for example, powf:

// part of powf doctest
let x = 2.0_f32;
let abs_difference = (x.powf(2.0) - (x * x)).abs();
assert!(abs_difference <= 8.0 * f32::EPSILON);

This is not wrong, but shouldn't it be noted that this only works with small values?
And I think this applies to a lot of operations.

RalfJung · 2025-03-30T18:34:31Z

r? @tgross35

to reflect the current status. Feel free to re-assign to me if the libs changes are fine.

LorrensP-2158466 · 2025-04-02T11:20:28Z

@RalfJung should this PR also alter the algebraic operations that were mentioned in #136457?

RalfJung · 2025-04-02T15:41:52Z

Let's avoid a conflict with the other in-flight PR, #136457. So let's deal with the algebraic and fast-math operations separately.

RalfJung · 2025-05-27T13:34:27Z

@rustbot author

bors · 2025-05-30T02:53:08Z

☔ The latest upstream changes (presumably #141739) made this pull request unmergeable. Please resolve the merge conflicts.

LorrensP-2158466 · 2025-05-30T08:17:46Z

Alright, so I have updated lots of tests, because Ralf's comment about using -Zmiri-many-seeds made me think about the current status. So I ran the following commands for f32 and f64 (env is used because I use fish):

env MIRIFLAGS=-Zmiri-many-seeds=0..100 ./x miri --no-doc --no-fail-fast std core coretests -- f32
# and docs
env MIRIFLAGS=-Zmiri-many-seeds=0..100 ./x miri --doc --no-fail-fast std core -- f32

And some tests still failed, especially for f32. So I took all the doctests that I changed (only these failed) and put them in a separate file, ran Miri on it with Zmiri-many-seeds and tried to find the limit to each test.

So now:

Use fixed constants in the doctests where necessary.
Changed some tests from assert_eq to assert_approx_eq.
Added more APPROX_DELTA to some tests that failed using many seeds.

Before pushing, I tested everything again with this command:

env MIRIFLAGS=-Zmiri-many-seeds ./x miri --no-fail-fast std core coretests -- f32 f64

And it succeeded (yes, this took some time :)).

The float tests from Miri also succeeded with -Zmiri-many-seeds.

LorrensP-2158466 · 2025-05-30T08:20:59Z

library/coretests/tests/floats/f32.rs

+/// Miri adds some extra errors to float functions; make sure the tests still pass.
+/// These values are purely used as a canary to test against and are thus not a stable guarantee Rust provides.
+/// They serve as a way to get an idea of the real precision of floating point operations on different platforms.
+const APPROX_DELTA: f32 = if cfg!(miri) { 1e-4 } else { 1e-6 };


I only use this once in this file, but it's the same reason as in library/std/tests/floats/f32.rs.

bors · 2025-05-30T09:25:51Z

☔ The latest upstream changes (presumably #141753) made this pull request unmergeable. Please resolve the merge conflicts.

LorrensP-2158466 · 2025-05-30T09:30:04Z

That's fun, I fix conflicts, push, and test. And then 2 more conflicts.

Anyway, nothing changed, and the tests were successful. So they should pass now as well.

@rustbot ready

RalfJung · 2025-05-30T10:08:26Z

src/tools/miri/tests/pass/float.rs

+    let halve_pi_single = std::f32::consts::FRAC_PI_2;
+    let halve_pi_double = std::f64::consts::FRAC_PI_2;
+    let pi_single = std::f32::consts::PI;
+    let pi_double = std::f64::consts::PI;


Please don't use "single"/"double" terminology, always prefer f32/f64.

RalfJung · 2025-05-30T10:11:18Z

This LGTM, thanks! Some of the intermediate commits are odd though, such as abbb5d1 which removes a whole bunch of tests. Please clean up the history.

Also, given that there were more doc tests changes: @tgross35, it'd be good if you could take another look at the library/ diff.

LorrensP-2158466 · 2025-05-30T10:23:07Z

Some of the intermediate commits are odd though, such as abbb5d1 which removes a whole bunch of tests. Please clean up the history.

Yeah, that was a dumb mistake of mine, during the squash and rebase, I made the mistake of thinking those tests were needed in those files, but they were done in core. So I removed them and made a separate commit

With "clean up the history", do you mean squashing that commit into the previous one? Or just a full squash?

RalfJung · 2025-05-30T10:24:15Z

I mean to squash them until each commit makes independent sense. That may mean a full squash :)

LorrensP-2158466 · 2025-05-30T10:44:11Z

I think the history is okay now, the first one is the original squash, when my branch was severely outdated. The rest are logical commits I wanted to do.

RalfJung · 2025-05-30T11:58:14Z

Commits 2 and 3 should IMO be merged, there's no reason to have this back-and-forth in the history. Just land the final result immediately.

Commit 4 can be merged into the first one.

RalfJung · 2025-05-30T14:14:34Z

@bors2 try

rust-bors · 2025-05-30T14:14:39Z

⌛ Trying commit 998f16c with merge c26092c…

…<try> Enable Non-determinism of float operations in Miri and change std tests    Links to [#4208](rust-lang/miri#4208) and [#3555](rust-lang/miri#3555) in Miri. Non-determinism of floating point operations was disabled in #137594 because it breaks the tests and doc-tests in core/coretests and std. This PR: - enables the float non-determinism but with a lower relative error of 4ULP instead of 16ULP - changes tests that made incorrect assumptions about the operations not to make that assumption anymore (from `assert_eq!` to `assert_approx_eq!`. - changes the `assert_approx_eq!` macro to allow up to 1e-4 to make the tests pass TODO: - I didn't touch the doc tests because I do not know nearly enough to come near them :) - probably change the `assert_approx_eq` to use the same technique as Miri (i.e., using ULP instead of EPSILON) try-job: x86_64-gnu-aux

pietroalbini · 2025-05-30T15:13:10Z

@bors2 help

rust-bors · 2025-05-30T15:13:12Z

r+ [p=<priority>] [rollup=<never|iffy|maybe|always>]: Approve this PR. Optionally, you can specify <priority>, <rollup>.
r=<user> [p=<priority>]: Approve this PR on behalf of <user>. Optionally, you can specify a <priority>.
r-: Unapprove this PR
p=<priority>: Set the priority of this PR
delegate=<try|review>: Delegate permissions to the PR author
delegate+: Delegate review permissions to the PR author
delegate-: Remove any previously granted delegation
try [parent=<parent>] [jobs=<jobs>]: Start a try build. Optionally, you can specify a <parent> SHA or a list of <jobs> to run
try cancel: Cancel a running try build
rollup=<never|iffy|maybe|always>: Mark the rollup status of the PR
info: Get information about the current PR including delegation, priority, merge status, and try build status
ping: Check if the bot is alive
help: Print this help message
treeclosed-, treeopen: Open the repository tree for merging
treeclosed=<priority>: Close the tree for PRs with priority less than <priority>

change tests in std, core and coretests.

LorrensP-2158466 · 2025-05-30T15:48:15Z

Thanks for the git help, Ralf!

rust-bors · 2025-05-30T15:57:56Z

☀️ Try build successful

CI ✅
Build commit: c26092c (c26092c87347e6f1b2ac51cdd250508b249529d9)

rustbot assigned tgross35 Mar 5, 2025

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-libs Relevant to the library team, which will review and decide on the PR/issue. labels Mar 5, 2025

LorrensP-2158466 changed the title ~~Enable Non-determinism of float operations and change std tests~~ Enable Non-determinism of float operations in Miri and change std tests Mar 5, 2025

rustbot assigned RalfJung and unassigned tgross35 Mar 5, 2025

Dante-Broggi reviewed Mar 6, 2025

View reviewed changes

src/tools/miri/tests/pass/float.rs Outdated Show resolved Hide resolved

RalfJung reviewed Mar 9, 2025

View reviewed changes

RalfJung reviewed Mar 19, 2025

View reviewed changes

rustbot assigned tgross35 and unassigned RalfJung Mar 30, 2025

RalfJung mentioned this pull request Mar 31, 2025

Expose algebraic floating point intrinsics #136457

Merged

rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels May 27, 2025

LorrensP-2158466 force-pushed the miri-enable-float-nondet branch from bddfc69 to 02f0817 Compare May 30, 2025 08:09

LorrensP-2158466 commented May 30, 2025

View reviewed changes

LorrensP-2158466 force-pushed the miri-enable-float-nondet branch from 02f0817 to 7a4dd13 Compare May 30, 2025 09:28

rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels May 30, 2025

RalfJung reviewed May 30, 2025

View reviewed changes

LorrensP-2158466 force-pushed the miri-enable-float-nondet branch from 7a4dd13 to 998f16c Compare May 30, 2025 10:40

RalfJung assigned tgross35 May 30, 2025

LorrensP-2158466 added 2 commits May 30, 2025 17:47

Enable Float non-determinism in miri. Update and add tests and change

1480b92

change tests in std, core and coretests.

change tests to use fixed constants to let them pass with miri

e9c64a4

LorrensP-2158466 force-pushed the miri-enable-float-nondet branch from 998f16c to e9c64a4 Compare May 30, 2025 15:47

		// TODO: How to test NaN inputs? f*::NAN is not guaranteed
		// to be any specific bit pattern (in std).

Enable Non-determinism of float operations in Miri and change std tests #138062

Are you sure you want to change the base?

Enable Non-determinism of float operations in Miri and change std tests #138062

Conversation

LorrensP-2158466 commented Mar 5, 2025 • edited by RalfJung Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rustbot commented Mar 5, 2025

Uh oh!

rustbot commented Mar 5, 2025

Uh oh!

tgross35 commented Mar 5, 2025

Uh oh!

Uh oh!

LorrensP-2158466 commented Mar 6, 2025

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

RalfJung commented Mar 9, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

LorrensP-2158466 commented Mar 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

RalfJung commented Mar 10, 2025

Uh oh!

LorrensP-2158466 commented Mar 10, 2025

Uh oh!

RalfJung commented Mar 10, 2025 via email

Uh oh!

LorrensP-2158466 commented Mar 10, 2025

Uh oh!

LorrensP-2158466 commented Mar 11, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

LorrensP-2158466 commented Mar 5, 2025 •

edited by RalfJung

Loading

LorrensP-2158466 commented Mar 10, 2025 •

edited

Loading

RalfJung commented Mar 19, 2025 •

edited

Loading

LorrensP-2158466 commented May 30, 2025 •

edited

Loading

RalfJung commented May 30, 2025 •

edited

Loading