Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect significant figure rounding #308

Open
Markos-Th09 opened this issue Sep 4, 2024 · 5 comments
Open

Incorrect significant figure rounding #308

Markos-Th09 opened this issue Sep 4, 2024 · 5 comments
Labels
bug Something isn't working

Comments

@Markos-Th09
Copy link
Contributor

Markos-Th09 commented Sep 4, 2024

When rounding using significant figures, the number is truncated instead of being rounded. For example

12.9 to 2 sf

outputs 12 instead of 13

@printfn
Copy link
Owner

printfn commented Sep 4, 2024

Thanks for the report, this would definitely be good to fix.

@printfn printfn added the bug Something isn't working label Sep 4, 2024
@Markos-Th09
Copy link
Contributor Author

also to <n> dp is also truncating but I don't what is the intended behavior of that

@BoolPurist
Copy link

@printfn Do you intent to fix it by yourself in the near future ?

If not.
I would be willing to fix it. However, after looking into the code for that, I came to the conclusion that making sf rounding work will take me a while to get it right.

What are thoughts about it ?

@Markos-Th09
Copy link
Contributor Author

@printfn also should regular rounding be preferred over to truncation for the to dp rounding mode? If so, should a truncation rounding mode be introduced?

@printfn
Copy link
Owner

printfn commented Sep 28, 2024

I think both to dp and to sf should round correctly. Supporting optional truncation would be nice too, but less important than getting the rounding right.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants