diff options
author | Szabolcs Nagy <nsz@port70.net> | 2016-09-04 04:51:03 +0200 |
---|---|---|
committer | Rich Felker <dalias@aerifal.cx> | 2016-10-20 13:29:34 -0400 |
commit | 6ffdc4579ffb34f4aab69ab4c081badabc7c0a9a (patch) | |
tree | 1ad511f097d1d0d999b2f9377b5027c99886ef7d /src/math | |
parent | d184a09e0529f33d8ddddb8825039133483a2c41 (diff) | |
download | musl-6ffdc4579ffb34f4aab69ab4c081badabc7c0a9a.tar.gz |
fix strtod int optimization in non-nearest rounding mode
the mid-sized integer optimization relies on lnz set up properly
to mark the last non-zero decimal digit, but this was not done
if the non-zero digit lied outside the KMAX digits of the base
10^9 number representation.
so if the fractional part was a very long list of zeros (>2048*9 on
x86) followed by non-zero digits then the integer optimization could
kick in discarding the tiny non-zero fraction which can mean wrong
result on non-nearest rounding mode.
strtof, strtod and strtold were all affected.
Diffstat (limited to 'src/math')
0 files changed, 0 insertions, 0 deletions