summaryrefslogtreecommitdiff
path: root/src/math
diff options
context:
space:
mode:
authorSzabolcs Nagy <nsz@port70.net>2016-09-04 04:51:03 +0200
committerRich Felker <dalias@aerifal.cx>2016-10-20 13:29:34 -0400
commit6ffdc4579ffb34f4aab69ab4c081badabc7c0a9a (patch)
tree1ad511f097d1d0d999b2f9377b5027c99886ef7d /src/math
parentd184a09e0529f33d8ddddb8825039133483a2c41 (diff)
downloadmusl-6ffdc4579ffb34f4aab69ab4c081badabc7c0a9a.tar.gz
fix strtod int optimization in non-nearest rounding mode
the mid-sized integer optimization relies on lnz set up properly to mark the last non-zero decimal digit, but this was not done if the non-zero digit lied outside the KMAX digits of the base 10^9 number representation. so if the fractional part was a very long list of zeros (>2048*9 on x86) followed by non-zero digits then the integer optimization could kick in discarding the tiny non-zero fraction which can mean wrong result on non-nearest rounding mode. strtof, strtod and strtold were all affected.
Diffstat (limited to 'src/math')
0 files changed, 0 insertions, 0 deletions