|author||nsz <email@example.com>||2012-06-20 23:25:58 +0200|
|committer||nsz <firstname.lastname@example.org>||2012-06-20 23:25:58 +0200|
math: fix fma bug on x86 (found by Bruno Haible with gnulib)
The long double adjustment was wrong: The usual check is mant_bits & 0x7ff == 0x400 before doing a mant_bits++ or mant_bits-- adjustment since this is the only case when rounding an inexact ld80 into double can go wrong. (only in nearest rounding mode) After such a check the ++ and -- is ok (the mantissa will end in 0x401 or 0x3ff). fma is a bit different (we need to add 3 numbers with correct rounding: hi_xy + lo_xy + z so we should survive two roundings at different places without precision loss) The adjustment in fma only checks for zero low bits mant_bits & 0x3ff == 0 this way the adjusted value is correct when rounded to double or *less* precision. (this is an important piece in the fma puzzle) Unfortunately in this case the -- is not a correct adjustment because mant_bits might underflow so further checks are needed and this was the source of the bug.
Diffstat (limited to 'arch/arm')
0 files changed, 0 insertions, 0 deletions