Igor Schein on Tue, 18 Feb 2003 20:22:02 -0500


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

precision question 

Hi,

? precision(511/10^3+.)
28
? precision(512/10^3+.)
38

Looks liking adding 1 extra binary place pushes the precision to the
next level.  I'm curious in principle, why rational-to-float
conversion of a small rational results in higher-than-working
precision. 

Thanks

Igor