Bill Allombert on Tue, 4 Mar 2003 14:10:43 +0100 |
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
Re: precision question |
On Tue, Feb 18, 2003 at 08:22:02PM -0500, Igor Schein wrote: > Hi, > > ? precision(511/10^3+.) > 28 > ? precision(512/10^3+.) > 38 > > Looks liking adding 1 extra binary place pushes the precision to the > next level. I'm curious in principle, why rational-to-float > conversion of a small rational results in higher-than-working > precision. The "correct" way of converting rationals and intergers to reals is to do: N*1., not N+0. N*1. return N with the default precision, when N+. return N with precision sizedigit(N)+defaultprecision. If you see reals numbers as range, this is logical: If the error is eps=10^-prec, x. stand for [x*(1.-eps),x*(1.+eps)], and 0 stand for [-eps,+eps]. N*1. give us [x*(1.-eps),x*(1.+eps)]=N. but N+. give us [N-eps,N+eps]=[N*(1-eps/N),N*(1+eps/N)] so we have now an error of only eps/N, so a precisio of prec+log(N)/log(10). Cheers, Bill