Walter Neumann on Fri, 13 May 2005 16:00:20 +0200


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: erfc() behavior change


Thanks for the clarification. I think you are saying that for a zero real only the precision is relevant. But the following is then still confusing:

? 0.e-10
%1 = 0.E-38
? precision(%)
%2 = 48
? %1*10^40
%3 = 0.E1
? precision(%)
%4 = 0

I would expect the answers to be 0.E-48, 48, 0.E-10, 10 respectively.

--walter

On Thu, 12 May 2005, Karim Belabas wrote:

* Walter Neumann [2005-05-12 17:15]:
There does seem to be a related incosistency however:

? erfc(-2^11)
%1 = 2.000000000000000000000000000
? 2-%1
%2 = 0.E-1821573
? precision(%2)
%3 = 1821573

...

? 0.0
%10 = 0.E-28
? precision(%)
%11 = 28
? 0.0e-90
%12 = 0.E-91
? precision(%)
%13 = 96
? 1.e-90
%14 = 1.000000000000000000000000000 E-90
? precision(%)
%15 = 28

Shouldn't precision in %3 and %13 be something like 28?

'precision' is not really defined in the documentation (should be...).

* For a basic type not involving floating point components, precision(x)
is +oo [ well, an absurdly large number, strictly larger than the precision
of any inexact object. ]

* For a _non_zero_ t_REAL x, precision(x) is the number of significant digits
for _non_zero_ x.  This is only approximately true: various radix
conversions and assorted roundings are involved.

* For a _zero_ t_REAL x, precision(x) is max(- expo_10(x), 0).
There are no significant digits in this case.

* For a t_COMPLEX, precision(x + I*y) is (roughly) precision(|x| + |y|).

* For another recursive object, precision(x) is the min of all involved
precisions.


The precision of

 X*(1. to 100 significant digits) + 0.e-1000

is 100.

 X*(1. to 100 significant digits) + 0.e+1000

is 0.

Hope this helps,

   Karim.