Bill Allombert on Sun, 14 Mar 2010 10:55:53 +0100 |
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
Re: floor function not doing what it should |
On Sun, Mar 14, 2010 at 06:54:54AM +0100, ewan.Delanoy@math.unicaen.fr wrote: > > Dear PARI-GP users, > > my problem can be summed up in a one-line command : Hello Ewan, > parisize = 4000000, primelimit = 500000 > ? floor(6400^(3/2)) > %1 = 511999 floor is not at fault: ? 6400^(3/2) %1 = 512000.0000000000000000000000 ? \p100 realprecision = 105 significant digits (100 digits displayed) ? %1 %2 = 511999.99999999999999999999999 ? 6400^(3/2)==(512000*(1-2^-95)) %3 = 1 So actually %1 is slightly less than 512000. 6400^(3/2) is computed using the formula exp(3/2*log(6400)) which incurrs some rounding errors. Using the formula sqrt(6400)^3 give a better result: ? sqrt(6400)^3 %2 = 512000.00000000000000000000000000000000 or even ? sqrtint(6400)^3 %3 = 512000 To convert real to i$ntegers, I would suggest you to use round: ? round(6400^(3/2)) %4 = 512000 Cheers, Bill.