Damien Wyart on Wed, 30 Jul 2003 22:38:22 +0200 |
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
Truncating real numbers |
Hello, I need to truncate (and round) reals to keep only n digits after the decimal separator. This is both for display and for evaluation. I would like not to interact with default precision (this is quite heavy to change it all the time). That is : if a is 2.7646535646854 I both need to output a with (say) 4 digits after ., and to store this truncated (or rounded) number in a variable. b = neededfunctiontrunc(a,4) -> b now contains 2.7646 b = neededfunctionround(a,4) -> b now contains 2.7647 I looked at the function precision, but it gives strange results ; the behaviour doesn't seem to follow the doc ! Could someone check this ? With default precision at startup, I get : ? a = 2.7646535646854 ? b = precision(a,4) %2 = 2.76465356 With isn't %2 2.7646 ?? Ok, I can do (notice I have to use '5' and not '4' so if the number is > 10, I have to ask for 6 significant digits) : ? \p 5 ? a %3 = 2.7646 But then, I have to set \p back for the next step if I want to remain precise enough. And a is not containing exactly the number 2.7646, it is only a display effect. I am missing something ? Many thanks if you can help. Truncating or rounding numbers after n qdecimal places seems quite common when working with reals, so I am surprised not to have found references to this problem in the archive of the list. Best regards, -- Damien Wyart