Jeroen Demeyer on Mon, 03 Oct 2016 17:05:00 +0200


[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: Use PROT_NONE for unused virtual stack memory 

On 2016-09-29 17:46, Bill Allombert wrote:

Do you have some framework to test this ?
Not really because it depends a lot on the OS. In order to properly test this, you need a Linux system with vm.overcommit = 2. So you need 

# echo 2 >/proc/sys/vm/overcommit_memory


The idea is to use PROT_NONE for the unused part of the PARI stack (between
vbot and bot). Memory mmap()ed with PROT_NONE is not committed: it does not
count against the available physical/swap memory available for the system.


Do you have some relevant documentation for this behaviour ?
No. As far as I know, this is undocumented. But the trick is known and used in the wild, see for example the discussion 

https://github.com/jemalloc/jemalloc/issues/255

and code

https://github.com/jemalloc/jemalloc/blob/3c8c3e9e9b59b6e34a222816a05f0a01a68919b3/src/pages.c#L134


One consequence of this patch is that increasing the actual size (during
some calculation which needs more stack space) can fail. This is not a
problem since we can handle this failure the same way that we handle a usual
PARI stack overflow.


It is unclear from the documentation of mprotect that it can actually
fail for that reason. Can you make it fail and what is the error code ?


Yes, it can fail with error code ENOMEM.


It remains the last point which is: is it the behaviour users of
vm.overcommit = 2 and parisizemax want, or would they prefer the stack
to be fully available up to parisizemax.
Well, the stack is always fully available up to "parisize". If you really want the stack to be available (regardless of overcommit options), you should use "parisize" and not "parisizemax".
The whole point of vm.overcommit = 2 is that the OOM killer should never kick in. Instead, memory allocations should fail. The increasing of the PARI stack above "parisize" is exactly such an allocation which might fail. I think it fits well within this philosophy. 


Jeroen.