| Karim Belabas on Sun, 15 Jul 2018 15:05:34 +0200 |
[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]
| Re: SIGSEGV on isprime |
* Bill Allombert [2018-07-11 17:25]:
> On Wed, Jul 11, 2018 at 04:48:24PM +0200, Ján Jančár wrote:
> > >> I have now tested:
> > >>
> > >> --enable-tls --kernel=none --> works
> > >> --enable-tls --kernel=none-gmp --> SEGVs!
> > >> --enalbe-tls --kernel=auto-none --> works
>
> My guess is that there is a buffer overflow in the function
> red_montgomery in the file src/kernel/gmp/mp.c which only
> cause a SEGV when the stack ends on a page boundary.
Not exactly a buffer overflow but an off-by-1 error causing us to read
one word beyond the boundary of the modulus 'N'. In this particular
case, N happens to be the first object on the PARI stack and we read
1 word in unallocated memory [ and disregard its value, making the bug
very hard to spot ].
I believe the problem is solved in 'master' HEAD.
Cheers,
K.B.
--
Karim Belabas, IMB (UMR 5251) Tel: (+33) (0)5 40 00 26 17
Universite de Bordeaux Fax: (+33) (0)5 40 00 21 23
351, cours de la Liberation http://www.math.u-bordeaux.fr/~kbelabas/
F-33405 Talence (France) http://pari.math.u-bordeaux.fr/ [PARI/GP]
`