Re: PARI/GP pthread questions

On Sun Dec 3, 2023, at 15:10, Karim Belabas wrote:

* Bill Allombert [2023-12-03 13:15]:
[...]
> > 2)
> > Under 3.4.22 nbthreads
> > ...
> > * pthread: number of threads (unlimited, default: number of cores)
> > ...
> >
> > On 16C/32T AMD 7950X CPU I see 3200% CPU in top when starting
> > GP script with parforeach.
> >
> > So should above doc be corrected to
> >
> > "default: #cores * #threads_per_cor"
> >
> > or better to be correct for multi-CPU systems:
>
> When we wrote that, hyperthreading was only used in mainframes.
> It would be much better if the default was the number of cores instead of the number
> of hyperthreads.
> Unfortunately the GNU C library only report the number of CPU threads
> see getconf "_NPROCESSORS_CONF".
> I recommends to set nbthreads to the total number of cores.

Is there a difference with the simpler 'nproc' (from coreutils) ?

For systems with hyper-threading, one can get the number of physical cores
as follows on my laptop:

# grep '^cpu cores' /proc/cpuinfo | uniq
cpu cores : 4

# getconf _NPROCESSORS_CONF
8

# nproc
8

Maybe we could include the number of physical cores in the output of
Configure --mt=pthreads ? And add to INSTALL.tex that we advise to
override the default value of nbthreads in gprc.

Not sure whether it would be wise to change the default in
pthread.c:pari_mt_init().

Maybe a good default could be to use the same method as nproc:

look at how many virtual cores are allowed for gp. Assuming a

32 virtual-core computer (16 physical cores, 2 threads/core),

would have default nbthreads set to 32

taskset -c0-15 gp

would have default nbthreads set to 16 (not 32)

taskset 0x8A gp

would have default nbthreads set to 3

I think this is what the user expects. If one runs a task on two

threads sharing the same core, then performance (may) suffer, but

not by that much. In all the tests I've done, running on half the virtual

threads has always been worse than running on all. Hence defaulting

on the number of physical cores may not be the best, while defaulting

on what the user made available leaves open the possibility that the

user knows what s/he wants.

There exists also the possibility that the computer has 16 physical cores

but the user is allowed to use only 4 threads, and defaulting nbthreads

to 16 (or 32) is definitely bad.

All in all, I remain convinced that the "nproc way", which is counting

the number of virtual cores in the result of sched_getaffinity, is the

best default nbthreads.

I'd appreciate the possibility to ask pari to pin the threads

to a specific (virtual)core, but that's a different matter...

Best,

Loïc