Re: unable to load a large vector from a file

[Nicolas Mascot <mascotn@maths.tcd.ie>]

Hi Max,

You could try to use writebin() instead of write(); this usually results 
in smaller files.

If your integers are small enough to fit on a long (I am not 100% sure, 
but I think that the range is -2^63 to 2^63-1), you would also save a 
lot of space by converting your vector to a vecsmall ( = vector of small 
integers):
writebin("myvector",Vecsmall(v));
If needed, you can then turn the result back into a (regular) vector 
with Vec():
Vec(read("myvector"))
but in most cases that should not be necessary.

Best regards,
Nicolas

On 28/02/2023 19:30, Max Alekseyev wrote:
> I have a vector v of size 10^9 with moderately-sized integers, which I 
> saved to a file with
> write("myvector.txt",v);
> However, I'm unable to read it back. PARI/GP either runs out of memory 
> (even with 96GB memory allocation!) or just killed by the system.
> I tried both
> v = read("myvector.txt");
> and
> \r myvector.txt
>
> Is there a way around?
>
> PS. The resulting file size is just 10GB and so it should not be an 
> issue to read it as a whole into memory and then parse it, but PARI/GP 
> somehow fails to do that.
>
> Regards,
> Max