Re: Problem with 2.3.3 testing ellglobalred and galois

["David A. Desrosiers" <desrod@gnu-designs.com>]

> Then, install it in a sandbox using

>  Configure --prefix /tmp/pari && make install && make test-all

> or somesuch. [ Why assume that the 'install' target is not itself 
> buggy, after all ? ] You can always revert to

>  Configure && make install && make test-all

Why not make the 'test-all' target behave like 100% of the other 
software out there, and test within the source tree itself? The extra 
tarballs already say to unpack into the source tree, why not continue 
in that context, and test there as well.

I don't know of ANY production environment, where software is deployed 
into production first, _then_ tested for functionality second. It 
never works that way.

>  "Users can check where gp will look for the packages by typing
>  'default(datadir)'"

Type that where? In what application? Nothing I've seen accepts that 
syntax as a valid command+args.

>  env GP_DATA_DIR=data make test-all

This could be one option, but (see above) easily supplanted by fixing 
the 'test-all' target to test within the source tree where pari was 
built.

>  Heavy-duty testing [Optional] :
>  There are a few extra tests which should be useful only for developpers.

Well, since pari is used mostly by developers, knowing that it doesn't 
falter in some dark corner of the compiler or the internals of pari 
would be useful, so 'dobench' is useful, but not for a large 
percentage of the pari audience, I'd think.

> In a sense. The problem is that PARI/GP is 99% functional without 
> any extra package and the latter can be installed manually at any 
> time [ just untar them to the place that default(datadir) or 
> $GP_DATA_DIR indicates ].

As long as you remember to set a global of that value. In our case, 
since we're building things in a tightly-controlled sandbox, managed 
by templates and scripts, that variable doesn't exist, and can't, so I 
have to pass a bogus --datadir to force it to look elsewhere, then 
_change_ it and build again, sans the 'test-all' test.

> I guess this is just a documentation problem. Would adding a longer 
> explanation in the above section documenting 'make test-all' be 
> satisfactory ? [ listing the tests that will fail unless packages x, 
> y, z are already present and documenting explicitly the override env 
> GP_DATA_DIR=data make test all for testing in the local tree ? ]

That's one option, but fixing the 'test-all' target would probably be 
the best solution. Again, *NOBODY* installs software into production 
first, and then tests after that. It just doesn't happen.

> 1) We *want* the tests to check the complete installation in the 
> end, not the one in a local tree.

That is what 'make check' is for, because you replicate the same 
environment you're going to install into, under ./Olinux-i686 for 
example.

Typically 'make test' tests functionality of the codebase, while 'make 
check' tests the validity of the environment or installation.

Again, this isn't something new, *EVERYTHING* with a proper test or 
'check' target does this (at least several thousand other Open Source 
packages, which admittedly rely on autoconf/autotools).

> 2) The packages we are talking about are inert data which are not 
> expected to change in a foreseeable future. Interested users will 
> download them once then forget about them, while happily upgrading 
> many versions of PARI/GP.

Then why not package these separately, with their own installation 
target, and make them a pre-requisite of building the "full" suite of 
'pari'? That circumvents the entire problem above.

IMHO, unpacking the data files into the source tree (as instructed), 
is entirely incorrect, since they don't need to be there. Nothing that 
I can see uses them, the tests aren't run against that structure, and 
they just end up being copied to <datadir> anyway, in the 'make 
install' target, where 'test-all' is instructed to find them for 
future tests.

> Perhaps 'make install' should not be responsible for checking then 
> handling what's in 'data' and we should just request the user to 
> extract the packages in the correct place. Would that be clearer ?

That presumes that the user has rights to write to production 
locations, and in most production cases, that will not be the case 
(i.e. developers might not have 'root' on production servers).

But this issue directly above would be fixed by a 'test-all' target 
that tests the code _within_ the pari source tree.

> If some non-trivial manipulation should ever become necessary to 
> install a package [ not currently the case ], we can always provide 
> a Makefile with the package ( which would possibly proceed to check 
> out /usr/local/share/lib/pari/pari.cfg if it needs information 
> provided at Configure time )

Let me provide another scenario:

Your development staff configures, builds and submits a request to 
install pari into the production servers, then that gets intsalled at 
some point in the coming days/weeks, then they run 'make test-all', 
and tests fail. Now your developers have to submit a request to have 
the code _removed_ from the production servers, so they can configure, 
build and install again. Lather, rinse, repeat.

The proper way to do this is to configure/build/test until you're sure 
the code is solid, THEN install into the production environment. I'm 
unclear about the history of pari where the decision to change this 
basic workflow was put into place. Why would you build, install and 
_then_ test? I don't follow the logic there.