David A. Desrosiers on Sat, 05 Apr 2008 16:14:45 +0200

[Date Prev] [Date Next] [Thread Prev] [Thread Next] [Date Index] [Thread Index]

Re: Problem with 2.3.3 testing ellglobalred and galois

Then, install it in a sandbox using

 Configure --prefix /tmp/pari && make install && make test-all

or somesuch. [ Why assume that the 'install' target is not itself buggy, after all ? ] You can always revert to

 Configure && make install && make test-all

Why not make the 'test-all' target behave like 100% of the other software out there, and test within the source tree itself? The extra tarballs already say to unpack into the source tree, why not continue in that context, and test there as well.

I don't know of ANY production environment, where software is deployed into production first, _then_ tested for functionality second. It never works that way.

 "Users can check where gp will look for the packages by typing

Type that where? In what application? Nothing I've seen accepts that syntax as a valid command+args.

 env GP_DATA_DIR=data make test-all

This could be one option, but (see above) easily supplanted by fixing the 'test-all' target to test within the source tree where pari was built.

 Heavy-duty testing [Optional] :
 There are a few extra tests which should be useful only for developpers.

Well, since pari is used mostly by developers, knowing that it doesn't falter in some dark corner of the compiler or the internals of pari would be useful, so 'dobench' is useful, but not for a large percentage of the pari audience, I'd think.

In a sense. The problem is that PARI/GP is 99% functional without any extra package and the latter can be installed manually at any time [ just untar them to the place that default(datadir) or $GP_DATA_DIR indicates ].

As long as you remember to set a global of that value. In our case, since we're building things in a tightly-controlled sandbox, managed by templates and scripts, that variable doesn't exist, and can't, so I have to pass a bogus --datadir to force it to look elsewhere, then _change_ it and build again, sans the 'test-all' test.

I guess this is just a documentation problem. Would adding a longer explanation in the above section documenting 'make test-all' be satisfactory ? [ listing the tests that will fail unless packages x, y, z are already present and documenting explicitly the override env GP_DATA_DIR=data make test all for testing in the local tree ? ]

That's one option, but fixing the 'test-all' target would probably be the best solution. Again, *NOBODY* installs software into production first, and then tests after that. It just doesn't happen.

1) We *want* the tests to check the complete installation in the end, not the one in a local tree.

That is what 'make check' is for, because you replicate the same environment you're going to install into, under ./Olinux-i686 for example.

Typically 'make test' tests functionality of the codebase, while 'make check' tests the validity of the environment or installation.

Again, this isn't something new, *EVERYTHING* with a proper test or 'check' target does this (at least several thousand other Open Source packages, which admittedly rely on autoconf/autotools).

2) The packages we are talking about are inert data which are not expected to change in a foreseeable future. Interested users will download them once then forget about them, while happily upgrading many versions of PARI/GP.

Then why not package these separately, with their own installation target, and make them a pre-requisite of building the "full" suite of 'pari'? That circumvents the entire problem above.

IMHO, unpacking the data files into the source tree (as instructed), is entirely incorrect, since they don't need to be there. Nothing that I can see uses them, the tests aren't run against that structure, and they just end up being copied to <datadir> anyway, in the 'make install' target, where 'test-all' is instructed to find them for future tests.

Perhaps 'make install' should not be responsible for checking then handling what's in 'data' and we should just request the user to extract the packages in the correct place. Would that be clearer ?

That presumes that the user has rights to write to production locations, and in most production cases, that will not be the case (i.e. developers might not have 'root' on production servers).

But this issue directly above would be fixed by a 'test-all' target that tests the code _within_ the pari source tree.

If some non-trivial manipulation should ever become necessary to install a package [ not currently the case ], we can always provide a Makefile with the package ( which would possibly proceed to check out /usr/local/share/lib/pari/pari.cfg if it needs information provided at Configure time )

Let me provide another scenario:

Your development staff configures, builds and submits a request to install pari into the production servers, then that gets intsalled at some point in the coming days/weeks, then they run 'make test-all', and tests fail. Now your developers have to submit a request to have the code _removed_ from the production servers, so they can configure, build and install again. Lather, rinse, repeat.

The proper way to do this is to configure/build/test until you're sure the code is solid, THEN install into the production environment. I'm unclear about the history of pari where the decision to change this basic workflow was put into place. Why would you build, install and _then_ test? I don't follow the logic there.