Getting Sage-3.0.4 to build and pass all tests on Skynet

Tarballs are here:

http://sage.math.washington.edu/home/was/tmp

To get from rc0 to rc1 from within a built sage do this:

sage: hg_sage.apply('http://sage.math.washington.edu/home/was/patches/a.hg')
sage: hg_sage.merge()
sage: hg_sage.ci()

Then do "./sage -br" at the prompt.

eno: x86_64-Linux-fc8 (status: 10/10)

Using gcc-4.3.1: I (william) can't build ntl. I'm totally stuck until getting past this. Fixed by using this environment (put in my .bash_profile):

# From mabshoff
if [ `hostname` = "eno" ]; then
   export PATH=/usr/local/gcc-4.3.1/x86_64-Linux-fc8/bin/:$PATH
   export LD_LIBRARY_PATH=/usr/local/gcc-4.3.1/x86_64-Linux-fc8/lib64:/usr/local/gmp-4.2.2/x86_64-Linux-fc8-gcc-4.1.2-rh/lib:/usr/local/mpfr-2.3.1/x86_64-Linux-fc8-gmp-4.2.2-gcc-4.1.2-rh/lib

fi

Me (mabshoff) just build 4.0.3.rc1 with gcc 4.3.1 and all tests pass with flying colors.

cicero: x86-Linux-fc8 (status: 10/10)

UPDATE: I just built rc2 using gcc-4.3.1, and I definitely do *not* get an illegal instruction when testing devel/sage/sage/functions/special.py. The only doctest failure I get is one digit being off in time_series.py (numerical noise), which I've fixed in the release tree.

wstein@cicero sage-3.0.4.rc2]$ ./sage -t devel/sage/sage/functions/special.py
sage -t  devel/sage/sage/functions/special.py
         [10.2 s]

----------------------------------------------------------------------
All tests passed!

I wonder if Mariah has some screwed up fortran libraries or something in her path? From a clean bash login shell I setup my environment as follows

        export PATH=/usr/local/bin/x86-Linux-fc8/:$PATH
        LDPATH=/usr/local/gcc-4.3.1/x86-Linux-fc8/lib
        LDPATH=${LDPATH}:/usr/local/gmp-4.2.2/x86-Linux-fc8-gcc-4.1.2-rh/lib
        LDPATH=${LDPATH}:/usr/local/mpfr-2.3.1/x86-Linux-fc8-gmp-4.2.2-gcc-4.1.2-rh/lib

OLD: Using gcc-4.1.2: built fine out of the box; all tests for rc1 pass.

Using gcc-4.3.1: Mariah gets an "illegal instruction" error when testing

sage -t  devel/sage/sage/functions/special.py

See http://trac.sagemath.org/sage_trac/ticket/2303

Replicate sage-free:

sage -python
>>> import scipy.special; scipy.special.iv(float(1),complex(1,0))
/tmp/foo/sage-3.0.4.rc0-x86-Linux-fc8/local/bin/sage-sage: line 359:  2638 Illegal instruction     python "$

* Michael suggested upgrading to the newest version of scipy. This is certainly worth a try.

cleo: ia64-Linux-rhel5 (status: ?/10)

UPDATE: This machine's Linux kernel or something is screwed up, so it is not useful for build/testing purposes until it gets fixed.

Using gcc-4.1.2: Built fine out of the box. Currently testing; many tests have timed out since this machine is so slow. Possible serious singular crash in groups/matrix_gps/matrix_group.py, though it could be a timeout. There is something massively foobar'd about this machine or its file system or something. For example:

[wstein@cleo sage-3.0.4.rc0]$ time ./sage -c "print 1+1"
2
real    0m26.532s
user    0m1.447s
sys     0m3.224s

A 26 second startup time? Ick. Is there something seriously wrong with the build of Python?

iras: ia64-Linux-suse (status: 10/10)

UPDATE: these fail for me:

Singular crashing on startup via pexpect for these 3:
        sage -t  devel/sage/sage/rings/polynomial/multi_polynomial_ideal_libsingular.pyx # 1 doctes
ts failed
        sage -t  devel/sage/sage/rings/polynomial/multi_polynomial_ideal.py # 6 doctests failed
        sage -t  devel/sage/sage/groups/matrix_gps/matrix_group.py # 3 doctests failed

Everything else listed below as failing does not fail for me anymore.

The above is all because somebody (malb) took the wrong singular spkg to make a new version thus completely deleting the important patch here:

I reapplied it and made a new singular spkg.


A complete crash example

sage: MS = MatrixSpace(GF(7),2,2); MatrixGroup([MS([[0,1],[-1,0]]),MS([[1,1],[2,3]])]).invariant_generators()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)

/home/wstein/iras/build/sage-3.0.4.rc0/<ipython console> in <module>()

/home/wstein/iras/build/sage-3.0.4.rc0/local/lib/python2.5/site-packages/sage/groups/matrix_g
ps/matrix_group.py in invariant_generators(self)
    799             singular.eval('list %s=group_reynolds((%s))'%(ReyName,Lgens))
    800             IRName = 't'+singular._next_var_name()
--> 801             singular.eval('matrix %s = invariant_algebra_reynolds(%s[1])'%(IRName,Rey
Name))
    802             OUT = [singular.eval(IRName+'[1,%d]'%(j)) for j in range(1,1+singular('nc
ols('+IRName+')'))]
    803             return [PR(gen) for gen in OUT]

/home/wstein/iras/build/sage-3.0.4.rc0/local/lib/python2.5/site-packages/sage/interfaces/sing
ular.py in eval(self, x, allow_semicolon, strip)
    425             x += ';'
    426
--> 427         s = Expect.eval(self, x)
    428
    429         if s.find("error") != -1 or s.find("Segment fault") != -1:

/home/wstein/iras/build/sage-3.0.4.rc0/local/lib/python2.5/site-packages/sage/interfaces/expe
ct.py in eval(self, code, strip, synchronize, **kwds)
    915         try:
    916             with gc_disabled():
--> 917                 return '\n'.join([self._eval_line(L, **kwds) for L in code.split('\n'
) if L != ''])
    918         except KeyboardInterrupt:
    919             # DO NOT CATCH KeyboardInterrupt, as it is being caught

/home/wstein/iras/build/sage-3.0.4.rc0/local/lib/python2.5/site-packages/sage/interfaces/expe
ct.py in _eval_line(self, line, allow_use_file, wait_for_prompt)
    642                         # we expect to get an EOF if we're quitting.
    643                         return ''
--> 644                     raise RuntimeError, "%s\n%s crashed executing %s"%(msg,self, line
)
    645                 out = E.before
    646             else:

RuntimeError: End Of File (EOF) in read_nonblocking(). Exception style platform.
version: 2.0 ($Revision: 1.151 $)
command: /home/wstein/iras/build/sage-3.0.4.rc0/local/bin/Singular
args: ['/home/wstein/iras/build/sage-3.0.4.rc0/local/bin/Singular', '-t', '--ticks-per-sec',
'1000']
patterns:
    >
buffer (last 100 chars):
before (last 100 chars):  `ncols` error at token `)`
   ? leaving finvar.lib::invariant_algebra_reynolds
Auf Wiedersehen.

after: <class 'pexpect.EOF'>
match: None
match_index: None
exitstatus: None
flag_eof: 1
pid: 6962
child_fd: 3
timeout: None
delimiter: <class 'pexpect.EOF'>
logfile: None
maxread: 1000
searchwindowsize: None
delaybeforesend: 0
Singular crashed executing matrix tsage4 = invariant_algebra_reynolds(tsage3[1]);

The following tests failed:

(done) sage -t  devel/sage/sage/rings/polynomial/polynomial_element.pyx # 1 doctests failed
        sage -t  devel/sage/sage/rings/polynomial/multi_polynomial_ideal_libsingular.pyx # 1 doctes
ts failed
        sage -t  devel/sage/sage/rings/polynomial/multi_polynomial_ideal.py # 6 doctests failed
        sage -t  devel/sage/sage/rings/number_field/totallyreal.py # Segfault
        sage -t  devel/sage/sage/plot/plot3d/transform.pyx # 1 doctests failed
        sage -t  devel/sage/sage/misc/prandom.py # 1 doctests failed
        sage -t  devel/sage/sage/matrix/matrix_symbolic_dense.pyx # 1 doctests failed
        sage -t  devel/sage/sage/groups/matrix_gps/matrix_group.py # 3 doctests failed
        sage -t  devel/sage/sage/functions/piecewise.py # 1 doctests failed
        sage -t  devel/sage/sage/dsage/tests/testdoc.py # 10 doctests failed
        sage -t  devel/sage/sage/dsage/interface/dsage_interface.py # 2 doctests failed
        sage -t  devel/sage/sage/rings/polynomial/polynomial_integer_dense_flint.pyx # 0 doctests f
ailed

Update to 3.0.4.alpha2: With a fix to the python.spkg (build without -fwrapv) results in:

        sage -t -long devel/sage/sage/finance/time_series.pyx
        sage -t -long devel/sage/sage/misc/cython.py
        sage -t -long devel/sage/sage/rings/number_field/totallyreal.py
        sage -t -long devel/sage/sage/rings/polynomial/polynomial_element.pyx
        sage -t -long devel/sage/sage/rings/polynomial/polynomial_integer_dense_flint.pyx

It seems that all the above failures are know issue and we have leads for all of them. In particular:

sh: line 1: 16098 Segmentation fault      /home/wstein/iras/build/sage-3.0.4.rc0/local/bin/python /home/wstein/iras/build/sage-3.0.4.rc0/tmp/.doctest_totallyreal.py >/tmp/tmpW-loSD 2>/tmp/tmp8ZjloU
sage -t  devel/sage/sage/rings/number_field/totallyreal.py

A mysterious error (perphaps a memory error?) occurred, which may have crashed doctest.

I think this is related to the valgrind issues that mabshoff observed and reported.

* Bill Hart says:

FLINT definitely isn't working on ia64.

I've tracked the problem a bit further through now that I know lots of
tests are failing in long_extras-test.

The problem appears to be in the function z_ll_mod_precomp in long_extras.c

This function uses udiv_qrnnd from GMP's longlong.h and not a whole
lot else, so that is a hint. This doesn't mean there is a bug in GMP,
it probably just means that on this architecture FLINT is setting
something up incorrectly before using longlong.h.

It shouldn't take too much to sort out. I mean the function that is
causing all the problems is only 20 lines of code and all pretty
basic.

Update from Bill Hart: There will be a FLINT 1.0.11 in a couple hours (i.e. July 9th, 2008)

menas: x86_64-Linux-suse (status: 10/10)

UPDATE: trac #3628 -- http://trac.sagemath.org/sage_trac/ticket/3628 completely resolves this with no special spkg needed.


With gcc-4.2.1 and reverting to the old version of clisp all tests pass for sage-4.3.1.rc0:

sage -i clisp-2.41.p14.spkg
sage -f maxima-5.13.0.p2.spkg

Without reverting clisp, maxima hangs on certain integrals:

integrate(1/(x^3 *(a+b*x)^(1/3)), x);

I think reverting clisp will be ok for this architecture and for this release since the only reason for the new version was build support on more architectures.

Fix so that new clisp will at least build without not finding readline: http://trac.sagemath.org/sage_trac/ticket/3628

Pentium4: Another "relevant" machine (status: 9/10)

        sage -t  devel/sage/sage/dsage/tests/testdoc.py
        sage -t  devel/sage/sage/server/simple/twist.py

These both involve network services. The testdoc failure is a pure timeout issue with a.wait(30), so safe to ignore -- everything else in that file passes fine. I think the twist.py issue also involves some sort of timeout. I am so tempted to redo the doctests for testdoc and twist or declare them undoctestable... but that is for later.