Final Pynac Switchover Push for Sage-4.0

Contents

Final Pynac Switchover Push for Sage-4.0

Latest Pynac spkg

sage -f -m http://sage.math.washington.edu/home/mhansen/pynac-0.1.6-mh.p7.spkg

Note that we do "-f -m" so that spkg/build/pynac-0.1.6-mh.p7 is left around. Now you can do

./sage -sh
cd spkg/build/pynac-0.1.6-mh.p7/src/ginac
# change anything
make install

William's patches:

http://sage.math.washington.edu/home/wstein/symbolics/

Roberts's patches:

http://sage.math.washington.edu/home/wstein/patches/pynac/

Mike's Repo

On sage.math, do

hg pull -u /scratch/mhansen/sage-3.4.2.alpha0-sage.math-only-x86_64-Linux/devel/sage-symbolics#symbolics_switch

And don't forget to do this from SAGE_ROOT the very first time:

rm devel/sage/build/*/sage/symbolic/constants*; rm devel/sage/build/sage/symbolic/constants.so

William's Repo

hg pull /home/wstein/build/sage-sym/devel/sage/sage/symbolic

TODO: make a refreshed pickle jar

TODO: linear_code.py -- lots of bugs when tested --long

linear_code.py has numerous long long failures and segfaults.

TODO: Numerical Evaluation of infinity

in expression.pyx

            sage: t = x + oo; t
            +Infinity
            sage: t.n()
            +infinity

but it actually returns Infinity.

TODO: Segfault taking a real part

This is from problem R1 from symbench

sage: def f(z): return sqrt(1/3)*z^2 + i/3
....: 
sage: real(f(f(i/2)))
/Users/wstein/build/sage-symbolics/local/bin/sage-sage: line 198: 92287 Segmentation fault      sage-ipython "$@" -i

bt gives:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fd959abe6e0 (LWP 5490)]
import_submodule (mod=0xcfdf68, subname=0x7fff612d7fe8 "sage", fullname=0x7fff612d7fd0 "sage.rings.number_field.sage") at Python/import.c:2360
2360    Python/import.c: No such file or directory.
        in Python/import.c
(gdb) bt
#0  import_submodule (mod=0xcfdf68, subname=0x7fff612d7fe8 "sage", fullname=0x7fff612d7fd0 "sage.rings.number_field.sage") at Python/import.c:2360
#1  0x00000000004a166b in load_next (mod=0xcfdf68, altmod=0x72b440, p_name=<value optimized out>, buf=0x7fff612d7fd0 "sage.rings.number_field.sage", 
    p_buflen=0x7fff612d7fc8) at Python/import.c:2220
#2  0x00000000004a18aa in import_module_level (name=0xd09499 "rings.number_field", globals=0x75a010, locals=<value optimized out>, fromlist=0x7fd954e95a70, 
    level=<value optimized out>) at Python/import.c:2001
#3  0x00000000004a1d55 in PyImport_ImportModuleLevel (name=0xd09494 "sage.rings.number_field", globals=0x12e46e0, locals=0xa0b490, fromlist=0x7fd954e95a70, level=-1)
    at Python/import.c:2072
#4  0x00000000004813a9 in builtin___import__ (self=<value optimized out>, args=<value optimized out>, kwds=<value optimized out>) at Python/bltinmodule.c:47
#5  0x000000000041ab6d in PyObject_CallFunctionObjArgs (callable=0x7fd959a8a5f0) at Objects/abstract.c:1861
#6  0x00007fd944657a08 in __Pyx_Import (name=0xd09470, from_list=0x7fd954e95a70) at sage/rings/number_field/number_field_element.cpp:22933
#7  0x00007fd94468539d in __pyx_pf_4sage_5rings_12number_field_20number_field_element_18NumberFieldElement___init__ (__pyx_v_self=0x41786c8, 
    __pyx_args=<value optimized out>, __pyx_kwds=<value optimized out>) at sage/rings/number_field/number_field_element.cpp:4568
#8  0x0000000000457bbc in wrap_init (self=0xcfdf68, args=0x7fff612d7fe8, wrapped=0x75a0a0, kwds=0x7fff612d7feb) at Objects/typeobject.c:4043
#9  0x0000000000417e63 in PyObject_Call (func=0xcfdf68, arg=0x7fff612d7fe8, kw=0x7fff612d7fd0) at Objects/abstract.c:1861
#10 0x0000000000481952 in PyEval_CallObjectWithKeywords (func=0x7fd954e88d10, arg=0x375eab8, kw=0x0) at Python/ceval.c:3442

Maybe a simpler example:

sage: real((1/9*(3*sqrt(1/3)*x^2 + I)^2*sqrt(1/3) + 1/3*I)(x=I))
/scratch/wstein/build/sage-symbolics/local/bin/sage-sage: line 198: 26443 Segmentation fault      sage-ipython "$@" -i

A simple example where something is wrong (but no segfault):

sage:  ((sqrt(2)*x+I)^2*sqrt(2))(x=I).imag()
0*sqrt(2)

Even simpler:

sage: ((sqrt(2)*I^2+I)*sqrt(2)).real()
/scratch/wstein/build/sage-symbolics/local/bin/sage-sage: line 198: 11471 Segmentation fault      sage-ipython "$@" -i

Even simpler:

sage: (x*(I-I)+I)*x
boom!


sage: a = (x*(I-I)+I)
sage: a
0*x + I
sage: a*a
/scratch/wstein/build/sage-symbolics/local/bin/sage-sage: line 198:  3083 Segmentation fault      sage-ipython "$@" -i

TODO: hashing symbolic expressions is pretty bad

sage: hash(sin(1))
3505120692
sage: var('x,y')
(x, y)
sage: hash(x^y)
0

Hashs of powers are always 0. hash of sin is totally random. This is very bad.

TODO: pynac gcd: infinite recursion in to_polynomial(...) leads to crash

sage: var('n,x')
(n, x)
sage: g= (n+1)/x^n - n/x^n 
sage: g.collect_common_factors() 
/home/wstein/build/sage-3.4.2-symbolics/local/bin/sage-sage: line 198: 29952 Segmentation fault      sage-ipython "$@" -i

Since the change in the definition of gcd of rational numbers in Sage, the gcd in pynac doesn't terminate in some cases. This can crash Sage as above. A quick fix might be to change sage.symbolic.pynac.py_gcd to handle gcd of rationals as before. AFAIK, collect_common_factors() is the only function we expose to the user that calls gcd() in pynac. -- burcin

The above isn't the solution, unfortunately. Putting print statements in py_gcd shows it is never called with rational input for the above example. In fact it isn't even called by the collect_common_factors line. What happens is that there is an infinite recurssion in normal.cpp involving convering something to a polynomial.

(DONE) Get doctest coverage for symbolic/* directory up to 100%

As of May 12, 2009:

callable.py: 100% (21 of 21)
constants.py: 100% (59 of 59)
constants_c.pyx: 100% (5 of 5)
expression.pyx: 95% (136 of 142)
expression_conversions.py: 100% (54 of 54)
function.pyx: 100% (30 of 30)
operators.py: 100% (4 of 4)
pynac.pyx: 69% (16 of 23)
ring.pyx: 100% (24 of 24)

Overall weighted coverage score:  96.1%
Total number of functions:  362
We need   10 more function to get to 99% coverage.

Optional: Maybe compare is still too slow?

sage: a = [sqrt(w) for w in [0..100]]
sage: time a.sort()
CPU times: user 0.41 s, sys: 0.00 s, total: 0.41 s
Wall time: 0.43 s

sage: L = [sqrt(SR(a)) for a in [1..1000]]

sage: time v = [(a>0).variables() for a in L]
CPU times: user 0.44 s, sys: 0.00 s, total: 0.44 s
Wall time: 0.44 s

sage: time v = [CIF(a) for a in L]
CPU times: user 0.41 s, sys: 0.00 s, total: 0.41 s
Wall time: 0.41 s

sage: time v = [(a>0).test_relation() for a in L]
CPU times: user 0.97 s, sys: 0.16 s, total: 1.13 s
Wall time: 1.19 s

Profiling reveals a call to pari to factor is the culprit here in both cases.

DONE: Fix gamma(2+I-I)

sage: gamma(2+I-I)
/scratch/wstein/build/sage-symbolics/local/lib/python2.5/site-packages/sage/functions/other.py:429: RuntimeWarning: tp_compare didn't return -1 or -2 for exception
  return x.gamma()
Exception exceptions.TypeError: "unsupported operand type(s) for %: 'sage.rings.number_field.number_field_element_quadratic.NumberFieldElement_quadratic' and 'int'" in 'sage.symbolic.pynac.py_is_integer' ignored
gamma(2)

DONE: rm constants.crap

put this in spkg-install for now.

rm -f devel/sage/build/*/sage/symbolic/constants*; rm -f devel/sage/build/sage/symbolic/constants.so

(DONE) Fix latex(factorial(...))

sage: latex(factorial(x))
\left(\text{x}\right)!
(and another)

Both in functions/other.py

Optional: Optimize gcd/lcm

Optimize py_gcd and py_lcm in pynac.pyx for the case when both inputs are small integers. In fact, try to avoid calling pynac.pyx at all in that case (just use C++ directly). This gives major speedups on the benchmarks.

(DONE): Bug exhibited by code in ell_generic.py

There is a doctest in ell_generic.py that involves verifying that a symbolic point is on a curve.

            sage: temp
            0

It's a brand new doctest added to ell_generic.py today. According to Nick A., it is a that we don't see temp printing as 0. Note that

    sage: bool(temp==0)
    True

works fine.

Nick remarks that

ncalexan: The fact that it prints as a big mess is in fact a bug -- there's a symbolic pi that is being considered two different pynac objects.

(Done) Run --long tests

Done and found that linear_code.py had issues...

(DONE): _sage_input_ gets broken, e.g., in complex_numper.pyx

I don't understand this but a doctest in complex_nubmer.pyx breaks due to _sage_input_'s implementation there uses SR in some clever way, but SR has changed so that -infinity is a lot smaller. I.e., SR is much more powerful, but that somehow breaks the test.

DONE: Test the tutorial and constructions guide

(DONE): Segfault in taking (2*I)^(1/2)

sage: (2*I)^(1/2)
/scratch/mhansen/sage-3.4.2.alpha0-sage.math-only-x86_64-Linux/local/bin/sage-sage: line 198: 25539 Segmentation fault      sage-ipython "$@" -i

* patch available here: http://sage.math.washington.edu/home/burcin/pynac/power_helper.patch

(DONE) Possible precision issues with Real Interval field coercion

In this example, the last *two* digits displayed are wrong. This I think violates the basic rule about RIF, which is that only the last digit can be wrong. Note that nearly every doctest related to symbolics in real_interval field fails.

sage:  a = factorial(100)/exp(2)
sage: RealIntervalField(10)(a)
1.2617?e157
sage: RealIntervalField(20)(a)
1.2630326?e157

This is almost certainly because RIF(pi) is *not* using mpfi to compute pi -- it is coercing pi to a RealField, then to RIF.

Also related:

sage: floor(SR(10^50 + 10^(-50)))
100000000000000007629769841091887003294964970946560
sage: RIF(sqrt(3)).diameter() == 0
 True

Mike has fixed this in his tree.

(DONE) Maxima still gets called way way too much

For example, right now, the code in expression.pyx for nonzero does this:

        from sage.symbolic.ring import SR
        if self.is_relational():
            res = relational_to_bool(self._gobj)
            if res is True:
                return True

            from sage.calculus.equations import SymbolicEquation
            return bool(SymbolicEquation(self.lhs(), self.rhs(), self.operator()))
        ...

Thus if one does bool(a == b) somewhere in some code, maxima is going to be used a lot evaluate that comparison. This is *not* good, since pynac should be doing *all* such comparisons. That's the whole point of switching to pynac.

(DONE) Issues with fast_callable

sage -t  devel/sage/sage/ext/fast_callable.pyx
**********************************************************************
File "/scratch/wstein/build/sage-sym/devel/sage-main/sage/ext/fast_callable.pyx", line 202:
    sage: f.op_list()
Expected:
    [('load_arg', 0), ('ipow', 7), ('load_const', 1.0), 'add', 'sqrt', 'return']
Got:
    [('load_arg', 0), ('load_const', 7.0), 'pow', ('load_const', 1.0), 'add', 'sqrt', 'return']
**********************************************************************
File "/scratch/wstein/build/sage-sym/devel/sage-main/sage/ext/fast_callable.pyx", line 2211:
    sage: fast_callable(sin(x)/x, vars=[x], domain=RDF).get_orig_args()
Expected:
    {'domain': Real Double Field, 'code': [0, 0, 16, 0, 0, 8, 2], 'py_constants': [], 'args': 1, 'stack': 2, 'constants': []}
Got:
    {'domain': Real Double Field, 'code': [1, 0, 0, 0, 8, 0, 0, 16, 7, 2], 'py_constants': [], 'args': 1, 'stack': 2, 'constants': [1.0]}

About this robertwb says:

robertwb: I don't like the ipow -> pow
[9:37pm] robertwb: something fishy (perhaps) about that other one too
[9:37pm] robertwb: the code stack looks a lot longer than it should

* Mike has fixed this in the branch.

(DONE) Assume is not finished

sage: assume(x>0)
boom!

(DONE) latexing symbolic compositions is busted

After fixing an obvious import error we have the following:

sage: s = ceil(x)
sage: latex(ceil(x))
\mbox{ceil}\left(x\right)
sage: latex(floor(x))
\mbox{floor}\left(x\right)
sage: ceil._latex_composition(x)
'\\left \\lceil x \\right \\rceil'

Note that _latex_composition is never being called. I fixed this by making the _latex_ method for Expression first check to see if the operand has a _latex_composition method, and if so call it.

(DONE) Weird new numpy array conversion issue

NOW:

import numpy; numpy.array([(pi,0)],dtype=float)
Traceback (most recent call last):
...
ValueError: setting an array element with a sequence.

But it used to do this:

sage: import numpy; numpy.array([(pi,0)],dtype=float)
array([[ 3.14159265,  0.        ]])

Solution: Make symbolic expressions not iterable.

(WONTFIX) Unpickling of old SymbolicExpressions

For backward compatibility, we need to be able to unpickle the old symbolic objects.

Wstein: I say screw it and we just deprecate them immediately. Broking calculus pickles doesn't break *any* pickles outside of symbolic calculus. Plus, actually maintaining the old pickles would be _really_ hard.

(DONE) A serious printing BUG

sage: var('A,B,n')
(A, B, n)
sage: (A*B)^n
A*B^n
sage: f = (A*B)^n; f(A=5,B=5)
25^n

It's just a printing bug. Also we have

sage: n*x^(n-1)
n*x^((n - 1))

http://sage.math.washington.edu/home/wstein/symbolics/pynac-fix_printing.patch

and

http://sage.math.washington.edu/home/wstein/symbolics/trac_5930-some_symbolic_doctests.patch
http://sage.math.washington.edu/home/wstein/symbolics/trac_5930-some_symbolic_doctests-part2-add_pyrepr.patch

(DONE) sqrt(2)^2

sage: sqrt(x)^2
x
sage: sqrt(2)^2
sqrt(2)^2

wstein@sage:~$ ginsh 
ginsh - GiNaC Interactive Shell (ginac V1.4.1)
...
> sqrt(2)^2;
2

Fix at http://sage.math.washington.edu/home/robertwb/patches/pynac-sqrt.patch

(DONE) sqrt(16)

sage: sqrt(SR(16))
sqrt(16)
sage: 27^(1/3)
27^(1/3)

$ ginsh
> sqrt(16);
4
> 27 ^ (1/3);
3

Fix at http://sage.math.washington.edu/home/robertwb/patches/pynac-pow.patch however

(DONE) sqrt(-4) segfaults

sage: sqrt(-4)
... seg fault ...

Fix at http://sage.math.washington.edu/home/robertwb/patches/pynac-rat-power.patch

(DONE) Small prime divisors square root simplification

sage: sqrt(27)
sqrt(27)

refuses to be simplified (despite the code to do so being executed).

(DONE) Missing parentheses

sage: a = (2/3) ^ (2/3); str(a)
'2/3^(2/3)'
sage: latex(a)
\frac{2}{3}^{\frac{2}{3}}
sage: (-x)^(1/4)
-x^(1/4)

(DONE) Missing mathematica conversions

All should be capitalized:

sage: (tan(x) + exp(x) + sin(x))._mathematica_init_()
 '(exp[x])+(Sin[x])+(tan[x])'

(seems like something we could do generically).

* Fixed in Mike's branch.

(DONE) Number Fields

_pow_ in number_field_element and _rational_ has some hacks since the Pynac library will cause infinite recursions.

An example of a related bug

sage: 2^I
None

... and Maxima is now running

(DONE) Formal Derivatives

Handling of fderivatives so that we can convert to Maxima.
Mike and Burcin came up with a clean solution that will only allow the conversion to be made when all of the arguments to the function are distinct variables. This covers the cases used by the differential equations interface. The fix is in Mike's tree.

(DONE) Massive Speed Regressions

The massive speed regressions, making pynac symbolics *slower* than Maxima for some benchmarks.

Pynac on May 9:

sage: _=var('x,y,z'); f=expand((x+y+z)^6); g=f+1
sage: timeit('(f*g).expand()')
5 loops, best of 3: 241 ms per loop

SAGE-3.2.3 with pynac:
sage: _=var('x,y,z',ns=1); f=expand((x+y+z)^6); g=f+1
sage: timeit('(f*g).expand()')
125 loops, best of 3: 3.19 ms per loop

Singular:
sage: R.<x,y,z> = QQ[]; f=(x+y+z)^6; g=f+1
sage: timeit('h=f*g')
625 loops, best of 3: 29.7 µs per loop

SAGE-3.2.3 (using Maxima on clisp; on ecl it can do this 3-4 times as fast):
sage: _=var('x,y,z'); f=expand((x+y+z)^6); g=f+1
sage: timeit('(f*g).expand()')
5 loops, best of 3: 118 ms per loop

SOLUTION: This was caused entirely by py_is_real being ridiculously slow when input is an int or Integer. By special casing those cases, the problem goes away.

(DONE) I = sqrt(-1)

The current handling of I as a wrapper around a number field element is a bit awkward since (I+2)*x can't be expanded as I+2 is "atomic". This also means that I.imag() returns 1.0 instead of 1. This causes issues in algebraic. Idea to solve this:

Mike:  I think what we want is just a QQ*I ring which automatically goes to SR when you do arithmetic with anything outside of it.
 me:  ok.
You suggested that before, and it makes some sense.
Can't we just make a quadratic number field, and enhance it a tiny spec.
 Mike:  I think if we had that, then everything should work out.
 Sent at 11:31 AM on Thursday
 me:  That sounds easy enough, and it would be super fast, since quadratic fields are very very fast.
 Mike:  We just don't have an exact way to work with complex numbers.
 me:  robertwb could whip it out, as he wrote quadratic fields, etc.
 Mike:  Yep
For RR we have QQ, but we don't have quite an analogue for CC.

Robert Bradshaw is going to work on this.
There is now a patch up at http://trac.sagemath.org/sage_trac/ticket/6005

Doctest status: May 15

As of 11:40am, we are at:

        sage -t  devel/sage-symbolics/sage/rings/arith.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/complex_number.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/wester.py # 7 doctests failed
        sage -t  devel/sage-symbolics/sage/schemes/elliptic_curves/ell_generic.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/special.py # 7 doctests failed
        sage -t  devel/sage-symbolics/sage/symbolic/expression.pyx # 3 doctests failed



Function Substitution
---------------------
        sage -t  devel/sage-symbolics/sage/calculus/var.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/calculus.py # 2 doctests failed


Pickling
--------
        sage -t  devel/sage-symbolics/sage/structure/sage_object.pyx # 1 doctests failed

Derivative of Polylog
---------------------
        sage -t  devel/sage-symbolics/sage/functions/log.py # 3 doctests failed

Extra Parens
------------
Look at the level in py_repr and py_latex
        sage -t  devel/sage-symbolics/sage/symbolic/tests.py # 2 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/other.py # 2 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/trig.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/calculus.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/symbolic/expression.pyx # 5 doctests failed


Hashing
-------
        sage -t  devel/sage-symbolics/sage/symbolic/assumptions.py # 1 doctests failed

Doctest status: May 14

As of 1:03am, we have 53 failures in 18 files.

        sage -t  devel/sage-symbolics/sage/rings/arith.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/rational_field.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/complex_number.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/structure/parent.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/structure/sage_object.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/var.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/wester.py # 7 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/calculus.py # 4 doctests failed
        sage -t  devel/sage-symbolics/sage/schemes/elliptic_curves/ell_generic.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/log.py # 3 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/trig.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/special.py # 7 doctests failed
        sage -t  devel/sage-symbolics/sage/symbolic/pynac.pyx # 2 doctests failed
        sage -t  devel/sage-symbolics/sage/symbolic/function.pyx # 2 doctests failed
        sage -t  devel/sage-symbolics/sage/symbolic/tests.py # 2 doctests failed
        sage -t  devel/sage-symbolics/sage/symbolic/assumptions.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/other.py # 2 doctests failed
        sage -t  devel/sage-symbolics/sage/symbolic/expression.pyx # 15 doctests failed

Doctest status: May 13

As of 3:26pm, we have 106 failures in 16 files

        sage -t  devel/sage-symbolics/sage/rings/rational_field.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/arith.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/complex_number.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/structure/sage_object.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/var.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/functional.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/wester.py # 14 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/calculus.py # 4 doctests failed
        sage -t  devel/sage-symbolics/sage/schemes/elliptic_curves/ell_generic.py # 2 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/log.py # 13 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/trig.py # 5 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/special.py # 12 doctests failed
        sage -t  devel/sage-symbolics/sage/symbolic/tests.py # 2 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/other.py # 16 doctests failed
        sage -t  devel/sage-symbolics/sage/symbolic/assumptions.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/symbolic/expression.pyx # 31 doctests failed

Doctest status: May 12

As of 10:38am, we have 264 failures in 27 files.

        sage -t  devel/sage-symbolics/sage/quadratic_forms/extras.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/matrix/tests.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/real_mpfi.pyx # 33 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/rational_field.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/arith.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/complex_number.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/infinity.py # Segfault
        sage -t  devel/sage-symbolics/sage/rings/real_lazy.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/rational.pyx # 6 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/qqbar.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/number_field/number_field_element.pyx # 0 doctests failed
        sage -t  devel/sage-symbolics/sage/rings/polynomial/polynomial_element.pyx # 2 doctests failed
        sage -t  devel/sage-symbolics/sage/structure/parent.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/structure/sage_object.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/var.pyx # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/wester.py # 15 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/functional.py # 1 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/equations.py # 97 doctests failed
        sage -t  devel/sage-symbolics/sage/calculus/calculus.py # 4 doctests failed
        sage -t  devel/sage-symbolics/sage/schemes/elliptic_curves/ell_generic.py # 2 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/log.py # 13 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/trig.py # 5 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/other.py # 21 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/special.py # 12 doctests failed
        sage -t  devel/sage-symbolics/sage/functions/piecewise.py # 7 doctests failed
        sage -t  devel/sage-symbolics/sage/symbolic/expression.pyx # 34 doctests failed
        sage -t  devel/sage-symbolics/sage/schemes/elliptic_curves/ell_rational_field.py # 1 doctests failed

Doctest status: May 9

The failures as of May 9 at 4:19pm are here:

http://sage.math.washington.edu/home/wstein/build/sage-sym/test-all-2.out

Doctests fail in 59 distinct files.

As of 7:08pm we have 47 files failing doctest files, and 766 failing doctests:

http://sage.math.washington.edu/home/wstein/build/sage-sym/test-all-3.out

Useful code snippet:

sage: os.system('grep "doctests failed" test-all-3.out   > a')
sage: sum([int(x.split('#')[1].split()[0]) for x in open('a').readlines()])
766

As of 11:16pm, we have 38 files with failing doctests, and a total of 671 tests failing.

        sage -t  devel/sage/sage/calculus/calculus.py # 50 doctests failed
        sage -t  devel/sage/sage/calculus/desolvers.py # 12 doctests failed
        sage -t  devel/sage/sage/calculus/equations.py # 115 doctests failed
        sage -t  devel/sage/sage/calculus/functional.py # 27 doctests failed
        sage -t  devel/sage/sage/calculus/functions.py # 6 doctests failed
        sage -t  devel/sage/sage/calculus/test_sympy.py # 13 doctests failed
        sage -t  devel/sage/sage/calculus/var.pyx # 11 doctests failed
        sage -t  devel/sage/sage/calculus/wester.py # 18 doctests failed
        sage -t  devel/sage/sage/ext/fast_callable.pyx # 2 doctests failed
        sage -t  devel/sage/sage/functions/hyperbolic.py # 11 doctests failed
        sage -t  devel/sage/sage/functions/log.py # 13 doctests failed
        sage -t  devel/sage/sage/functions/other.py # 26 doctests failed
        sage -t  devel/sage/sage/functions/piecewise.py # 39 doctests failed
        sage -t  devel/sage/sage/functions/special.py # 12 doctests failed
        sage -t  devel/sage/sage/functions/trig.py # 23 doctests failed
        sage -t  devel/sage/sage/interfaces/qepcad.py # 29 doctests failed
        sage -t  devel/sage/sage/rings/complex_number.pyx # 1 doctests failed
        sage -t  devel/sage/sage/rings/integer.pyx # 2 doctests failed
        sage -t  devel/sage/sage/rings/rational_field.py # 1 doctests failed
        sage -t  devel/sage/sage/rings/rational.pyx # 1 doctests failed
        sage -t  devel/sage/sage/rings/real_double.pyx # 1 doctests failed
        sage -t  devel/sage/sage/rings/real_lazy.pyx # 1 doctests failed
        sage -t  devel/sage/sage/rings/real_mpfi.pyx # 33 doctests failed
        sage -t  devel/sage/sage/schemes/elliptic_curves/ell_generic.py # 1 doctests failed
        sage -t  devel/sage/sage/structure/sage_object.pyx # 1 doctests failed
        sage -t  devel/sage/sage/symbolic/callable.py # 2 doctests failed
        sage -t  devel/sage/sage/symbolic/constants.py # 29 doctests failed
        sage -t  devel/sage/sage/symbolic/expression_conversions.py # 6 doctests failed
        sage -t  devel/sage/sage/symbolic/expression.pyx # 135 doctests failed


DONE    sage -t  devel/sage/sage/calculus/tests.py # 38 doctests failed
DONE    sage -t  devel/sage/sage/interfaces/expect.py # 1 doctests failed
DONE    sage -t  devel/sage/sage/interfaces/gp.py # 2 doctests failed
DONE    sage -t  devel/sage/sage/plot/plot3d/transform.pyx # 1 doctests failed
DONE    sage -t  devel/sage/sage/rings/arith.py # 4 doctests failed
DONE    sage -t  devel/sage/sage/rings/complex_interval_field.py # 1 doctests failed
DONE    sage -t  devel/sage/sage/rings/real_mpfr.pyx # 1 doctests failed
DONE    sage -t  devel/sage/sage/sets/set.py # 1 doctests failed
DONE    sage -t  devel/sage/sage/structure/element.pyx # 1 doctests failed

Doctest status: May 7-8

As of right now -- May 7 at 11:38 am, doing

sage -tp 20 devel/sage/sage

results in http://sage.math.washington.edu/home/wstein/build/sage-3.4.2-symbolics/test-all.out

Doctests fail in 90 files.

Nearly all of the errors have to do with sqrt, specifically,

sage: sqrt(SR(16))
 sqrt(16)
sage: sqrt(16 + x - x)
 sqrt(16)

and

$ grep "CombinatorialFreeModule instance as first argument" /home/wstein/build/sage-3.4.2-symbolics/test-all.out | wc
    240    4080   34320

The issue is that ginac treats sqrt(x) as having an ambiguous sign, whereas before we choose a branch. The symbolic sqrt(perfect_square) is used all over in the library, so I think we need to allow this. We should not, however, simplify sqrt(x^2).

mhansen has fixed the failures in misc/ below in his branch.
robertwb is looking at the failures in structure/ and matrix/
- quadratic_forms all pass now
- Much weirdness (e.g. the CombinatorialFreeModule.basis error) fixed by http://sage.math.washington.edu/home/robertwb/patches/pynac-sqrt.patch
- Some powering issues fixed in http://sage.math.washington.edu/home/robertwb/patches/pynac-pow.patch
- sage -tp 20 devel/sage/sage (May 8, 6am) at http://sage.math.washington.edu/home/robertwb/sage-3.4.2-symbolics/annotated.log (annotated)
  - ```
  Of 499 failed doctests:
    216 were equal
    89 were not equal
    194 were incomparable.
```

symbolics/pynac_todo/push