Differences between revisions 75 and 76

Final Pynac Switchover Push for Sage-4.0

Latest Pynac spkg

sage -f -m http://sage.math.washington.edu/home/mhansen/pynac-0.1.6-mh.p4.spkg

Note that we do "-f -m" so that spkg/build/pynac-0.1.6-mh.p4 is left around. Now you can do

./sage -sh
cd spkg/build/pynac-0.1.6-mh.p4/src/ginac
# change anything
make install

William's patches:

http://sage.math.washington.edu/home/wstein/symbolics/

Mike's Repo

On sage.math, do

hg pull -u /scratch/mhansen/sage-3.4.2.alpha0-sage.math-only-x86_64-Linux/devel/sage-symbolics#symbolics_switch

And don't forget to do this from SAGE_ROOT the very first time:

rm devel/sage/build/*/sage/symbolic/constants*; rm devel/sage/build/sage/symbolic/constants.so

William's Repo

hg pull /home/wstein/build/sage-sym/devel/sage/sage/symbolic

Contents

Final Pynac Switchover Push for Sage-4.0

Precision issues with Real Interval field coercion

In this example, the last *two* digits displayed are wrong. This I think violates the basic rule about RIF, which is that only the last digit can be wrong. Note that nearly every doctest related to symbolics in real_interval field fails.

sage:  a = factorial(100)/exp(2)
sage: RealIntervalField(10)(a)
1.2617?e157
sage: RealIntervalField(20)(a)
1.2630326?e157

_sage_input_ gets broken, e.g., in complex_numper.pyx

I don't understand this but a doctest in complex_nubmer.pyx breaks due to _sage_input_'s implementation there uses SR in some clever way, but SR has changed so that -infinity is a lot smaller. I.e., SR is much more powerful, but that somehow breaks the test.

Get doctest coverage for symbolic/* directory up to 100%

As of May 10, 2009:

wstein@sage:~/build/sage-sym/devel/sage/sage/symbolic$ sage -coverage .
callable.py: 85% (18 of 21)
constants.py: 88% (53 of 60)
constants_c.pyx: 0% (0 of 4)
expression.pyx: 95% (131 of 137)
expression_conversions.py: 75% (39 of 52)
function.pyx: 86% (26 of 30)
operators.py: 100% (4 of 4)
pynac.pyx: 66% (14 of 21)
ring.pyx: 100% (24 of 24)
Overall weighted coverage score:  87.1%
We need   42 more function to get to 99% coverage.

hashing symbolic expressions is pretty bad

sage: hash(sin(1))
3505120692
sage: var('x,y')
(x, y)
sage: hash(x^y)
0

Hashs of powers are always 0. hash of sin is totally random. This is very bad. }}}

Maxima still gets called way way too much

For example, right now, the code in expression.pyx for nonzero does this:

        from sage.symbolic.ring import SR
        if self.is_relational():
            res = relational_to_bool(self._gobj)
            if res is True:
                return True

            from sage.calculus.equations import SymbolicEquation
            return bool(SymbolicEquation(self.lhs(), self.rhs(), self.operator()))
        ...

Thus if one does bool(a == b) somewhere in some code, maxima is going to be used a lot evaluate that comparison. This is *not* good, since pynac should be doing *all* such comparisons. That's the whole point of switching to pynac.

Bug exhibited by code in ell_generic.py

There is a doctest in ell_generic.py that involves verifying that a symbolic point is on a curve.

            sage: temp
            0

It's a brand new doctest added to ell_generic.py today. According to Nick A., it is a that we don't see temp printing as 0. Note that

    sage: bool(temp==0)
    True

works fine.

Nick remarks that

ncalexan: The fact that it prints as a big mess is in fact a bug -- there's a symbolic pi that is being considered two different pynac objects.

Issues with fast_callable

sage -t  devel/sage/sage/ext/fast_callable.pyx
**********************************************************************
File "/scratch/wstein/build/sage-sym/devel/sage-main/sage/ext/fast_callable.pyx", line 202:
    sage: f.op_list()
Expected:
    [('load_arg', 0), ('ipow', 7), ('load_const', 1.0), 'add', 'sqrt', 'return']
Got:
    [('load_arg', 0), ('load_const', 7.0), 'pow', ('load_const', 1.0), 'add', 'sqrt', 'return']
**********************************************************************
File "/scratch/wstein/build/sage-sym/devel/sage-main/sage/ext/fast_callable.pyx", line 2211:
    sage: fast_callable(sin(x)/x, vars=[x], domain=RDF).get_orig_args()
Expected:
    {'domain': Real Double Field, 'code': [0, 0, 16, 0, 0, 8, 2], 'py_constants': [], 'args': 1, 'stack': 2, 'constants': []}
Got:
    {'domain': Real Double Field, 'code': [1, 0, 0, 0, 8, 0, 0, 16, 7, 2], 'py_constants': [], 'args': 1, 'stack': 2, 'constants': [1.0]}

About this robertwb says:

robertwb: I don't like the ipow -> pow
[9:37pm] robertwb: something fishy (perhaps) about that other one too
[9:37pm] robertwb: the code stack looks a lot longer than it should

(DONE) Assume is not finished

sage: assume(x>0)
boom!

(DONE) latexing symbolic compositions is busted

After fixing an obvious import error we have the following:

sage: s = ceil(x)
sage: latex(ceil(x))
\mbox{ceil}\left(x\right)
sage: latex(floor(x))
\mbox{floor}\left(x\right)
sage: ceil._latex_composition(x)
'\\left \\lceil x \\right \\rceil'

Note that _latex_composition is never being called. I fixed this by making the _latex_ method for Expression first check to see if the operand has a _latex_composition method, and if so call it.

(DONE) Weird new numpy array conversion issue

NOW:

import numpy; numpy.array([(pi,0)],dtype=float)
Traceback (most recent call last):
...
ValueError: setting an array element with a sequence.

But it used to do this:

sage: import numpy; numpy.array([(pi,0)],dtype=float)
array([[ 3.14159265,  0.        ]])

Solution: Make symbolic expressions not iterable.

(Maybe WONTFIX) Unpickling of old SymbolicExpressions

For backward compatibility, we need to be able to unpickle the old symbolic objects.

Wstein: I say screw it and we just deprecate them immediately. Broking calculus pickles doesn't break *any* pickles outside of symbolic calculus. Plus, actually maintaining the old pickles would be _really_ hard.

(DONE) A serious printing BUG

sage: var('A,B,n')
(A, B, n)
sage: (A*B)^n
A*B^n
sage: f = (A*B)^n; f(A=5,B=5)
25^n

It's just a printing bug. Also we have

sage: n*x^(n-1)
n*x^((n - 1))

http://sage.math.washington.edu/home/wstein/symbolics/pynac-fix_printing.patch

and

http://sage.math.washington.edu/home/wstein/symbolics/trac_5930-some_symbolic_doctests.patch
http://sage.math.washington.edu/home/wstein/symbolics/trac_5930-some_symbolic_doctests-part2-add_pyrepr.patch

(DONE) sqrt(2)^2

sage: sqrt(x)^2
x
sage: sqrt(2)^2
sqrt(2)^2

wstein@sage:~$ ginsh 
ginsh - GiNaC Interactive Shell (ginac V1.4.1)
...
> sqrt(2)^2;
2

Fix at http://sage.math.washington.edu/home/robertwb/patches/pynac-sqrt.patch

(DONE) sqrt(16)

sage: sqrt(SR(16))
sqrt(16)
sage: 27^(1/3)
27^(1/3)

$ ginsh
> sqrt(16);
4
> 27 ^ (1/3);
3

Fix at http://sage.math.washington.edu/home/robertwb/patches/pynac-pow.patch however

(DONE) Small prime divisors square root simplification

sage: sqrt(27)
sqrt(27)

refuses to be simplified (despite the code to do so being executed).

(DONE) Missing parentheses

sage: a = (2/3) ^ (2/3); str(a)
'2/3^(2/3)'
sage: latex(a)
\frac{2}{3}^{\frac{2}{3}}
sage: (-x)^(1/4)
-x^(1/4)

(DONE) Missing mathematica conversions

All should be capitalized:

sage: (tan(x) + exp(x) + sin(x))._mathematica_init_()
 '(exp[x])+(Sin[x])+(tan[x])'

(seems like something we could do generically).

* Fixed in Mike's branch.

pynac gcd: infinite recursion in to_polynomial(...) leads to crash

sage: var('n,x')
(n, x)
sage: g= (n+1)/x^n - n/x^n 
sage: g.collect_common_factors() 
/home/wstein/build/sage-3.4.2-symbolics/local/bin/sage-sage: line 198: 29952 Segmentation fault      sage-ipython "$@" -i

Since the change in the definition of gcd of rational numbers in Sage, the gcd in pynac doesn't terminate in some cases. This can crash Sage as above. A quick fix might be to change sage.symbolic.pynac.py_gcd to handle gcd of rationals as before. AFAIK, collect_common_factors() is the only function we expose to the user that calls gcd() in pynac. -- burcin

The above isn't the solution, unfortunately. Putting print statements in py_gcd shows it is never called with rational input for the above example. In fact it isn't even called by the collect_common_factors line. What happens is that there is an infinite recurssion in normal.cpp involving convering something to a polynomial.

Number Fields

_pow_ in number_field_element and _rational_ has some hacks since the Pynac library will cause infinite recursions.

An example of a related bug

sage: 2^I
None

... and Maxima is now running

(DONE) Formal Derivatives

Handling of fderivatives so that we can convert to Maxima.
Mike and Burcin came up with a clean solution that will only allow the conversion to be made when all of the arguments to the function are distinct variables. This covers the cases used by the differential equations interface. The fix is in Mike's tree.

Massive Speed Regressions

The massive speed regressions, making pynac symbolics *slower* than Maxima for some benchmarks.

Pynac on May 9:

sage: _=var('x,y,z'); f=expand((x+y+z)^6); g=f+1
sage: timeit('(f*g).expand()')
5 loops, best of 3: 241 ms per loop

SAGE-3.2.3 with pynac:
sage: _=var('x,y,z',ns=1); f=expand((x+y+z)^6); g=f+1
sage: timeit('(f*g).expand()')
125 loops, best of 3: 3.19 ms per loop

Singular:
sage: R.<x,y,z> = QQ[]; f=(x+y+z)^6; g=f+1
sage: timeit('h=f*g')
625 loops, best of 3: 29.7 µs per loop

SAGE-3.2.3 (using Maxima on clisp; on ecl it can do this 3-4 times as fast):
sage: _=var('x,y,z'); f=expand((x+y+z)^6); g=f+1
sage: timeit('(f*g).expand()')
5 loops, best of 3: 118 ms per loop

(DONE) I = sqrt(-1)

The current handling of I as a wrapper around a number field element is a bit awkward since (I+2)*x can't be expanded as I+2 is "atomic". This also means that I.imag() returns 1.0 instead of 1. This causes issues in algebraic. Idea to solve this:

Mike:  I think what we want is just a QQ*I ring which automatically goes to SR when you do arithmetic with anything outside of it.
 me:  ok.
You suggested that before, and it makes some sense.
Can't we just make a quadratic number field, and enhance it a tiny spec.
 Mike:  I think if we had that, then everything should work out.
 Sent at 11:31 AM on Thursday
 me:  That sounds easy enough, and it would be super fast, since quadratic fields are very very fast.
 Mike:  We just don't have an exact way to work with complex numbers.
 me:  robertwb could whip it out, as he wrote quadratic fields, etc.
 Mike:  Yep
For RR we have QQ, but we don't have quite an analogue for CC.

Robert Bradshaw is going to work on this.
There is now a patch up at http://trac.sagemath.org/sage_trac/ticket/6005

Doctest status: May 9

The failures as of May 9 at 4:19pm are here:

http://sage.math.washington.edu/home/wstein/build/sage-sym/test-all-2.out

Doctests fail in 59 distinct files.

As of 7:08pm we have 47 files failing doctest files, and 766 failing doctests:

http://sage.math.washington.edu/home/wstein/build/sage-sym/test-all-3.out

Useful code snippet:

sage: os.system('grep "doctests failed" test-all-3.out   > a')
sage: sum([int(x.split('#')[1].split()[0]) for x in open('a').readlines()])
766

As of 11:16pm, we have 38 files with failing doctests, and a total of 671 tests failing.

        sage -t  devel/sage/sage/calculus/calculus.py # 50 doctests failed
        sage -t  devel/sage/sage/calculus/desolvers.py # 12 doctests failed
        sage -t  devel/sage/sage/calculus/equations.py # 115 doctests failed
        sage -t  devel/sage/sage/calculus/functional.py # 27 doctests failed
        sage -t  devel/sage/sage/calculus/functions.py # 6 doctests failed
        sage -t  devel/sage/sage/calculus/tests.py # 38 doctests failed
        sage -t  devel/sage/sage/calculus/test_sympy.py # 13 doctests failed
        sage -t  devel/sage/sage/calculus/var.pyx # 11 doctests failed
        sage -t  devel/sage/sage/calculus/wester.py # 18 doctests failed
        sage -t  devel/sage/sage/ext/fast_callable.pyx # 2 doctests failed
        sage -t  devel/sage/sage/functions/hyperbolic.py # 11 doctests failed
        sage -t  devel/sage/sage/functions/log.py # 13 doctests failed
        sage -t  devel/sage/sage/functions/other.py # 26 doctests failed
        sage -t  devel/sage/sage/functions/piecewise.py # 39 doctests failed
        sage -t  devel/sage/sage/functions/special.py # 12 doctests failed
        sage -t  devel/sage/sage/functions/trig.py # 23 doctests failed
        sage -t  devel/sage/sage/interfaces/qepcad.py # 29 doctests failed
        sage -t  devel/sage/sage/rings/complex_number.pyx # 1 doctests failed
        sage -t  devel/sage/sage/rings/integer.pyx # 2 doctests failed
        sage -t  devel/sage/sage/rings/rational_field.py # 1 doctests failed
        sage -t  devel/sage/sage/rings/rational.pyx # 1 doctests failed
        sage -t  devel/sage/sage/rings/real_double.pyx # 1 doctests failed
        sage -t  devel/sage/sage/rings/real_lazy.pyx # 1 doctests failed
        sage -t  devel/sage/sage/rings/real_mpfi.pyx # 33 doctests failed
        sage -t  devel/sage/sage/schemes/elliptic_curves/ell_generic.py # 1 doctests failed
        sage -t  devel/sage/sage/structure/sage_object.pyx # 1 doctests failed
        sage -t  devel/sage/sage/symbolic/callable.py # 2 doctests failed
        sage -t  devel/sage/sage/symbolic/constants.py # 29 doctests failed
        sage -t  devel/sage/sage/symbolic/expression_conversions.py # 6 doctests failed
        sage -t  devel/sage/sage/symbolic/expression.pyx # 135 doctests failed


DONE    sage -t  devel/sage/sage/interfaces/expect.py # 1 doctests failed
DONE    sage -t  devel/sage/sage/interfaces/gp.py # 2 doctests failed
DONE    sage -t  devel/sage/sage/plot/plot3d/transform.pyx # 1 doctests failed
DONE    sage -t  devel/sage/sage/rings/arith.py # 4 doctests failed
DONE    sage -t  devel/sage/sage/rings/complex_interval_field.py # 1 doctests failed
DONE    sage -t  devel/sage/sage/rings/real_mpfr.pyx # 1 doctests failed
DONE    sage -t  devel/sage/sage/sets/set.py # 1 doctests failed
DONE    sage -t  devel/sage/sage/structure/element.pyx # 1 doctests failed

Doctest status: May 7-8

As of right now -- May 7 at 11:38 am, doing

sage -tp 20 devel/sage/sage

results in http://sage.math.washington.edu/home/wstein/build/sage-3.4.2-symbolics/test-all.out

Doctests fail in 90 files.

Nearly all of the errors have to do with sqrt, specifically,

sage: sqrt(SR(16))
 sqrt(16)
sage: sqrt(16 + x - x)
 sqrt(16)

and

$ grep "CombinatorialFreeModule instance as first argument" /home/wstein/build/sage-3.4.2-symbolics/test-all.out | wc
    240    4080   34320

The issue is that ginac treats sqrt(x) as having an ambiguous sign, whereas before we choose a branch. The symbolic sqrt(perfect_square) is used all over in the library, so I think we need to allow this. We should not, however, simplify sqrt(x^2).

mhansen has fixed the failures in misc/ below in his branch.
robertwb is looking at the failures in structure/ and matrix/
- quadratic_forms all pass now
- Much weirdness (e.g. the CombinatorialFreeModule.basis error) fixed by http://sage.math.washington.edu/home/robertwb/patches/pynac-sqrt.patch
- Some powering issues fixed in http://sage.math.washington.edu/home/robertwb/patches/pynac-pow.patch
- sage -tp 20 devel/sage/sage (May 8, 6am) at http://sage.math.washington.edu/home/robertwb/sage-3.4.2-symbolics/annotated.log (annotated)
  - ```
  Of 499 failed doctests:
    216 were equal
    89 were not equal
    194 were incomparable.
```

-  ⇤ ← Revision 75 as of 2009-05-10 08:45:16 → 
  Size: 16111
  Editor: was
  Comment:
+   ← Revision 76 as of 2009-05-10 08:52:34 → ⇥
  Size: 16551
  Editor: was
  Comment:
-Deletions are marked like this.
+Additions are marked like this.
 Line 43:
+== Precision issues with Real Interval field coercion ==

In this example, the last *two* digits displayed are wrong.  This I think violates the basic rule about RIF, which is that only the last digit can be wrong.  Note that nearly every doctest related to symbolics in real_interval field fails.
{{{
sage:  a = factorial(100)/exp(2)
sage: RealIntervalField(10)(a)
1.2617?e157
sage: RealIntervalField(20)(a)
1.2630326?e157
}}}

Diff for "symbolics/pynac_todo/push"