Frequently Asked Questions About Sage Development

I'm a beginner to Sage development. What can I do to help develop Sage?

See the guide How to Begin Sage Development.


I'm not a programmer. Is there another way I can help out?

Yes. Like any free open source software project, there are numerous ways in which you could help out with developing Sage. And programming is one of many ways to contribute. If you can speak, read, and write in another (natural) language, there are many ways you can help out. Say you know Italian. Then you can write a Sage tutorial in Italian, or help out with translating the official Sage tutorial to Italian. For the graphics designers or artistically creative, you can help out with improving the design of the Sage website. Or you can cast your critical artistic eyes over the Sage notebook interface and find out where it needs improvement.

Many people like writing technical tutorials. One of the joys of doing so is that you also learn something new in the process. At the same time, you communicate your knowledge to beginners, a skill which is useful in fields other than technical writing. A main point about technical writing is that you communicate a technical subject to beginners, so keep technical jargons to a minimum. Darrell Anderson has written some tips on technical writing, which you are highly recommended to read

The above is a very short list. I’m sure there are many, many more ways in which you can help out.


Where can I find resources on Python or Cython?

See the wiki page on Python/Cython resources.


I want to develop code using functional programming. Where can I find resources on Python functional programming?

See the Sage Constructions document for tips on Python functional programming. These tips are organized under the chapter Python Functional Programming for Mathematicians.


Are there any coding conventions I need to follow?

You should follow the standard Python conventions as documented at PEP 0008 and PEP 0257. Also consult the Sage Developers' Guide, especially the chapter Conventions for Coding in Sage.


I have written some code for the Sage library. How do I test the whole Sage library with my new code?

See the chapter Parallel Testing the Sage Library of the Sage Developers' Guide.


Are there any guidelines on using the Sage bug server?

See the chapter The Sage Trac Server: Submitting Patches and Packages in the Developers' Guide.


What is an spkg?

The abbreviation "spkg" stands for "Sage package". The directory SAGE_ROOT/spkg/standard contains spkg's. In a source install, these are all Sage spkg files (actually .tar or .tar.bz2 files), which are the source code that defines Sage. In a binary install some of these may be small placeholder files to save space.

Sage packages are distributed as .spkg files, which are .tar.bz2 files (or tar files) but have the extension .spkg to discourage confusion. Although Sage packages are packed using tar and/or bzip2, please note that .spkg files contain control information (installation scripts and metadata) that are necessary for building and installing them. For source distributions, when you compile Sage the file SAGE_ROOT/makefile takes care of the unpacking, compilation, and installation of Sage packages for you. For more information on the structure of .spkg files, please refer to the Sage Developer's Guide in your local installation of Sage at

SAGE_ROOT/sage/doc/output/html/en/developer/index.html

If you cannot locate that file in your local installation of Sage, you might want to consider (re)building the standard Sage documentation using this command:

SAGE_ROOT/sage -docbuild all html

or visit the URL http://www.sagemath.org/doc/developer, especially the section "Producing New Sage Packages". Additional Sage packages can be found at http://www.sagemath.org/download-packages.html.


How do I patch an spkg?

Make sure you are familiar with the structure and conventions relating to spkg's; see http://www.sagemath.org/doc/developer/producing_spkgs.html for details. Patching an spkg involves patching the installation script of the spkg and/or patching the upstream source code contained in the spkg. Say you want to patch the GAP package gap-4.4.10.p17. Note that "p17" denotes the patch level of the spkg. The installation script of that spkg is

gap-4.4.10.p17/spkg-install

In general, a script with the name spkg-install is an installation script for an spkg. To patch the installation script, use a text editor to edit that script. Then in the log file SPKG.txt, provide a high-level description of your changes. Once you are satisfied with your changes in the installation script and the log file SPKG.txt, use Mercurial to check in your changes and make sure to provide a meaningful commit message.

The directory src/ contains the source code provided by the upstream project. For example, the source code of GAP 4.4.10 is contained under

gap-4.4.10.p17/src/

To patch the upstream source code, you should first copy the relevant file under src/ over to the directory patches/. Then edit that copied file rather than the relevant file under src/. Once you are satisfied with your changes to the copied file under patches/, generate a unified diff between the original file under src/ and the corresponding copied (and edited) file under patches/. Save the unified diff to a file with the same name, but use the file extension ".patch", and place the diff file under patches/. Note that both of the directories patches/ and src/ should not be under revision control. The Mercurial configuration file .hgignore should contain the following two lines

patches/
src/

Make sure that the installation script spkg-install contains code to copy over the patched file under patches/ to the relevant place under src/. For example, the file

gap-4.4.10.p17/patches/configure.out-ia64

is a patched version of the file

gap-4.4.10.p17/src/cnf/configure.out

The installation script gap-4.4.10.p17/spkg-install contains the following conditional to test under which conditions to copy over the patched file to the relevant place under src/:

# If we're on an Itanium Linux box, we overwite configure.out with a slightly
# modified version.  The modified version has all -O2's replaced by -O0.
# See http://www.gap-system.org/Faq/Hardware-OS/hardware-os8.html
# On the San Diego Super computer `uname -p` is unknown, but
# uname -a includes ia64 in the output.  So this is a better
# detection method.
# Note that GAP was "fixed" to work fine on Itanium without this -O0 hack, but with
# GCC-4.4.0, GAP mysterious stopped working again.  So we revert to using -O0.
if [ `uname` = "Linux" ]; then
   uname -a |grep ia64
   if [ $? = 0 -o `uname -p` = "ia64" ]; then
       cp patches/configure.out-ia64 src/cnf/configure.out
       echo "The file configure.out was patched for SAGE!" > src/cnf/configure.out.README
   fi
fi

Now provide a high-level explanation of your changes in SPKG.txt. Once you are satisfied with your changes, use Mercurial to check in your changes with a meaningful commit message. Next, increment the patch level of the spkg by one, e.g. rename gap-4.4.10.p17 to gap-4.4.10.p18. If you want your patched spkg to be reviewed and available in the next release of Sage, you need to compress it using tar and bz2. Under Linux and OS X, you can compress your patched spkg as follows:

tar -jcf gap-4.4.10.p18.spkg gap-4.4.10.p18

As part of your request for your patched spkg to be reviewed, you need to provide a URL to your spkg on the relevant trac ticket and/or on an email to the relevant mailing list. Usually, you should not upload your spkg to the relevant trac ticket.


A ticket provides a replacement spkg for reviewing. How do I integrate such a replacement spkg into the latest development version of Sage in order to review the ticket?

The details of the review are slightly different depending on whether the replacement spkg is:

  1. a standard spkg;
  2. an optional spkg; or
  3. an experimental spkg.

Standard spkg's live in the directory

SAGE_ROOT/spkg/standard

First, ensure that you have access to the latest development version of Sage. To review a replacement standard spkg, you could integrate it into the latest development version by installing the spkg. The relevant command is

./sage -f URL/to/spkg

This would download the spkg, compile it, and install it---all done automatically for you. Once all these steps are done (i.e. the spkg has been integrated into your copy of Sage for reviewing), you should rebuild the Sage library (in case the new spkg also affects the Sage library):

./sage -b main

Now you are good to review that the replacement spkg fixes any reported problems.

Another way to integrate a replacement standard spkg is to take a source tarball of the latest development version of Sage, uncompress it, remove the current version of the spkg as found under

SAGE_ROOT/spkg/standard

and drop the replacement spkg under that directory. After that, issue make from SAGE_ROOT to compile the whole Sage distribution. This will compile everything, including the replacement spkg you have placed under

SAGE_ROOT/spkg/standard

This is usually a safe way to ensure that the replacement spkg also plays nicely with the other standard spkg's and the Sage library. It also ensures that everything still compiles from source without any errors.

Integrating optional and experimental spkg's work similarly as described above. However, you only need to install the replacement spkg as follows:

./sage -f URL/to/spkg

So it can take more work to integrate a standard spkg than it does to integrate an optional or experimental spkg. The extra effort is required since a replacement standard spkg must not break anything when integrated into Sage.


When I rebuild the Sage library, can I save everything under SAGE_ROOT to a backup directory and later on copy it back when I want to restore to the original after a patch?

Yes you can. Just to clarify, you could do this: You could copy your Sage directory to somewhere else with something like

cp -r sage-x.y.z some/where/else

Then you could proceed with playing around/reviewing a ticket on top of your Sage library as contained under sage-x.y.z/devel/sage-main. (But you need to first apply the patch.) When you are done with sage-x.y.z, you could trash it if you want. Now you still have that pristine copy of Sage you have copied to

some/where/else

Let's call that copied version sage-copy. Now sage-copy is not in a working state because when you copied, everything from sage-x.y.z (including binaries and path variables) were copied over to sage-copy. To get sage-copy into a working state you could use, you need to cd to its SAGE_ROOT and regenerate lots of stuff, including path variables, as follows:

./sage

What you could also do is make a clone of your working Sage version, which is significantly faster and takes up much less space on your hard drive. Say you start with sage-x.y.z, which can be a binary version you've downloaded, or you have compiled it from source. To make a clone of the Sage library you could experiment with, cd to SAGE_ROOT and do

./sage -clone <name-of-clone>

This would create a working copy of the Sage library and named

SAGE_ROOT/devel/sage-<name-of-clone>

The alias

SAGE_ROOT/devel/sage

always points to the current branch of the Sage library you're using. The main branch is called

SAGE_ROOT/devel/sage-main

To switch from the main branch to your cloned branch, do

./sage -b <name-of-clone>

For example, say you built Sage from source and SAGE_ROOT is ~/sage_current. You then made a backup copy with

cp -r ~/sage_current ~/sage_backup

and you applied patches or install new spkg's to ~/sage_current in order to review those patches or spkg's. After you're done reviewing, you delete your copy of Sage, and restore the backup copy, with

rm -r ~/sage_current
cp -r ~/sage_backup ~/sage_current

This would still give you a working copy of Sage. Both ~/sage_backup and the newer ~/sage_current, copied from ~/sage_backup, are still working copies of Sage that you originally copied from the original ~/sage_current. Now you could do

cd ~/sage_current  # or cd ~/sage_backup
./sage

and you'd be presented with a working terminal session of Sage. What you could also do is

cd ~/sage_current  # or cd ~/sage_backup
./sage -b

Warning: This would regenerate/re-compile files in the Sage library to reflect the (possibly) new paths of these copies. The regeneration/re-compilation takes a long time even on a high-end server like a machine on the Sage cluster.

Apart from the above strategy for making backups of your working Sage installation, you could also consider making your own binary distribution. The steps are simple. First, take a source tarball and compile Sage from source. Once it's compiled, do

./sage -bdist x.y.z  # or some name other than x.y.z

to get a binary distribution. Once the binary distribution is all packed up for you, the binary tarball could be found under

SAGE_ROOT/dist

You could take that binary tarball as your backup copy. Every time you want to experiment with new code, uncompress that binary tarball, and play away. Note that the binary tarball contains the source of the Sage library and binary versions of standard spkg's. Unlike the source tarball, the binary tarball doesn't have the source of standard spkg's. For more information on Sage command line arguments, do

./sage -advanced

This would list all command line options you could pass to Sage.

devel/DevelFAQ (last edited 2010-01-20 16:37:10 by Minh Nguyen)