Frequently Asked Questions About Sage Development
Contents

Frequently Asked Questions About Sage Development
 I'm a beginner to Sage development. What can I do to help develop Sage?
 I'm not a programmer. Is there another way I can help out?
 Where can I find resources on Python or Cython?
 I want to develop code using functional programming. Where can I find resources on Python functional programming?
 Are there any coding conventions I need to follow?
 I have written some code for the Sage library. How do I test the whole Sage library with my new code?
 Are there any guidelines on using the Sage bug server?
 What is an spkg?
 How do I patch an spkg?
 A ticket provides a replacement spkg for reviewing. How do I integrate such a replacement spkg into the latest development version of Sage in order to review the ticket?
 When I rebuild the Sage library, can I save everything under SAGE_ROOT to a backup directory and later on copy it back when I want to restore to the original after a patch?
I'm a beginner to Sage development. What can I do to help develop Sage?
See the guide How to Begin Sage Development.
I'm not a programmer. Is there another way I can help out?
Yes. Like any free open source software project, there are numerous ways in which you could help out with developing Sage. And programming is one of many ways to contribute. If you can speak, read, and write in another (natural) language, there are many ways you can help out. Say you know Italian. Then you can write a Sage tutorial in Italian, or help out with translating the official Sage tutorial to Italian. For the graphics designers or artistically creative, you can help out with improving the design of the Sage website. Or you can cast your critical artistic eyes over the Sage notebook interface and find out where it needs improvement.
Many people like writing technical tutorials. One of the joys of doing so is that you also learn something new in the process. At the same time, you communicate your knowledge to beginners, a skill which is useful in fields other than technical writing. A main point about technical writing is that you communicate a technical subject to beginners, so keep technical jargons to a minimum. Darrell Anderson has written some tips on technical writing, which you are highly recommended to read
The above is a very short list. I’m sure there are many, many more ways in which you can help out.
Where can I find resources on Python or Cython?
See the wiki page on Python/Cython resources.
I want to develop code using functional programming. Where can I find resources on Python functional programming?
See the Sage Constructions document for tips on Python functional programming. These tips are organized under the chapter Python Functional Programming for Mathematicians.
Are there any coding conventions I need to follow?
You should follow the standard Python conventions as documented at PEP 0008 and PEP 0257. Also consult the Sage Developers' Guide, especially the chapter Conventions for Coding in Sage.
I have written some code for the Sage library. How do I test the whole Sage library with my new code?
See the chapter Parallel Testing the Sage Library of the Sage Developers' Guide.
Are there any guidelines on using the Sage bug server?
See the chapter The Sage Trac Server: Submitting Patches and Packages in the Developers' Guide.
What is an spkg?
The abbreviation "spkg" stands for "Sage package". The directory SAGE_ROOT/spkg/standard contains spkg's. In a source install, these are all Sage spkg files (actually .tar or .tar.bz2 files), which are the source code that defines Sage. In a binary install some of these may be small placeholder files to save space.
Sage packages are distributed as .spkg files, which are .tar.bz2 files (or tar files) but have the extension .spkg to discourage confusion. Although Sage packages are packed using tar and/or bzip2, please note that .spkg files contain control information (installation scripts and metadata) that are necessary for building and installing them. For source distributions, when you compile Sage the file SAGE_ROOT/makefile takes care of the unpacking, compilation, and installation of Sage packages for you. For more information on the structure of .spkg files, please refer to the Sage Developer's Guide in your local installation of Sage at
SAGE_ROOT/sage/doc/output/html/en/developer/index.html
If you cannot locate that file in your local installation of Sage, you might want to consider (re)building the standard Sage documentation using this command:
SAGE_ROOT/sage docbuild all html
or visit the URL http://www.sagemath.org/doc/developer, especially the section "Producing New Sage Packages". Additional Sage packages can be found at http://www.sagemath.org/downloadpackages.html.
How do I patch an spkg?
Make sure you are familiar with the structure and conventions relating to spkg's; see http://www.sagemath.org/doc/developer/producing_spkgs.html for details. Patching an spkg involves patching the installation script of the spkg and/or patching the upstream source code contained in the spkg. Say you want to patch the GAP package gap4.4.10.p17. Note that "p17" denotes the patch level of the spkg. The installation script of that spkg is
gap4.4.10.p17/spkginstall
In general, a script with the name spkginstall is an installation script for an spkg. To patch the installation script, use a text editor to edit that script. Then in the log file SPKG.txt, provide a highlevel description of your changes. Once you are satisfied with your changes in the installation script and the log file SPKG.txt, use Mercurial to check in your changes and make sure to provide a meaningful commit message.
The directory src/ contains the source code provided by the upstream project. For example, the source code of GAP 4.4.10 is contained under
gap4.4.10.p17/src/
To patch the upstream source code, you should first copy the relevant file under src/ over to the directory patches/. Then edit that copied file rather than the relevant file under src/. Once you are satisfied with your changes to the copied file under patches/, generate a unified diff between the original file under src/ and the corresponding copied (and edited) file under patches/. Save the unified diff to a file with the same name, but use the file extension ".patch", and place the diff file under patches/. Note that both of the directories patches/ and src/ should not be under revision control. The Mercurial configuration file .hgignore should contain the following two lines
patches/ src/
Make sure that the installation script spkginstall contains code to copy over the patched file under patches/ to the relevant place under src/. For example, the file
gap4.4.10.p17/patches/configure.outia64
is a patched version of the file
gap4.4.10.p17/src/cnf/configure.out
The installation script gap4.4.10.p17/spkginstall contains the following conditional to test under which conditions to copy over the patched file to the relevant place under src/:
# If we're on an Itanium Linux box, we overwite configure.out with a slightly # modified version. The modified version has all O2's replaced by O0. # See http://www.gapsystem.org/Faq/HardwareOS/hardwareos8.html # On the San Diego Super computer `uname p` is unknown, but # uname a includes ia64 in the output. So this is a better # detection method. # Note that GAP was "fixed" to work fine on Itanium without this O0 hack, but with # GCC4.4.0, GAP mysterious stopped working again. So we revert to using O0. if [ `uname` = "Linux" ]; then uname a grep ia64 if [ $? = 0 o `uname p` = "ia64" ]; then cp patches/configure.outia64 src/cnf/configure.out echo "The file configure.out was patched for SAGE!" > src/cnf/configure.out.README fi fi
Now provide a highlevel explanation of your changes in SPKG.txt. Once you are satisfied with your changes, use Mercurial to check in your changes with a meaningful commit message. Next, increment the patch level of the spkg by one, e.g. rename gap4.4.10.p17 to gap4.4.10.p18. If you want your patched spkg to be reviewed and available in the next release of Sage, you need to compress it using tar and bz2. Under Linux and OS X, you can compress your patched spkg as follows:
tar jcf gap4.4.10.p18.spkg gap4.4.10.p18
As part of your request for your patched spkg to be reviewed, you need to provide a URL to your spkg on the relevant trac ticket and/or on an email to the relevant mailing list. Usually, you should not upload your spkg to the relevant trac ticket.
A ticket provides a replacement spkg for reviewing. How do I integrate such a replacement spkg into the latest development version of Sage in order to review the ticket?
The details of the review are slightly different depending on whether the replacement spkg is:
 a standard spkg;
 an optional spkg; or
 an experimental spkg.
Standard spkg's live in the directory
SAGE_ROOT/spkg/standard
First, ensure that you have access to the latest development version of Sage. To review a replacement standard spkg, you could integrate it into the latest development version by installing the spkg. The relevant command is
./sage f URL/to/spkg
This would download the spkg, compile it, and install itall done automatically for you. Once all these steps are done (i.e. the spkg has been integrated into your copy of Sage for reviewing), you should rebuild the Sage library (in case the new spkg also affects the Sage library):
./sage b main
Now you are good to review that the replacement spkg fixes any reported problems.
Another way to integrate a replacement standard spkg is to take a source tarball of the latest development version of Sage, uncompress it, remove the current version of the spkg as found under
SAGE_ROOT/spkg/standard
and drop the replacement spkg under that directory. After that, issue make from SAGE_ROOT to compile the whole Sage distribution. This will compile everything, including the replacement spkg you have placed under
SAGE_ROOT/spkg/standard
This is usually a safe way to ensure that the replacement spkg also plays nicely with the other standard spkg's and the Sage library. It also ensures that everything still compiles from source without any errors.
Integrating optional and experimental spkg's work similarly as described above. However, you only need to install the replacement spkg as follows:
./sage f URL/to/spkg
So it can take more work to integrate a standard spkg than it does to integrate an optional or experimental spkg. The extra effort is required since a replacement standard spkg must not break anything when integrated into Sage.
When I rebuild the Sage library, can I save everything under SAGE_ROOT to a backup directory and later on copy it back when I want to restore to the original after a patch?
Yes you can. Just to clarify, you could do this: You could copy your Sage directory to somewhere else with something like
cp r sagex.y.z some/where/else
Then you could proceed with playing around/reviewing a ticket on top of your Sage library as contained under sagex.y.z/devel/sagemain. (But you need to first apply the patch.) When you are done with sagex.y.z, you could trash it if you want. Now you still have that pristine copy of Sage you have copied to
some/where/else
Let's call that copied version sagecopy. Now sagecopy is not in a working state because when you copied, everything from sagex.y.z (including binaries and path variables) were copied over to sagecopy. To get sagecopy into a working state you could use, you need to cd to its SAGE_ROOT and regenerate lots of stuff, including path variables, as follows:
./sage
What you could also do is make a clone of your working Sage version, which is significantly faster and takes up much less space on your hard drive. Say you start with sagex.y.z, which can be a binary version you've downloaded, or you have compiled it from source. To make a clone of the Sage library you could experiment with, cd to SAGE_ROOT and do
./sage clone <nameofclone>
This would create a working copy of the Sage library and named
SAGE_ROOT/devel/sage<nameofclone>
The alias
SAGE_ROOT/devel/sage
always points to the current branch of the Sage library you're using. The main branch is called
SAGE_ROOT/devel/sagemain
To switch from the main branch to your cloned branch, do
./sage b <nameofclone>
For example, say you built Sage from source and SAGE_ROOT is ~/sage_current. You then made a backup copy with
cp r ~/sage_current ~/sage_backup
and you applied patches or install new spkg's to ~/sage_current in order to review those patches or spkg's. After you're done reviewing, you delete your copy of Sage, and restore the backup copy, with
rm r ~/sage_current cp r ~/sage_backup ~/sage_current
This would still give you a working copy of Sage. Both ~/sage_backup and the newer ~/sage_current, copied from ~/sage_backup, are still working copies of Sage that you originally copied from the original ~/sage_current. Now you could do
cd ~/sage_current # or cd ~/sage_backup ./sage
and you'd be presented with a working terminal session of Sage. What you could also do is
cd ~/sage_current # or cd ~/sage_backup ./sage b
Warning: This would regenerate/recompile files in the Sage library to reflect the (possibly) new paths of these copies. The regeneration/recompilation takes a long time even on a highend server like a machine on the Sage cluster.
Apart from the above strategy for making backups of your working Sage installation, you could also consider making your own binary distribution. The steps are simple. First, take a source tarball and compile Sage from source. Once it's compiled, do
./sage bdist x.y.z # or some name other than x.y.z
to get a binary distribution. Once the binary distribution is all packed up for you, the binary tarball could be found under
SAGE_ROOT/dist
You could take that binary tarball as your backup copy. Every time you want to experiment with new code, uncompress that binary tarball, and play away. Note that the binary tarball contains the source of the Sage library and binary versions of standard spkg's. Unlike the source tarball, the binary tarball doesn't have the source of standard spkg's. For more information on Sage command line arguments, do
./sage advanced
This would list all command line options you could pass to Sage.