Sphinx Documentation SEP (Draft)

The sage documentation is just too complicated for latex2html at this
point.  latex2html has a screwy non-gpl compatible license.
We really need to find some way forward, and latex2html maybe
isn't it.  Perhaps something like sphinx is.

    --William Stein

The purpose of this SEP is to propose that Sage adopt the use of Sphinx for its documentation system.

What is Sphinx?

Sphinx is a tool that makes it easy to create intelligent and beautiful documentation for Python projects, written by Georg Brandl and licensed under the BSD license.

Sphinx uses reStructuredText ( http://docutils.sourceforge.net/rst.html ) as its markup language, and many of its strengths come from the power and straightforwardness of reStructuredText and its parsing and translating suite, the Docutils.

It was originally created to translate the new Python documentation, but has now been cleaned up in the hope that it will be useful to many other projects. (Of course, this site is also created from reStructuredText sources using Sphinx!)

Current Documentation System

The current method of producing the Sage documentation involves generating LaTeX source code either automatically based on the Sage source code. The HTML documentation is then produced by using latex2html. Almost all of the work for constructing the reference manual is in http://www.sagemath.org/hg/doc-main/file/e3941ce97fe4/ref/update_script.py. Other documentation such as the tutorial, programming guide, and constructions documents are just plain LaTeX documents.

There are some drawbacks for the current system of documentation. There are currently very few people that know how the code for producing the reference manual works, and the code itself is relatively difficult to extend.

By requiring LaTeX and latex2html, we reduce the number of people that can work on the documentation. People rarely check that their docstrings are correctly converted into LaTeX, and at almost every release, someone has to go through and fix all of these errors so that the documentation actually builds.

One thing that the current system does have going for it is that it is already in place and works for the most part.

Switching to Sphinx

Sphinx is quickly becoming the standard way to create documentation for Python projects. It is the documentation tool being used for Python 2.6 and 3.0. Numpy, Scipy, IPython, and Matplotlib have all switched to using Sphinx for their documentation. Sphinx can output the documentation as HTML, PDF (via LaTeX), and as a Windows Help File.

Using reStructuredText as a markup language increases the usefulness of the docstrings and documentation. For example, Epydoc can use reStructuredText docstrings to produce nice-looking autogenerated API documentation. MoinMoin has a plugin to support reStructredText which can facilitate transfer of data between the wiki and the Sage documentation. Some reasons why reStructuredText was adopted for Python docstrings can be found at http://www.python.org/dev/peps/pep-0287/ .

Since LaTeX / latex2html is not needed to produce the HTML version of the documentation and the time it takes produce the HTML documentation is much shorter. Hopefully this will encourage people to work more on the documentation when writing their patches.

Switching everything over to Sphinx and reStructuredText will be a fair amount of work. We will also need to verify that all of the tools work with Cython; if they don't, then we will have some more code to write (which will luckily be useful to many more people). The first option would be to write a script which goes through and changes all docstrings to the reStructuredText markup. This seems somewhat feasible and would be a relatively clean solution if it works. The second option would be to do things piecewise and insert hooks into the current documentation system to convert and reStructuredText docstrings into the current LaTeXish docstring format. Once all of the doctstring have been converted over, then we can make the switch to Sphinx.