|
Size: 1873
Comment:
|
Size: 2468
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| = Parallelization Plans For SAGE = | = SEP 2: Parallelization Plans For SAGE = This is a SAGE enhancement proposal. AUTHOR: William Stein COPYRIGHT: GNU Free Documentation License, 2007. |
| Line 21: | Line 26: |
| === 1. Low -- shared memory (smp or multicore) === | === 1. Low -- shared memory (mostly multicore desktop/laptop) === |
| Line 25: | Line 30: |
| Justification: * pthread is available on all target platforms and is well supported * mature * with some thought I think we can make it usable from sagex |
|
| Line 27: | Line 38: |
| Proposed tool: ipython1 | Proposed tool: ipython1 (with mpi) |
| Line 29: | Line 40: |
| === 3. High -- heterogenous task farm === | Justification: * This is the hardware that the ipython developers use. * It's written in Python, well tested, and will be included in SAGE anyways. === 3. High -- heterogenous task farm (both trusted and untrusted) === |
| Line 32: | Line 47: |
Justification: * Written in Python to address specific problems we have. |
SEP 2: Parallelization Plans For SAGE
This is a SAGE enhancement proposal.
AUTHOR: William Stein COPYRIGHT: GNU Free Documentation License, 2007.
The core SAGE library is a collection of Python and sagex files.
Basic Principles
Many of these are motivated by my (Stein's) perspective as the maintainer and integrated of SAGE, and recruiter of new developers...
- Parallel methods should always be viewed as a means to an end -- speedups. Never parallelize any computation except to speed up a calculation beyond what can be done using sequential techniques.
Parallel methods should never completely replace sequential implementations. Parallel algorithms are often very complicated to understand and test, so we need to at a minimum have a randomized test function that compares with that output of purely sequential code.
Do not write insanely complicated parallel code that nobody can understand or maintain. Because SAGE is an open source system that is widely developed, it is crucial that it be readable.
- It is *crucial* that implementation of parallel methods in SAGE have the following properties:
- It can be done incrementally. One must be able to start with almost any specific operation or algorithm in SAGE and make a parallel version without having to drastically change code all over SAGE. Any proposed solutions that violate this fail our needs.
- It doesn't depend on any libraries or tools that are not open source and free, and all dependencies must work on the SAGE target platforms: Linux, OS X, Windows, (and soon Solaris).
- For any core tools that are needed must be made part of SAGE.
Architecture
There are three levels to consider.
1. Low -- shared memory (mostly multicore desktop/laptop)
Proposed tool: pthread
Justification:
- pthread is available on all target platforms and is well supported
- mature
- with some thought I think we can make it usable from sagex
2. Middle -- homogeneous trusted cluster
Proposed tool: ipython1 (with mpi)
Justification:
- This is the hardware that the ipython developers use.
- It's written in Python, well tested, and will be included in SAGE anyways.
3. High -- heterogenous task farm (both trusted and untrusted)
Proposed tool: dsage
Justification:
- Written in Python to address specific problems we have.