Differences between revisions 6 and 26 (spanning 20 versions)
Revision 6 as of 2009-09-30 14:55:24
Size: 8389
Editor: qed777
Comment: Continued reorg.
Revision 26 as of 2009-10-17 00:15:28
Size: 14074
Editor: qed777
Comment: Update.
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= Sage Usability Tasks =

Source: [[SageUsability | Sage Usability Improvements List]]. TODO: Assign difficulties to individual tasks, if it's appropriate.
= Sage Tasks Outline =

This is a general outline of targeted areas, including specific tasks, for improving the usability of [[http://www.sagemath.org/ | Sage]]. The sources include the [[SageUsability | Sage Usability Improvements List]] and [[http://sage.math.washington.edu:8100/file/tip/sagenb/todo.txt | sagenb/todo.txt]], but the scope here is wider. Feel free to make changes.
Line 13: Line 13:
 * [[http://groups.google.com/group/sage-devel/browse_thread/thread/258c07ae672a3617 | Sage Forge]].
 * [[http://groups.google.com/group/sage-devel/browse_thread/thread/258c07ae672a3617 | SageForge]] - semi-automated repository for contributions to Sage:
   * Scripts, worksheets, databases, and packages of [nearly] all sizes.
   * Supplementary documentation, including multimedia.
   * Optional reviews and ratings.
   * Live, ulimited cellular paste-bin.
   * RSS feeds.


############################## BUILD / TEST ##############################
== Build / Test ==

 * Tests:
   * Increase coverage to 100%.
   * Fix all sagenb doctests and run them from Sage library.
   * Meaningful statistics / benchmarks, e.g., %timeit for individual doctests.

 * Build logs:
   * Separate file for each spkg for parallel builds.
   * Group by install, upgrade, or spkg install event.
   * Remotely aggregate and display refreshed statistics.

 * Porting:
   * !VirtualBox Sage.
   * andLinux Sage.
   * Windows.
   * Windows build farm.
   * Sagelite for Windows.
   * Binary distributions that don't use SSE3, etc.

 * Automated notebook benchmarking and testing:
   * Functional tests: [[http://seleniumhq.org/ | Selenium]], [[http://www.getwindmill.com/ | Windmill]].
   * Load tests: [[http://funkload.nuxeo.org/ | FunkLoad]], [[http://www.pylot.org/ | Pylot]], [[http://watir.com/ | Watir]], [[http://pypi.python.org/pypi/zope.testbrowser | Zope.testbrowser]].


############################## DEVELOPMENT ##############################
== Development ==

 * Auto-reload / auto-build changed files - [[http://www.sagemath.org/doc/reference/sage/misc/attach.html | attach]], [[http://pythonpaste.org/script/ | paster serve --reload]], !CherryPy's [[http://www.cherrypy.org/wiki/AutoReload | AutoReload]].

 * [[http://groups.google.com/group/sage-devel/msg/fb95603488afd9de | More convenient development workflow]]:
   * Code review tools - [[http://code.google.com/p/rietveld/ | Rietveld]].
   * Automated, continuous builds and testing for new patches.

 * [[http://www.doughellmann.com/articles/CompletelyDifferent-2008-03-linters/index.html | Static Python code analysis]].


############################## DOCUMENTATION ##############################
Line 17: Line 62:
 * [[http://groups.google.com/group/sage-devel/browse_thread/thread/8fa0d854b6928f2b/dc4cfe0a941762aa#dc4cfe0a941762aa | Allow $ for reST math]] - [[http://trac.sagemath.org/sage_trac/ticket/6892 | #6892]].
Line 20: Line 63:
   * `doc/en/sagenb` for new notebook.
   * Add modules to the reference manual.
Line 22: Line 67:
   * [[SageNotebook | Notebook server technical overview]].
   * [[http://groups.google.com/group/sage-support/browse_thread/thread/b0d15b82eea3d652 | Run notebook server as daemon]].
Line 27: Line 73:
 * Authentication:  * Don't list '''all''' users' worksheets.

 * Authentication - abstract interface:
Line 31: Line 79:
 * Dashboard for server settings / status:
   * View logged-in users, load, processes, etc.
   * Backups: all critical data, selective.
   * Configure thresholds, alerts, snapshot behavior.
   * Manage user accounts, groups, privileges.
   * Restricted version for users.
 * Dashboard:
   * Real-time and cumulative statistics, aggregate and per user.
   * Backups: complete or selective.
   * Settings: "everything," hidden analytics tags, geolocation ([[http://www.clustrmaps.com/ | ClustrMaps]]).
   * Manage user accounts, passwords, groups, privileges.
   * Send email to all registered users.
   * Install new / updated modules and spkgs.
Line 39: Line 88:
   * [[http://recaptcha.net/ | reCAPTCHA]].
   * Challenge questions.
   * Admin approval.
   * Revokable custom tokens.
   * Admin moderated.
   * Challenge-response - simple questions, [[http://recaptcha.net/ | reCAPTCHA]], revokable tokens - [[http://trac.sagemath.org/sage_trac/ticket/7158 | #7158]].
Line 48: Line 95:
   * Use secure tokens to reset passwords.    * Secure tokens to reset passwords.
Line 75: Line 122:

 * %cython mode, full and outside `sage`.

 * Slideshow mode.

 * interacts:
   * Auto updates.
   * Live published.
   * Control over widget layout.
   * Examples to include with Sage.
   * [[http://groups.google.com/group/sage-devel/browse_thread/thread/8aa6a3a84365f06d/42c7edf1c23ca169?#42c7edf1c23ca169 | Free-form workspace with pipelines]].
   * Display nicely formatted docstrings (or captions).
   * Simplify extension, via HTML templates or an API.
   * More widgets: vertical sliders, spreadsheets, interactive plots, etc.
Line 86: Line 147:
 * Math-enabled chat. [[http://qwebirc.org/ | qwebirc]] / [[http://webchat.freenode.net/ | Web IRC]].

* Python debugger a la [[https://developer.mozilla.org/En/Pylons | Pylons]].
 * Chat - [[http://qwebirc.org/ | qwebirc]], [[http://webchat.freenode.net/ | Web IRC]].
   * Rich content, e.g., equations.
   * Sage compute bot.
   * Add to help page.

 * Debugging:
   * Prettier tracebacks for %modes.
  
* Python debugger a la [[https://developer.mozilla.org/En/Pylons | Pylons]].
Line 94: Line 160:
 * Editor a la [[https://bespin.mozilla.com/ | BeSpin]]  * Code editor a la [[https://bespin.mozilla.com/ | BeSpin]]
Line 98: Line 164:
   * Browse worksheet directories like a file system.
Line 100: Line 167:


######################### NOTEBOOK: UNDER THE HOOD #########################
== Notebook: Under the hood ==

 * Implement `base_url`.

 * Deprecation message at top of '''old''' notebook `.py` files.

 * Move `is_valid_username`, etc., out of `twist.py`.

 * Updated jsMath image fonts spkg - [[http://trac.sagemath.org/sage_trac/ticket/7229 | #7229]].

 * `notebook()`:
   * Allow default systems besides 'sage'.
   * Allow a different sage command on a remote machine.

 * Manage Jmol applet count / memory use.

 * Use Python's [[http://docs.python.org/library/logging.html | logging facility]].

 * [[http://bitbucket.org/brickenstein/rumcomponent/src/tip/rumcomponent/ | Component architecture]].

 * Cythonize pexpect.

 * Processes:
   * Balance server pool loads.
   * Run the worksheet process as another user.
   * Reference implementation:
     * Make ctrl-c quit.
     * Don't put "data" link in all output cells.
   * Upload worksheets without blocking the server.
   * Remote worksheet processes.
   * Use fork, named pipes, and/or [[http://docs.python.org/library/mmap.html | memory-mapped]] files for worksheet processes.

 * Client-server and server-server interaction:
   * TinyMCE problem.
   * Load JS / CSS components on demand - [[http://ajile.net/ | Ajile]].
   * Load browser-specific code only in the specific browser.
   * Better handling of concurrent worksheet use.
   * Server-to-server backups, possibly with authentication.

 * Decouple secure transport from sage/dsage. Use OpenSSL.

 * Remote access API - [[http://code.google.com/apis/documents/overview.html | Google Documents List Data API]].
   * (Non-)authenticated modes.
   * Manipulate, share, publish worksheets at a given Sage(Forge) server.
   * Python module, included with Sage.

 * Storage:
   * Use HTML format for worksheets.
   * Fix `worksheet.autosave()`.
   * Always retrieve worksheet content from disk on open.
   * Use a revision control tool (e.g., Mercurial) for undo, history, snapshots, etc.
   * Deprecate the [unused] `objects` directory.
   * File size limit - auto-delete and warn above threshold.
   * Access a user's data directory securely, e.g., via an API.
   * Handle permissions better for directories, history, output, conf files, etc.
   * Filesystem or a database determines worksheet list, not an sobj.

 * Migrate to server / framework: [[http://docs.djangoproject.com/en/dev/ | Django]], [[http://pylonshq.com/ | Pylons]], [[http://pyjs.org/ | Pyjamas]], [[http://www.tornadoweb.org/ | Tornado]], etc.:
   * Switch from twisted.web2 to [[http://jcalderone.livejournal.com/tag/sixty+seconds | twisted.web]].
   * [[http://routes.groovie.org/ | URL routes]].
Line 106: Line 236:
   * Reorganize help page.    * Reorganize help page:
     * Search for commands.
     * Linking to similar and related commands.
Line 110: Line 242:
   * Parens matching - [[http://trac.sagemath.org/sage_trac/ticket/3646 | #3646]].
Line 113: Line 244:
   * Typeset output default.
   * I
mplicit multiplication default.
   * Defaults: typeset, implicit multiplication, rename "Untitled" worksheets.
Line 117: Line 247:
   * Easy backups of own work, with or w/o data, output, snapshots, etc.    * Easy backups, selective or complete.
Line 121: Line 251:
   * Restart worksheet shortcut.
   * Splash-screen with common keys.
Line 122: Line 254:
   * Restart worksheet shortcut.
   * Custom shortcuts, e.g., ctrl-bksp to delete previous word, not join cells.
   * Use js-hotkeys.
   * Splash-screen with common keys.

 * 2D / 3D plots:
   * !JavaScript animations.
   * Mouse-over for [[http://groups.google.com/group/sage-support/browse_thread/thread/c29abdaaa4f0ee13 | coordinates]] - [[http://sourceforge.net/mailarchive/forum.php?thread_name=6e8d907b0909012053s19e7c5fl29cdcd0e1fab1d50%40mail.gmail.com&forum_name=matplotlib-devel | matplotlib]].
   * Interactively zoom, pan, annotate - canvas, SVG, [[http://code.google.com/p/flot/ | Flot]], [[http://teethgrinder.co.uk/open-flash-chart-2/ | Open Flash Chart]].
   * Custom shortcuts, e.g., ctrl-bksp to delete previous word, not join cells - [[http://code.google.com/p/js-hotkeys/ | js-hotkeys]].
Line 133: Line 257:
   * Slideshow mode.
Line 138: Line 261:
   * Toolbar with buttons to interrupt running code, save, attach files, etc.
Line 141: Line 263:

 * 2D / 3D plots:
   * !JavaScript animations.
   * [[http://deanm.github.com/pre3d/ | Pre3d]]-[[http://groups.google.com/group/sage-devel/browse_thread/thread/0315e138d5783377/742d9f9bb00b0cd7?#742d9f9bb00b0cd7 | enabled `canvas3d` viewer]].
   * Mouse-over for [[http://groups.google.com/group/sage-support/browse_thread/thread/c29abdaaa4f0ee13 | coordinates]] - [[http://sourceforge.net/mailarchive/forum.php?thread_name=6e8d907b0909012053s19e7c5fl29cdcd0e1fab1d50%40mail.gmail.com&forum_name=matplotlib-devel | matplotlib]].
   * Interactively zoom, pan, annotate - canvas, SVG, [[http://code.google.com/p/flot/ | Flot]], [[http://teethgrinder.co.uk/open-flash-chart-2/ | Open Flash Chart]].

 * Worksheets:
   * Export screen with checkboxes for images, data, etc.
   * Use notebook as IDE for running sage instance.
   * Optional always visible menubar at top of worksheet.
   * Make "Browse your computer" upload fields editable.
   * Fix "undefined" bug on first save in Safari 4.0.3.
   * Don't put "..." in default filename when saving.
   * Don't list "Sage Users" when sharing.
   * Rate worksheet title shows false error.
   * Page through worksheet list.
   * "Save as" option, e.g., for downloading, that prompts for a name.
   * Toolbar with buttons to interrupt running code, save, attach files, etc.
   * Parens matching - [[http://trac.sagemath.org/sage_trac/ticket/3646 | #3646]].
   * TinyMCE [[http://groups.google.com/group/sage-devel/msg/e2eb3f848a21faa3 | synchronization]] [[http://groups.google.com/group/sage-devel/browse_frm/thread/4cd1cc188470c723/c2477e1bdba68028?q=tinymce+bug+group:sage-devel#c2477e1bdba68028 | problem]].
Line 147: Line 290:
   * Publishing should update "last changed" time, even for old worksheets.
Line 149: Line 293:
   * Suppress browser warning for Konquereor.
   * Unique cookies for logins from different browsers / computers.
   * Automatically save / restore worksheet state. Pause / resume.
Line 153: Line 294:
   * Suppress browser warning for Konqueror.
   * Manage logins:
     * View recent account activity. Add last login to `user.py`.
     * Invalidate other sessions' cookies.
     * Unique cookies for logins from different browsers / computers.
Line 155: Line 301:
 * Worksheets:
   * Optional always visible menubar at top of worksheet.
   * TinyMCE [[http://groups.google.com/group/sage-devel/msg/e2eb3f848a21faa3 | synchronization]] [[http://groups.google.com/group/sage-devel/browse_frm/thread/4cd1cc188470c723/c2477e1bdba68028?q=tinymce+bug+group:sage-devel#c2477e1bdba68028 | problem]].

 * Fix MoinMoin.

 * Use notebook as IDE for running sage instance.

 * Externally embed live cells / worksheets. Live published workseets. [[http://trac.sagemath.org/sage_trac/ticket/6855 | #6855]].


######################### NOTEBOOK: UNDER THE HOOD #########################
== Notebook: Under the hood ==

 * Better handling of concurrent worksheet use.

 * Smartly manage Jmol applet count, or memory use.

 * [[http://bitbucket.org/brickenstein/rumcomponent/src/tip/rumcomponent/ | Component architecture]].

 * Cythonize pexpect.

 * Client-server synchronization:
   * TinyMCE problem.
   * Server-to-server for backups, possibly with authentication.

 * Track worksheets and user data with Mercurial.

 * Balance server pool loads.

 * Load JS / CSS components on demand.

 * Migrate to server / framework: [[http://docs.djangoproject.com/en/dev/ | Django]], [[http://pylonshq.com/ | Pylons]], [[http://pyjs.org/ | Pyjamas]], [[http://www.tornadoweb.org/ | Tornado]], etc.:
  * [[http://routes.groovie.org/ | URL routes]].


############################## BUILD / TEST ##############################
== Build / Test ==

 * !VirtualBox Sage.

 * andLinux Sage.

 * Binary distibutions that don't use SSE3, etc.

 * Windows build farm.

 * Windows port.

 * Notebook testing infrastructure: [[http://seleniumhq.org/ | Selenium]], [[http://testled.com/ | Testled]], [[http://code.google.com/p/webdriver/ | Webdriver]], [[http://pylonshq.com/docs/en/0.9.7/testing/ | WebTest]], [[http://www.getwindmill.com/ | Windmill]].
 * Fix !MoinMoin.

 * External:
   * Embed live cells / worksheets - [[http://trac.sagemath.org/sage_trac/ticket/6855 | #6855]].
   * [[http://wave.google.com/ | Wave]] [[http://code.google.com/apis/wave/guide.html | gadgets and robots]].
   * Live published worksheets.

Sage Tasks Outline

This is a general outline of targeted areas, including specific tasks, for improving the usability of Sage. The sources include the Sage Usability Improvements List and sagenb/todo.txt, but the scope here is wider. Feel free to make changes.

1. General

  • Y. Aner's notebook threat model (PDF).

  • Keith Lang's Top 10 UI Myths.

  • SageForge - semi-automated repository for contributions to Sage:

    • Scripts, worksheets, databases, and packages of [nearly] all sizes.
    • Supplementary documentation, including multimedia.
    • Optional reviews and ratings.
    • Live, ulimited cellular paste-bin.
    • RSS feeds.

2. Build / Test

  • Tests:
    • Increase coverage to 100%.
    • Fix all sagenb doctests and run them from Sage library.
    • Meaningful statistics / benchmarks, e.g., %timeit for individual doctests.
  • Build logs:
    • Separate file for each spkg for parallel builds.
    • Group by install, upgrade, or spkg install event.
    • Remotely aggregate and display refreshed statistics.
  • Porting:
    • VirtualBox Sage.

    • andLinux Sage.
    • Windows.
    • Windows build farm.
    • Sagelite for Windows.
    • Binary distributions that don't use SSE3, etc.
  • Automated notebook benchmarking and testing:

3. Development

4. Documentation

5. Notebook: Admin

  • Don't list all users' worksheets.

  • Authentication - abstract interface:
    • LDAP.
    • UNIX accounts.
  • Dashboard:
    • Real-time and cumulative statistics, aggregate and per user.
    • Backups: complete or selective.
    • Settings: "everything," hidden analytics tags, geolocation (ClustrMaps).

    • Manage user accounts, passwords, groups, privileges.
    • Send email to all registered users.
    • Install new / updated modules and spkgs.
  • Registration:
    • Admin moderated.
    • Challenge-response - simple questions, reCAPTCHA, revokable tokens - #7158.

  • Users:
    • Groups.
    • Privileges.
    • Email address as username.
    • Secure tokens to reset passwords.

6. Notebook: Introspection

  • Use Sphinx for sagenb.org docstrings.
  • Matrix B mentioned twice in B.solve_left?: B.solve_left(B, ...).

  • Better error detection, e.g., for Maxima command completion.
  • Completions link for mobiles.
  • Live introspection cells.
  • Introspection cross-links.
  • Improve introspection / completion interface: floating div, RHS div.
  • New documentation browser. Send worksheet docstrings to this browser.

7. Notebook: "Modes"

  • %timeit.
  • %cython mode, full and outside sage.

  • Slideshow mode.
  • interacts:
    • Auto updates.
    • Live published.
    • Control over widget layout.
    • Examples to include with Sage.
    • Free-form workspace with pipelines.

    • Display nicely formatted docstrings (or captions).
    • Simplify extension, via HTML templates or an API.
    • More widgets: vertical sliders, spreadsheets, interactive plots, etc.
  • Graph editor - #1321.

  • LaTeX doc mode.

  • reST:
  • Preparse in the browser: implicit multiplication, less work for server.
  • Chat - qwebirc, Web IRC.

    • Rich content, e.g., equations.
    • Sage compute bot.
    • Add to help page.
  • Debugging:
    • Prettier tracebacks for %modes.
    • Python debugger a la Pylons.

  • Vector graphics - #6973.

    • Draw - SVG-edit.

    • Annotate output from matplotlib's SVG backend.
  • Code editor a la BeSpin

    • Command-line history.
    • Syntax highlighting.
    • Context-sensitive completion.
    • Browse worksheet directories like a file system.
  • Math input GUI plug-ins.

8. Notebook: Under the hood

  • Implement base_url.

  • Deprecation message at top of old notebook .py files.

  • Move is_valid_username, etc., out of twist.py.

  • Updated jsMath image fonts spkg - #7229.

  • notebook():

    • Allow default systems besides 'sage'.
    • Allow a different sage command on a remote machine.
  • Manage Jmol applet count / memory use.
  • Use Python's logging facility.

  • Component architecture.

  • Cythonize pexpect.
  • Processes:
    • Balance server pool loads.
    • Run the worksheet process as another user.
    • Reference implementation:
      • Make ctrl-c quit.
      • Don't put "data" link in all output cells.
    • Upload worksheets without blocking the server.
    • Remote worksheet processes.
    • Use fork, named pipes, and/or memory-mapped files for worksheet processes.

  • Client-server and server-server interaction:
    • TinyMCE problem.
    • Load JS / CSS components on demand - Ajile.

    • Load browser-specific code only in the specific browser.
    • Better handling of concurrent worksheet use.
    • Server-to-server backups, possibly with authentication.
  • Decouple secure transport from sage/dsage. Use OpenSSL.
  • Remote access API - Google Documents List Data API.

    • (Non-)authenticated modes.
    • Manipulate, share, publish worksheets at a given Sage(Forge) server.
    • Python module, included with Sage.
  • Storage:
    • Use HTML format for worksheets.
    • Fix worksheet.autosave().

    • Always retrieve worksheet content from disk on open.
    • Use a revision control tool (e.g., Mercurial) for undo, history, snapshots, etc.
    • Deprecate the [unused] objects directory.

    • File size limit - auto-delete and warn above threshold.
    • Access a user's data directory securely, e.g., via an API.
    • Handle permissions better for directories, history, output, conf files, etc.
    • Filesystem or a database determines worksheet list, not an sobj.
  • Migrate to server / framework: Django, Pylons, Pyjamas, Tornado, etc.:

9. Notebook: User Interaction

  • Help:
    • Reorganize help page:
      • Search for commands.
      • Linking to similar and related commands.
    • Easy set up of shipped jsMath fonts.
    • Automatic tips for keyboard shortcuts, editing features, etc. Opt out.
    • Friendly bug buddy that emails reports, bundles session history, etc.
  • Customization:
    • Defaults: typeset, implicit multiplication, rename "Untitled" worksheets.
    • Custom window close callback.
    • Function to update worksheet titlebar.
    • Easy backups, selective or complete.
    • Download zip files of multiple worksheets, even if accounts=True.

  • Keys
    • Restart worksheet shortcut.
    • Splash-screen with common keys.
    • Insert new cell after current cell shortcut.
    • Custom shortcuts, e.g., ctrl-bksp to delete previous word, not join cells - js-hotkeys.

  • Cells:
    • Auto-updating cells, e.g., interacts.
    • Prevent text movement by a pixel on cell frame focus.
    • Always show links for long output cells, regardless of length or interruption.
    • Easily identify idle, queued, and running cells.
    • Select, delete, (un)group, move multiple cells.

    • Leading space input bug - #6729.

  • 2D / 3D plots:
  • Worksheets:
    • Export screen with checkboxes for images, data, etc.
    • Use notebook as IDE for running sage instance.
    • Optional always visible menubar at top of worksheet.
    • Make "Browse your computer" upload fields editable.
    • Fix "undefined" bug on first save in Safari 4.0.3.
    • Don't put "..." in default filename when saving.
    • Don't list "Sage Users" when sharing.
    • Rate worksheet title shows false error.
    • Page through worksheet list.
    • "Save as" option, e.g., for downloading, that prompts for a name.
    • Toolbar with buttons to interrupt running code, save, attach files, etc.
    • Parens matching - #3646.

    • TinyMCE synchronization problem.

  • Published worksheets:
    • Delete them.
    • RSS feeds.
    • Live cells, e.g, interacts.
    • Only to selected users or groups.
    • Publishing should update "last changed" time, even for old worksheets.
  • Notebook:
    • Tags.

    • Suppress browser warning for Konqueror.
    • Manage logins:
      • View recent account activity. Add last login to user.py.

      • Invalidate other sessions' cookies.
      • Unique cookies for logins from different browsers / computers.
    • Full-text search: SQLalchemy, Whoosh (codenode-devel).

  • Fix MoinMoin.

  • External: