Differences between revisions 4 and 22 (spanning 18 versions)
Revision 4 as of 2009-09-30 14:07:18
Size: 7678
Editor: qed777
Comment: Continued import.
Revision 22 as of 2009-10-03 07:15:19
Size: 12419
Editor: qed777
Comment: Test frameworks.
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= Sage Usability Tasks =

Source: [[SageUsability | Sage Usability Improvements List]]. This is an attempt to organize the desired changes into tasks of varying difficulty (e.g., easy, medium, hard).
= Sage Tasks Outline =

This is a general outline of targeted areas, including specific tasks, for improving the usability of [[http://www.sagemath.org/ | Sage]]. The sources include the [[SageUsability | Sage Usability Improvements List]] and [[http://sage.math.washington.edu:8100/file/tip/sagenb/todo.txt | sagenb/todo.txt]], but the scope here is wider. Feel free to make changes.


############################## GENERAL ##############################
Line 11: Line 13:
 * [[http://groups.google.com/group/sage-devel/browse_thread/thread/258c07ae672a3617 | Sage Forge]].
 * [[http://groups.google.com/group/sage-devel/browse_thread/thread/258c07ae672a3617 | SageForge]].


############################## BUILD / TEST ##############################
== Build / Test ==

 * Tests:
   * Increase coverage.
   * Meaningful statistics / benchmarks, e.g., %timeit for individual doctests.

 * Build logs:
   * Separate file for each spkg for parallel builds.
   * Group by install, upgrade, or spkg install event.
   * Remotely aggregate and display refreshed statistics.

 * Porting:
   * !VirtualBox Sage.
   * andLinux Sage.
   * Windows.
   * Windows build farm.
   * Sagelite for Windows.
   * Binary distributions that don't use SSE3, etc.

 * Automated notebook benchmarking and testing:
   * Functional tests: [[http://seleniumhq.org/ | Selenium]], [[http://www.getwindmill.com/ | Windmill]].
   * Load tests: [[http://funkload.nuxeo.org/ | FunkLoad]], [[http://watir.com/ | Watir]], [[http://pypi.python.org/pypi/zope.testbrowser | Zope.testbrowser]].


############################## DEVELOPMENT ##############################
== Development ==

 * Auto-reload / auto-build changed files - [[http://www.sagemath.org/doc/reference/sage/misc/attach.html | attach]], [[http://pythonpaste.org/script/ | paster serve --reload]], !CherryPy's [[http://www.cherrypy.org/wiki/AutoReload | AutoReload]].

 * [[http://groups.google.com/group/sage-devel/msg/fb95603488afd9de | More convenient development workflow]]:
   * Code review tools - [[http://code.google.com/p/rietveld/ | Rietveld]].
   * Automated, continuous builds and testing for new patches.

 * [[http://www.doughellmann.com/articles/CompletelyDifferent-2008-03-linters/index.html | Static Python code analysis]].


############################## DOCUMENTATION ##############################
Line 17: Line 58:
 * [[http://groups.google.com/group/sage-devel/msg/35e97a1d1cedfdfd | Document pexpect interface]].

* [[http://groups.google.com/group/sage-devel/browse_thread/thread/6d94c8655e41ba24# | Document notebook port forwarding]].
 * Document:
  
* [[http://groups.google.com/group/sage-devel/msg/35e97a1d1cedfdfd | pexpect interface for users]].
  
* [[http://groups.google.com/group/sage-devel/browse_thread/thread/6d94c8655e41ba24# | notebook port forwarding]].
   * [[SageNotebook | Notebook server technical overview]].


############################## NOTEBOOK: A
DMIN ##############################
Line 23: Line 67:
 * Dashboard for server settings / status.
   * View
logged-in users, load, processes, etc.
   * Backup all critical data.
 * Authentication - abstract interface:
   * LDAP.
   * UNIX accounts.

* Dashboard for server settings / status:
   * Statistics: all /
logged-in users, load, processes, worksheets [per user], etc.
   * Backups: all critical data, selective.
Line 29: Line 77:

 * Selective registration with [[http://recaptcha.net/ | reCAPTCHA]], challenge questions, admin approval, and/or revokable custom tokens.

 * Custom authentication: LDAP, UNIX accounts, etc.

 * Password "recovery:" Use secure tokens.

 * Email address as username.

 * User groups.

 * Privileged users, e.g, admins.
   * Install new / updated modules and spkgs.

 * Registration:
   * [[http://recaptcha.net/ | reCAPTCHA]].
   * Challenge questions.
   * Admin approval.
   * Revokable custom tokens.

 * Users:
   * Groups.
   * Privileges.
   * Email address as username.
   * Use secure tokens to reset passwords.


######################### NOTEBOOK: INTROSPECTION #########################
Line 46: Line 97:
 * Matrix `B` mentioned twice in `B.solve_left?`: `B.solve_left(B, ...)`.

 * Better error detection, e.g., for Maxima command completion.

 * Completions link for mobiles.
Line 50: Line 107:
 * [[http://groups.google.com/group/sage-devel/browse_thread/thread/e9d0fc2c75ec6354/2634152df8f85f59?#2634152df8f85f59 | New documentation browser]]. Send docstrings to this browser.

 * Better error detection, e.g., for Maxima command completion.

 * Matrix `B` mentioned twice in `B.solve_left?`: `B.solve_left(B, ...)`.

 * Completions link for mobiles.
Line 60: Line 109:
 * [[http://groups.google.com/group/sage-devel/browse_thread/thread/e9d0fc2c75ec6354/2634152df8f85f59?#2634152df8f85f59 | New documentation browser]]. Send worksheet docstrings to this browser.


############################## NOTEBOOK: "MODES" ##############################
Line 62: Line 115:
 * %timeit.

 * %cython mode, full and outside `sage`.

 * Slideshow mode.

 * interacts:
   * Auto updates.
   * Live published.
   * Control over widget layout.
   * Examples to include with Sage.
   * [[http://groups.google.com/group/sage-devel/browse_thread/thread/8aa6a3a84365f06d/42c7edf1c23ca169?#42c7edf1c23ca169 | Free-form workspace with pipelines]].
   * Display nicely formatted docstrings (or captions).
   * Simplify extension, via HTML templates or an API.
   * More widgets: vertical sliders, spreadsheets, interactive plots, etc.
Line 66: Line 135:
 * reST editing.  * reST:
Line 68: Line 137:
   * `%rest` cells (auto-hide, double-click to show) sent to Sphinx on-the-fly.

 * %timeit.
   * `%rest` cells (auto-hide, double-click to show) converted to HTML by Sphinx.
Line 74: Line 141:
 * Chat - [[http://qwebirc.org/ | qwebirc]], [[http://webchat.freenode.net/ | Web IRC]].
   * Rich content, e.g., equations.
   * Sage compute bot.
   * Add to help page.

 * Debugging:
   * Prettier tracebacks for %modes.
   * Python debugger a la [[https://developer.mozilla.org/En/Pylons | Pylons]].

 * [[http://groups.google.com/group/sage-devel/browse_thread/thread/86429406fb3bba17/dd3163cb2a47fa38?#dd3163cb2a47fa38 | Vector graphics]] - [[http://trac.sagemath.org/sage_trac/ticket/6973 | #6973]].
   * Draw - [[http://code.google.com/p/svg-edit/ | SVG-edit]].
   * Annotate output from matplotlib's SVG backend.

 * Code editor a la [[https://bespin.mozilla.com/ | BeSpin]]
   * Command-line history.
   * Syntax highlighting.
   * Context-sensitive completion.
Line 76: Line 161:
 * [[http://groups.google.com/group/sage-devel/browse_thread/thread/86429406fb3bba17/dd3163cb2a47fa38?#dd3163cb2a47fa38 | Vector graphics editor]] - [[http://code.google.com/p/svg-edit/ | SVG-edit]].

######################### NOTEBOOK: UNDER THE HOOD #########################
== Notebook: Under the hood ==

 * Implement `base_url`.

 * Move `is_valid_username`, etc., out of `twist.py`.

 * `notebook()`:
   * Allow `default_mode='python'`, i.e., not 'sage'.
   * Allow a different sage command on a remote machine.

 * Smartly manage Jmol applet count, or memory use.

 * Use Python's [[http://docs.python.org/library/logging.html | logging facility]].

 * [[http://bitbucket.org/brickenstein/rumcomponent/src/tip/rumcomponent/ | Component architecture]].

 * Cythonize pexpect.

 * Processes:
   * Balance server pool loads.
   * Run the worksheet process as another user.
   * Make ctrl-c quit the server's reference implementation.
   * Upload worksheets without blocking the server.
   * Remote worksheet processes.
   * Use fork, named pipes, and/or [[http://docs.python.org/library/mmap.html | memory-mapped]] files for worksheet processes.

 * Client-server and server-server interaction:
   * TinyMCE problem.
   * Load JS / CSS components on demand.
   * Load browser-specific code only in the specific browser.
   * Better handling of concurrent worksheet use.
   * Server-to-server for backups, possibly with authentication.

 * Decouple secure transport from sage/dsage. Use OpenSSL.

 * Storage:
   * Implement infinite undo with revision logs, not snapshots.
   * Track user data / worksheets with Mercurial.
   * Deprecate the [unused] `objects` directory.
   * Access a user's data directory securely, e.g., via an API.
   * Use the filesystem or a database, '''not''' an sobj:
     * Worksheet list.
     * Notebook configuration.

 * Migrate to server / framework: [[http://docs.djangoproject.com/en/dev/ | Django]], [[http://pylonshq.com/ | Pylons]], [[http://pyjs.org/ | Pyjamas]], [[http://www.tornadoweb.org/ | Tornado]], etc.:
   * Switch from twisted.web2 to [[http://jcalderone.livejournal.com/tag/sixty+seconds | twisted.web]].
   * [[http://routes.groovie.org/ | URL routes]].


######################### NOTEBOOK: USER INTERACTION #########################
Line 80: Line 215:
 * Automatic tips for keyboard shortcuts, editing features, etc. Opt out.

 * Friendly bug buddy that emails reports, bundles session history, etc.

 * Easy backups of own work, with or w/o data, output, snapshots, etc.

 * Toolbar with buttons to interrupt running code, save, get key help, etc.

 * Easily identify idle, queued, and running cells.
 * Help:
   * Reorganize help page.
        * Search for commands
        * linking to similar and related commands
   * Easy set up of shipped jsMath fonts.
   * Automatic tips for keyboard shortcuts, editing features, etc. Opt out.
   * Friendly bug buddy that emails reports, bundles session history, etc.

 * Customization:
   * Typeset output default setting.
   * Implicit multiplication default setting.
   * Custom window close callback.
   * Function to update worksheet titlebar.
   * Easy backups of own work, with or w/o data, output, snapshots, etc.
   * Download zip files of multiple worksheets, even if `accounts=True`.
Line 91: Line 232:
   * Restart worksheet shortcut.
   * Splash-screen with common keys.
Line 92: Line 235:
   * Restart worksheet shortcut.
   * Custom shortcuts, e.g., ctrl-bksp to delete previous word, not join cells.
   * Use js-hotkeys.
   * Splash-screen with common keys.

 * Interactive 2D/3D plots
   * [[http://groups.google.com/group/sage-support/browse_thread/thread/c29abdaaa4f0ee13 | coordinates]], zoom, pan, annotate
   * [[http://sourceforge.net/mailarchive/forum.php?thread_name=6e8d907b0909012053s19e7c5fl29cdcd0e1fab1d50%40mail.gmail.com&forum_name=matplotlib-devel | matplotlib]], canvas, SVG, [[http://code.google.com/p/flot/ | Flot]], [[http://teethgrinder.co.uk/open-flash-chart-2/ | Open Flash Chart]].

 * Animated 2D/3D plots using only !JavaScript.

 * Always show long output links, regardless of interrupt or length.

 * Custom window close callback.

 * Suppress browser warning for Konquereor.

 * Unique cookie for each login.

 * Cells: Select, delete, (un)group, move [[http://boagworld.com/demos/sitemap/ | multiple cells]].

 * Slideshow mode.
   * Custom shortcuts, e.g., ctrl-bksp to delete previous word, not join cells - [[http://code.google.com/p/js-hotkeys/ | js-hotkeys]].

 * Cells:
   * Auto-updating cells, e.g., interacts.
   * Prevent text movement by a pixel on cell frame focus.
   * Always show links for long output cells, regardless of length or interruption.
   * Easily identify idle, queued, and running cells.
   * Select, delete, (un)group, move [[http://boagworld.com/demos/sitemap/ | multiple cells]].
   * Leading space input bug - [[http://trac.sagemath.org/sage_trac/ticket/6729 | #6729]].

 * 2D / 3D plots:
   * !JavaScript animations.
   * Mouse-over for [[http://groups.google.com/group/sage-support/browse_thread/thread/c29abdaaa4f0ee13 | coordinates]] - [[http://sourceforge.net/mailarchive/forum.php?thread_name=6e8d907b0909012053s19e7c5fl29cdcd0e1fab1d50%40mail.gmail.com&forum_name=matplotlib-devel | matplotlib]].
   * Interactively zoom, pan, annotate - canvas, SVG, [[http://code.google.com/p/flot/ | Flot]], [[http://teethgrinder.co.uk/open-flash-chart-2/ | Open Flash Chart]].

 * Worksheets:
   * Export screen with checkboxes for images, data, etc.
   * Use notebook as IDE for running sage instance.
   * Always visible menubar at top of worksheet.
   * Make "Browse your computer" upload fields editable.
   * Fix "undefined" bug on first save in Safari 4.0.3.
   * Don't list "Sage Users" when sharing.
   * "Save as" option, e.g., for downloading, that prompts for a name.
   * Toolbar with buttons to interrupt running code, save, attach files, etc.
   * Parens matching - [[http://trac.sagemath.org/sage_trac/ticket/3646 | #3646]].
   * TinyMCE [[http://groups.google.com/group/sage-devel/msg/e2eb3f848a21faa3 | synchronization]] [[http://groups.google.com/group/sage-devel/browse_frm/thread/4cd1cc188470c723/c2477e1bdba68028?q=tinymce+bug+group:sage-devel#c2477e1bdba68028 | problem]].
Line 121: Line 268:
 * jsMath fonts:
   * Ship with Sage.
   * Clear install directions.

 * TinyMCE [[http://groups.google.com/group/sage-devel/msg/e2eb3f848a21faa3 | synchronization]] [[http://groups.google.com/group/sage-devel/browse_frm/thread/4cd1cc188470c723/c2477e1bdba68028?q=tinymce+bug+group:sage-devel#c2477e1bdba68028 | problem]].

 * Default options:
   * Implicit multiplication.
   * Typeset output.

 * Reorganize help page.

 * Automatically save / restore worksheet state. Pause / resume.

 * Function to update worksheet titlebar.

 * Auto-update cells, e.g., interacts.

 * Full-text search: SQLalchemy, [[http://whoosh.ca/ | Whoosh]] ([[http://groups.google.com/group/codenode-devel/browse_thread/thread/d3ffefa3b09937b6/98bdf00f65441934?#98bdf00f65441934 | codenode-devel]]).

 * Math-enabled chat. [[http://qwebirc.org/ | qwebirc]] / [[http://webchat.freenode.net/ | Web IRC]].

 * [[http://groups.google.com/group/sage-devel/browse_thread/thread/e46f0a59d5cf70dc | Tags]].

 * Externally embed live cells / worksheets. Live published workseets. [[http://trac.sagemath.org/sage_trac/ticket/6855 | #6855]].

 * Prevent text movement by a pixel on cell frame focus.

 * Leading space input bug - [[http://trac.sagemath.org/sage_trac/ticket/6729 | #6729]].

 * Python debugger a la [[https://developer.mozilla.org/En/Pylons | Pylons]].

 * Editor a la [[https://bespin.mozilla.com/ | BeSpin]]
   * Command-line history.
   * Syntax highlighting.
   * Context-sensitive completion.

 * Fix MoinMoin.

 * Use notebook as IDE for running sage instance.

 * Download zip files of multiple worksheets, even if `accounts=True`.

 * Parens matching - [[http://trac.sagemath.org/sage_trac/ticket/3646 | #3646]].

 * Optional always visible menubar at top of worksheet.

== Notebook: Under the hood ==

 * Better handling of concurrent worksheet use.

 * Smartly manage Jmol applet count, or memory use.

 * [[http://bitbucket.org/brickenstein/rumcomponent/src/tip/rumcomponent/ | Component architecture]].

 * Cythonize pexpect.

 * Client-server synchronization:
   * TinyMCE problem
   * Server-to-server sync for backups, possibly with authentication.

 * Track worksheets and user data with Mercurial.

 * Balance server pool loads.

 * Load JS / CSS components on demand.

 * Migrate to another server and/or framework: [[http://docs.djangoproject.com/en/dev/ | Django]], [[http://pylonshq.com/ | Pylons]], [[http://pyjs.org/ | Pyjamas]], [[http://www.tornadoweb.org/ | Tornado]], etc.

 * [[http://routes.groovie.org/ | URL routes]].

== Build / Tests ==

 * Notebook testing infrastructure: [[http://seleniumhq.org/ | Selenium]], [[http://testled.com/ | Testled]], [[http://code.google.com/p/webdriver/ | Webdriver]], [[http://pylonshq.com/docs/en/0.9.7/testing/ | WebTest]], [[http://www.getwindmill.com/ | Windmill]].

 * Binary distibutions that don't use SSE3, etc.

 * Windows port.

 * Windows build farm.

 * !VirtualBox Sage.

 * andLinux Sage.
 * Notebook:
   * [[http://groups.google.com/group/sage-devel/browse_thread/thread/e46f0a59d5cf70dc | Tags]].
   * Suppress browser warning for Konquereor.
   * Manage logins:
     * View recent account activity.
     * Invalidate other sessions' cookies.
     * Unique cookies for logins from different browsers / computers.
   * Full-text search: SQLalchemy, [[http://whoosh.ca/ | Whoosh]] ([[http://groups.google.com/group/codenode-devel/browse_thread/thread/d3ffefa3b09937b6/98bdf00f65441934?#98bdf00f65441934 | codenode-devel]]).

 * Fix !MoinMoin.

 * External:
   * Embed live cells / worksheets - [[http://trac.sagemath.org/sage_trac/ticket/6855 | #6855]].
   * [[http://wave.google.com/ | Wave]] [[http://code.google.com/apis/wave/guide.html | gadgets and robots]].
   * Live published worksheets.

Sage Tasks Outline

This is a general outline of targeted areas, including specific tasks, for improving the usability of Sage. The sources include the Sage Usability Improvements List and sagenb/todo.txt, but the scope here is wider. Feel free to make changes.

General

Build / Test

  • Tests:
    • Increase coverage.
    • Meaningful statistics / benchmarks, e.g., %timeit for individual doctests.
  • Build logs:
    • Separate file for each spkg for parallel builds.
    • Group by install, upgrade, or spkg install event.
    • Remotely aggregate and display refreshed statistics.
  • Porting:
    • VirtualBox Sage.

    • andLinux Sage.
    • Windows.
    • Windows build farm.
    • Sagelite for Windows.
    • Binary distributions that don't use SSE3, etc.
  • Automated notebook benchmarking and testing:

Development

Documentation

Notebook: Admin

  • Authentication - abstract interface:
    • LDAP.
    • UNIX accounts.
  • Dashboard for server settings / status:
    • Statistics: all / logged-in users, load, processes, worksheets [per user], etc.
    • Backups: all critical data, selective.
    • Configure thresholds, alerts, snapshot behavior.
    • Manage user accounts, groups, privileges.
    • Restricted version for users.
    • Install new / updated modules and spkgs.
  • Registration:
    • reCAPTCHA.

    • Challenge questions.
    • Admin approval.
    • Revokable custom tokens.
  • Users:
    • Groups.
    • Privileges.
    • Email address as username.
    • Use secure tokens to reset passwords.

Notebook: Introspection

  • Use Sphinx for sagenb.org docstrings.
  • Matrix B mentioned twice in B.solve_left?: B.solve_left(B, ...).

  • Better error detection, e.g., for Maxima command completion.
  • Completions link for mobiles.
  • Live introspection cells.
  • Introspection cross-links.
  • Improve introspection / completion interface: floating div, RHS div.
  • New documentation browser. Send worksheet docstrings to this browser.

Notebook: "Modes"

  • %timeit.
  • %cython mode, full and outside sage.

  • Slideshow mode.
  • interacts:
    • Auto updates.
    • Live published.
    • Control over widget layout.
    • Examples to include with Sage.
    • Free-form workspace with pipelines.

    • Display nicely formatted docstrings (or captions).
    • Simplify extension, via HTML templates or an API.
    • More widgets: vertical sliders, spreadsheets, interactive plots, etc.
  • Graph editor - #1321.

  • LaTeX doc mode.

  • reST:
  • Preparse in the browser: implicit multiplication, less work for server.
  • Chat - qwebirc, Web IRC.

    • Rich content, e.g., equations.
    • Sage compute bot.
    • Add to help page.
  • Debugging:
    • Prettier tracebacks for %modes.
    • Python debugger a la Pylons.

  • Vector graphics - #6973.

    • Draw - SVG-edit.

    • Annotate output from matplotlib's SVG backend.
  • Code editor a la BeSpin

    • Command-line history.
    • Syntax highlighting.
    • Context-sensitive completion.
  • Math input GUI plug-ins.

Notebook: Under the hood

  • Implement base_url.

  • Move is_valid_username, etc., out of twist.py.

  • notebook():

    • Allow default_mode='python', i.e., not 'sage'.

    • Allow a different sage command on a remote machine.
  • Smartly manage Jmol applet count, or memory use.
  • Use Python's logging facility.

  • Component architecture.

  • Cythonize pexpect.
  • Processes:
    • Balance server pool loads.
    • Run the worksheet process as another user.
    • Make ctrl-c quit the server's reference implementation.
    • Upload worksheets without blocking the server.
    • Remote worksheet processes.
    • Use fork, named pipes, and/or memory-mapped files for worksheet processes.

  • Client-server and server-server interaction:
    • TinyMCE problem.
    • Load JS / CSS components on demand.
    • Load browser-specific code only in the specific browser.
    • Better handling of concurrent worksheet use.
    • Server-to-server for backups, possibly with authentication.
  • Decouple secure transport from sage/dsage. Use OpenSSL.
  • Storage:
    • Implement infinite undo with revision logs, not snapshots.
    • Track user data / worksheets with Mercurial.
    • Deprecate the [unused] objects directory.

    • Access a user's data directory securely, e.g., via an API.
    • Use the filesystem or a database, not an sobj:

      • Worksheet list.
      • Notebook configuration.
  • Migrate to server / framework: Django, Pylons, Pyjamas, Tornado, etc.:

Notebook: User Interaction

  • Help:
    • Reorganize help page.
      • Search for commands
      • linking to similar and related commands
    • Easy set up of shipped jsMath fonts.
    • Automatic tips for keyboard shortcuts, editing features, etc. Opt out.
    • Friendly bug buddy that emails reports, bundles session history, etc.
  • Customization:
    • Typeset output default setting.
    • Implicit multiplication default setting.
    • Custom window close callback.
    • Function to update worksheet titlebar.
    • Easy backups of own work, with or w/o data, output, snapshots, etc.
    • Download zip files of multiple worksheets, even if accounts=True.

  • Keys
    • Restart worksheet shortcut.
    • Splash-screen with common keys.
    • Insert new cell after current cell shortcut.
    • Custom shortcuts, e.g., ctrl-bksp to delete previous word, not join cells - js-hotkeys.

  • Cells:
    • Auto-updating cells, e.g., interacts.
    • Prevent text movement by a pixel on cell frame focus.
    • Always show links for long output cells, regardless of length or interruption.
    • Easily identify idle, queued, and running cells.
    • Select, delete, (un)group, move multiple cells.

    • Leading space input bug - #6729.

  • 2D / 3D plots:
  • Worksheets:
    • Export screen with checkboxes for images, data, etc.
    • Use notebook as IDE for running sage instance.
    • Always visible menubar at top of worksheet.
    • Make "Browse your computer" upload fields editable.
    • Fix "undefined" bug on first save in Safari 4.0.3.
    • Don't list "Sage Users" when sharing.
    • "Save as" option, e.g., for downloading, that prompts for a name.
    • Toolbar with buttons to interrupt running code, save, attach files, etc.
    • Parens matching - #3646.

    • TinyMCE synchronization problem.

  • Published worksheets:
    • Delete them.
    • RSS feeds.
    • Live cells, e.g, interacts.
    • Only to selected users or groups.
  • Notebook:
    • Tags.

    • Suppress browser warning for Konquereor.
    • Manage logins:
      • View recent account activity.
      • Invalidate other sessions' cookies.
      • Unique cookies for logins from different browsers / computers.
    • Full-text search: SQLalchemy, Whoosh (codenode-devel).

  • Fix MoinMoin.

  • External: