Differences between revisions 13 and 19 (spanning 6 versions)
⇤ ← Revision 13 as of 2009-09-30 18:51:09 →
Size: 11509
Editor: qed777
Comment: mechanize + Add TODO: Add todo.txt.
← Revision 19 as of 2009-10-01 13:09:06 → ⇥
Size: 12329
Editor: qed777
Comment: Doc mod.
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= Sage Usability Tasks =

Source: [[SageUsability | Sage Usability Improvements List]]. TODO: Assign difficulties to individual tasks, if it's appropriate, or just order or group by difficulty.		 = Sage Tasks Outline =

This is a general outline of targeted areas, including specific tasks, for improving the usability of [[http://www.sagemath.org/ | Sage]]. The sources include the [[SageUsability | Sage Usability Improvements List]] and [[http://sage.math.washington.edu:8100/file/tip/sagenb/todo.txt | sagenb/todo.txt]], but the scope here is wider. Feel free to make changes.
Line 15: Line 15:

############################## BUILD / TEST ##############################
== Build / Test ==

 * Tests:
   * Increase coverage.
   * Meaningful statistics / benchmarks, e.g., %timeit for individual doctests.

 * Build logs:
   * Separate file for each spkg for parallel builds.
   * Group by install, upgrade, or spkg install event.
   * Remotely aggregate and display refreshed statistics.

 * Porting:
   * !VirtualBox Sage.
   * andLinux Sage.
   * Windows.
   * Windows build farm.
   * Sagelite for Windows.
   * Binary distributions that don't use SSE3, etc.

 * Automated notebook benchmarking and [stress] testing: [[http://wwwsearch.sourceforge.net/mechanize/ | mechanize]], [[http://seleniumhq.org/ | Selenium]], [[http://testled.com/ | Testled]], [[http://code.google.com/p/webdriver/ | Webdriver]], [[http://pylonshq.com/docs/en/0.9.7/testing/ | WebTest]], [[http://www.getwindmill.com/ | Windmill]].


############################## DEVELOPMENT ##############################
== Development ==

 * Auto-reload / auto-build changed files - [[http://www.sagemath.org/doc/reference/sage/misc/attach.html | attach]], [[http://pythonpaste.org/script/ | paster serve --reload]], !CherryPy's [[http://www.cherrypy.org/wiki/AutoReload | AutoReload]].

 * [[http://groups.google.com/group/sage-devel/msg/fb95603488afd9de | More convenient development workflow]]:
   * Code review tools - [[http://code.google.com/p/rietveld/ | Rietveld]].
   * Automated, continuous builds and testing for new patches.

 * [[http://www.doughellmann.com/articles/CompletelyDifferent-2008-03-linters/index.html | Static Python code analysis]].


############################## DOCUMENTATION ##############################
Line 22: Line 59:

== Development ==

 * More convenient development workflow
   * Current workflow is:
     1. Change file
     2. Restart Sage Notebook
     3. Wash, rinse, repeat
   * Possible improvements:
     * Autoreload and autobuild mechanism (dev mode)

{{{
On Thu, Sep 24, 2009 at 11:07 AM, Robert Bradshaw wrote:

Here's my ideal system (which rietveld may or may not be a part of,
but certainly it'd be better to not start something from scratch).

0) I hack on my own code, committing as I want, the normal mercurial
way.
(1) I run a sage command that takes all my changes and uploads them
to trac (or rietveld, or Review Board, or ...). It may or may not
flatten them. The ticket description should be no more than the
commit description (i.e. our commit descriptions should be a lot more
verbose.)
(2) This patch automatically gets applied to the current alpha, any
merge problems, build problems (including documentation), or test
failures get reported and bounced back. Linting could happen at this
point too (everything from no tabs to coverage to "Sage:" in docstrings)
(3) Upon a successful build (or even not) this build gets saved and
can be accessed and played with (via the notebook or by ssh-ing into
a virtual machine). The code can be browsed and commented on.
(4) Reviewer makes comments, as a whole or line-by-line. They can use
(3), browse the code, and have a sage command like sage -merge that
will quickly update their local copy to the state of the ticket, and
allow them to push as well.
(5) I can quickly address those comments via a sage -b into that
branch (or otherwise using queues/etc. to get back to my ticket
state), make my changes, and re-push to trac. Building and tests
happen after every push. This point here is not to be underestimated--
I think it will greatly improve the quality of the code to lower the
overhead of the feedback loop.
(6) When the reviewer is happy, he marks it as positive review.
(7) The release manager(s) approve the patch. At this point they know
it builds, passes tests, and has at least one positive review. It
goes into a queue.
(8) Patches in the queue get applied to the current alpha. This
implies a moving alpha, which one knows builds and passes all tests
on at least one system. It should be easy to upgrade to that alpha.
(9) People can (easily) donate computer time to build and test
alphas, which will greatly increase hardware coverage. Bisection can
be used to locate the offending ticket in case of failure.
}}}		    * [[SageNotebook | Notebook server technical overview]].
Line 78: Line 65:
 * Authentication:  * Authentication - abstract interface:
Line 83: Line 70:
   * View logged-in users, load, processes, etc.    * Statistics: all / logged-in users, load, processes, worksheets [per user], etc.
Line 88: Line 75:
   * Install new / updated modules and spkgs.
Line 127: Line 115:
 * %cython mode, full and outside `sage`.

 * Slideshow mode.

 * interacts:
   * Auto updates.
   * Live published.
   * Control over widget layout.
   * Examples to include with Sage.
   * [[http://groups.google.com/group/sage-devel/browse_thread/thread/8aa6a3a84365f06d/42c7edf1c23ca169?#42c7edf1c23ca169 | Free-form workspace with pipelines]].
   * Display nicely formatted docstrings (or captions).
   * Simplify extension, via HTML templates or an API.
   * More widgets: vertical sliders, spreadsheets, interactive plots, etc.
Line 142: Line 144:
 * Python debugger a la [[https://developer.mozilla.org/En/Pylons | Pylons]].  * Debugging:
   * Prettier tracebacks for %modes.
   * Python debugger a la [[https://developer.mozilla.org/En/Pylons | Pylons]].
Line 154: Line 158:


######################### NOTEBOOK: UNDER THE HOOD #########################
== Notebook: Under the hood ==

 * Implement `base_url`.

 * Move `is_valid_username`, etc., out of `twist.py`.

 * `notebook()`:
   * Allow `default_mode='python'`, i.e., not 'sage'.
   * Allow a different sage command on a remote machine.

 * Smartly manage Jmol applet count, or memory use.

 * Use Python's [[http://docs.python.org/library/logging.html | logging facility]].

 * [[http://bitbucket.org/brickenstein/rumcomponent/src/tip/rumcomponent/ | Component architecture]].

 * Cythonize pexpect.

 * Processes:
   * Balance server pool loads.
   * Run the worksheet process as another user.
   * Make ctrl-c quit the server's reference implementation.
   * Upload worksheets without blocking the server.
   * Remote worksheet processes.
   * Use fork, named pipes, and/or [[http://docs.python.org/library/mmap.html | memory-mapped]] files for worksheet processes.

 * Client-server and server-server interaction:
   * TinyMCE problem.
   * Load JS / CSS components on demand.
   * Load browser-specific code only in the specific browser.
   * Better handling of concurrent worksheet use.
   * Server-to-server for backups, possibly with authentication.

 * Decouple secure transport from sage/dsage. Use OpenSSL.

 * Storage:
   * Implement infinite undo with revision logs, not snapshots.
   * Track user data / worksheets with Mercurial.
   * Deprecate the [unused] `objects` directory.
   * Access a user's data directory securely, e.g., via an API.
   * Use the filesystem or a database, '''not''' an sobj:
     * Worksheet list.
     * Notebook configuration.

 * Migrate to server / framework: [[http://docs.djangoproject.com/en/dev/ | Django]], [[http://pylonshq.com/ | Pylons]], [[http://pyjs.org/ | Pyjamas]], [[http://www.tornadoweb.org/ | Tornado]], etc.:
   * Switch from twisted.web2 to [[http://jcalderone.livejournal.com/tag/sixty+seconds | twisted.web]].
   * [[http://routes.groovie.org/ | URL routes]].
Line 195: Line 249:
   * Slideshow mode.    * Export screen with checkboxes for images, data, etc.
Line 198: Line 252:
   * Make "Browse your computer" upload fields editable.
   * Fix "undefined" bug on first save in Safari 4.0.3.
   * Don't list "Sage Users" when sharing.
   * "Save as" option, e.g., for downloading, that prompts for a name.
Line 211: Line 269:
   * Unique cookies for logins from different browsers / computers.    * Manage logins:
     * View recent account activity.
     * Invalidate other sessions' cookies.
   * Unique cookies for logins from different browsers / computers.
Line 217: Line 278:


######################### NOTEBOOK: UNDER THE HOOD #########################
== Notebook: Under the hood ==

 * Better handling of concurrent worksheet use.

 * Smartly manage Jmol applet count, or memory use.

 * Use Python's [[http://docs.python.org/library/logging.html | logging facility]].

 * [[http://bitbucket.org/brickenstein/rumcomponent/src/tip/rumcomponent/ | Component architecture]].

 * Cythonize pexpect.

 * Client-server synchronization:
   * TinyMCE problem.
   * Server-to-server for backups, possibly with authentication.

 * Track worksheets and user data with Mercurial.

 * Balance server pool loads.

 * Load JS / CSS components on demand.

 * Migrate to server / framework: [[http://docs.djangoproject.com/en/dev/ | Django]], [[http://pylonshq.com/ | Pylons]], [[http://pyjs.org/ | Pyjamas]], [[http://www.tornadoweb.org/ | Tornado]], etc.:
  * [[http://routes.groovie.org/ | URL routes]].

######################## NOTEBOOK: DEVELOPMENT ###########################

== Notebook: Development ==

 * Auto-reloading (similar to `paster serve --reload` or !CherryPy's [[http://www.cherrypy.org/wiki/AutoReload | AutoReload]]).


############################## BUILD / TEST ##############################
== Build / Test ==

 * !VirtualBox Sage.

 * andLinux Sage.

 * Binary distibutions that don't use SSE3, etc.

 * Sagelite for Windows.

 * Windows build farm.

 * Windows port.

 * Notebook testing infrastructure: [[http://wwwsearch.sourceforge.net/mechanize/ | mechanize]], [[http://seleniumhq.org/ | Selenium]], [[http://testled.com/ | Testled]], [[http://code.google.com/p/webdriver/ | Webdriver]], [[http://pylonshq.com/docs/en/0.9.7/testing/ | WebTest]], [[http://www.getwindmill.com/ | Windmill]].

Sage Tasks Outline

This is a general outline of targeted areas, including specific tasks, for improving the usability of Sage. The sources include the Sage Usability Improvements List and sagenb/todo.txt, but the scope here is wider. Feel free to make changes.

1. General

2. Build / Test

  • Tests:
    • Increase coverage.
    • Meaningful statistics / benchmarks, e.g., %timeit for individual doctests.
  • Build logs:
    • Separate file for each spkg for parallel builds.
    • Group by install, upgrade, or spkg install event.
    • Remotely aggregate and display refreshed statistics.
  • Porting:

    • VirtualBox Sage.

    • andLinux Sage.
    • Windows.
    • Windows build farm.
    • Sagelite for Windows.
    • Binary distributions that don't use SSE3, etc.

  • Automated notebook benchmarking and [stress] testing: mechanize, Selenium, Testled, Webdriver, WebTest, Windmill.

3. Development

4. Documentation

5. Notebook: Admin

  • Authentication - abstract interface:
    • LDAP.
    • UNIX accounts.
  • Dashboard for server settings / status:
    • Statistics: all / logged-in users, load, processes, worksheets [per user], etc.
    • Backups: all critical data, selective.
    • Configure thresholds, alerts, snapshot behavior.
    • Manage user accounts, groups, privileges.
    • Restricted version for users.
    • Install new / updated modules and spkgs.
  • Registration:

    • reCAPTCHA.

    • Challenge questions.
    • Admin approval.
    • Revokable custom tokens.
  • Users:
    • Groups.
    • Privileges.
    • Email address as username.
    • Use secure tokens to reset passwords.

6. Notebook: Introspection

  • Use Sphinx for sagenb.org docstrings.

  • Matrix B mentioned twice in B.solve_left?: B.solve_left(B, ...).

  • Better error detection, e.g., for Maxima command completion.
  • Completions link for mobiles.
  • Live introspection cells.
  • Introspection cross-links.
  • Improve introspection / completion interface: floating div, RHS div.

  • New documentation browser. Send worksheet docstrings to this browser.

7. Notebook: "Modes"

  • %timeit.

  • %cython mode, full and outside sage.

  • Slideshow mode.
  • interacts:
    • Auto updates.
    • Live published.
    • Control over widget layout.
    • Examples to include with Sage.

    • Free-form workspace with pipelines.

    • Display nicely formatted docstrings (or captions).
    • Simplify extension, via HTML templates or an API.
    • More widgets: vertical sliders, spreadsheets, interactive plots, etc.

  • Graph editor - #1321.

  • LaTeX doc mode.

  • reST:
  • Preparse in the browser: implicit multiplication, less work for server.

  • Chat - qwebirc, Web IRC.

    • Rich content, e.g., equations.
    • Sage compute bot.
    • Add to help page.
  • Debugging:
    • Prettier tracebacks for %modes.

    • Python debugger a la Pylons.

  • Vector graphics - #6973.

    • Draw - SVG-edit.

    • Annotate output from matplotlib's SVG backend.

  • Code editor a la BeSpin

    • Command-line history.
    • Syntax highlighting.
    • Context-sensitive completion.

  • Math input GUI plug-ins.

8. Notebook: Under the hood

  • Implement base_url.

  • Move is_valid_username, etc., out of twist.py.

  • notebook():

    • Allow default_mode='python', i.e., not 'sage'.

    • Allow a different sage command on a remote machine.
  • Smartly manage Jmol applet count, or memory use.

  • Use Python's logging facility.

  • Component architecture.

  • Cythonize pexpect.
  • Processes:
    • Balance server pool loads.
    • Run the worksheet process as another user.
    • Make ctrl-c quit the server's reference implementation.
    • Upload worksheets without blocking the server.
    • Remote worksheet processes.

    • Use fork, named pipes, and/or memory-mapped files for worksheet processes.

  • Client-server and server-server interaction:
    • TinyMCE problem.
    • Load JS / CSS components on demand.
    • Load browser-specific code only in the specific browser.
    • Better handling of concurrent worksheet use.
    • Server-to-server for backups, possibly with authentication.
  • Decouple secure transport from sage/dsage. Use OpenSSL.
  • Storage:
    • Implement infinite undo with revision logs, not snapshots.
    • Track user data / worksheets with Mercurial.

    • Deprecate the [unused] objects directory.

    • Access a user's data directory securely, e.g., via an API.

    • Use the filesystem or a database, not an sobj:

      • Worksheet list.
      • Notebook configuration.

  • Migrate to server / framework: Django, Pylons, Pyjamas, Tornado, etc.:

9. Notebook: User Interaction

  • Help:
    • Reorganize help page.
      • Search for commands
      • linking to similar and related commands
    • Easy set up of shipped jsMath fonts.
    • Automatic tips for keyboard shortcuts, editing features, etc. Opt out.
    • Friendly bug buddy that emails reports, bundles session history, etc.
  • Customization:
    • Typeset output default setting.
    • Implicit multiplication default setting.
    • Custom window close callback.
    • Function to update worksheet titlebar.
    • Easy backups of own work, with or w/o data, output, snapshots, etc.

    • Download zip files of multiple worksheets, even if accounts=True.

  • Keys
    • Restart worksheet shortcut.
    • Splash-screen with common keys.
    • Insert new cell after current cell shortcut.

    • Custom shortcuts, e.g., ctrl-bksp to delete previous word, not join cells - js-hotkeys.

  • Cells:
    • Auto-updating cells, e.g., interacts.
    • Prevent text movement by a pixel on cell frame focus.
    • Always show links for long output cells, regardless of length or interruption.
    • Easily identify idle, queued, and running cells.

    • Select, delete, (un)group, move multiple cells.

    • Leading space input bug - #6729.

  • 2D / 3D plots:
  • Worksheets:
    • Export screen with checkboxes for images, data, etc.
    • Use notebook as IDE for running sage instance.
    • Always visible menubar at top of worksheet.
    • Make "Browse your computer" upload fields editable.
    • Fix "undefined" bug on first save in Safari 4.0.3.
    • Don't list "Sage Users" when sharing.
    • "Save as" option, e.g., for downloading, that prompts for a name.
    • Toolbar with buttons to interrupt running code, save, attach files, etc.

    • Parens matching - #3646.

    • TinyMCE synchronization problem.

  • Published worksheets:
    • Delete them.
    • RSS feeds.
    • Live cells, e.g, interacts.
    • Only to selected users or groups.
  • Notebook:

    • Tags.

    • Suppress browser warning for Konquereor.
    • Manage logins:
      • View recent account activity.
      • Invalidate other sessions' cookies.
      • Unique cookies for logins from different browsers / computers.

    • Full-text search: SQLalchemy, Whoosh (codenode-devel).

  • Fix MoinMoin.

  • Externally embed live cells / worksheets. Live published workseets. #6855.