Sage Usability Improvements

This is an official wish list for development mainly on the Sage notebook in the near future. Please feel free to add or clarify ideas, link to trac tickets, make comments, reorganize items, point out useful links, etc.

Note: There's an "outline" version here.

Documentation

• LaTeX doc mode.

• Use $ instead of back ticks for math in reST - #6892.

• Use Sphinx for sagenb.org docstrings.
• reST mode(s)
  1. replace tinymce by reST editor?

     • You might be able to make TinyMCE *be* a reST editor. See this example of making TinyMCE into a BBCode editor.

     • reST's implicit closing tags may make this difficult.
  2. This is better: %rest in cells, and get reST output. Note that it would be good to autohide and have doubleclick be show. Same with %html. Use sagenb-0.2's sphinxify.

• Document pexpect interface for users.

• Live introspection cells.
• Introspection cross links.

• Tear-out docstrings - #6001.

• Automatic tips: if the user uses their mouse to perform a function, pop up a little tip (similar to gmail's "still working...") that explains how to use that functionality from the keyboard. For editor-like features, pop up little tips now and then. Add a way to disable these infernally obnoxious tips from showing up (but put 'em in by default).
• Restart reports: if the user tries to interrupt the worksheet and fails and resorts to restarting, that's evidence of a bug. We should offer to bundle the session history as a tar file to send in a bug report. Or even better, we could have it automatically email the bug report to a special mailbox if the user agrees.

• Detect errors better in introspection. See maxima completion puke.png

Notebook

• stress/load testing:

  • http://funkload.nuxeo.org

  • http://pypi.python.org/pypi?:action=display&name=zope.testbrowser

Admin

• architecture: http://bitbucket.org/brickenstein/rumcomponent/src/tip/rumcomponent/

• Rewrite pexpect in cython?
• A good monitoring script that does sophisticated monitoring of a running sage notebook server, including having a nice web-based interface/view. It should also have configurable thresholds which when hit cause emails to be sent, etc. It could also give info about currently logged in users, usage, processes, etc.

• Streamlined deployment - account management, fewer bugs, etc. - Django #6897.

• Captcha for notebook sign-up - reCAPTCHA. This is *only* needed at registration. - Django #6897.

• LDAP authentication. (tdumont) -- do this by refactoring auth code so that user/password is 100% separate from anything else in the notebook. Then LDAP or anything else is completely trivial. - Django #6897.

• Guaranteed worksheet-server synchronization:
  • at least have a timestamp on the server
  • TinyMCE never gets sent back somehow leading to corrupt worksheets.
  • maybe use the javascript ping function to push changes out.
  • A bit wider(?) in scope: Hashing, with immediate feedback. The notebook keeps a message-plus-metadata digest and sets an indicator (one per cell, at least for diagnostics) to red upon sending the message. The server hashes on receipt and sends just the hash back to the notebook. If the hashes agree, notebook sets indicator to green; otherwise, it sends again. The indicator warns the user about loss of sync. The user may use a purely client-side "save as" function as a precaution. An alternative: fail-over servers.

• Reduce data loss possibilites. (dataloss.txt): easy backup of all critical data in a notebook server. Get via web interface as admin. Archives.

• Recover passwords. (Byungchul Cha)
• Email address as username.

• Add users as admins. (fprimex) - Django #6897.

• User groups. - Django #6897.

• Security -- see http://www.gingerlime.com/20090901_securing_sage_notebook.pdf

• Functions to delete worksheets/USER/*/BLAH where BLAH is cells, code, data, or snapshots? Does this affect the size of .sws files? Admin pane showing usage for various things, e.g., snapshots. Allow easy trimming of old snapshots, configuring how their kepts, etc.

• Balancing server_pool:

I managed to do a little of this by exporting the directory that the
notebook is running in via NFS to the other machines and maintaining
the same permissions and file structure for everything else.  However,
this raised another issue in that not all my machines are of
comparable power.  How complicated would it be to add something that
allows you to balance how many processes go to each server?  I'm
thinking something along the lines of distcc's host list.

• Account approval: Possible solution -- just have an interface like Google groups where new users can be "moderated by default".

> We have a small server and would like provide SAGE (in form of the
> notebook) for the students. However, it maybe a bit
> "risky" (especially, we have no experts to manage it) to open it up to
> everyone. So we would like to have some kind of "account approval"
> procedure. Can that be done, if so how?

• Load JS various libraries only on demand (i.e., lazily). jsMath already does this with its extensions/plug-ins. Ajile?

Users

• make it so arbitrarily many 3d plots at once work:

Jonathan <[email protected]>
I have my own answer to #3.  I think that using document.write to
replace the applet with a picture when not in use should work.  I
haven't investigated yet to see if it really saves memory.  My
suggestion is to use the list of applets JmolApplet0,
JmolApplet1, ...JmolAppletn to keep only three to four applets active.
When the fifth is opened (#4) then #0 should be replaced with a
picture that has an onclick action that would reload the applet and
unload another.  Doesn't sound like too bad a bit of javascript.

Since I don't yet understand exactly how Jmol is used inside the
notebook, I don't think I can actually do the coding now.  If I get a
chance later I will try it on a plain vanilla test page and pass along
the javascript that works for that.

• Graph editor http://trac.sagemath.org/sage_trac/ticket/1321

• Create a big red "stop" button to interrupt running code.
• Improve "Return":

It's easy to understand what do "return"

And now I only use Alt-return and Backspace. I feel it's also possible
to use Ctrl-return and Ctrl-backspace.

A splash-screen at the very beginning of sage session with only the most
useful key may be nice.

return = creates a newline inside a cell
alt-return = evaluates a cell and creates a new one
backspace = erases an empty-cell
ctrl-backspace = joins this cell to the previous one

• Clearer indication of what is running and what isn't: "I'm in this case when I look at students works. I don't understand what is running, what is waiting, where is the computation. "A backward color of the current cell may also indicate where sage is running."
• Javascript animations of plots (etc.), both 2d and 3d.
• When notebook has a long output, make sure link to long output is displayed at all times, so if user interprets run, then they can still get at the output.

• Introspection in notebook: B a matrix, B.solve_left? has at the top B.solve_left(B, ...). Notice the B twice.

• user configuration: make it so close_callback can just do window.close() (or whatever). (requested by Soroosh)

• Interactively zoom, pan, and annotate plots. SVG? Png's? FLOT? Canvas + matplotlib? Open Flash Chart: OFC 1.x, OFC v2.x. Not sure about zooming with OFC.

• way in the notebook to mouse over a plot and see coordinates. for non-png canvas5 see http://sourceforge.net/mailarchive/forum.php?thread_name=6e8d907b0909012053s19e7c5fl29cdcd0e1fab1d50%40mail.gmail.com&forum_name=matplotlib-devel; but I want to do both png and non-png.

See this for some excellent ideas from Jason Grout: http://groups.google.com/group/sage-support/browse_thread/thread/c29abdaaa4f0ee13

• jzmer from irc says the sage notebook works fine in konqueror -- the only problem is the annoying message saying it might not.
• The cookie system for staying logged into sage notebooks is seriously foobar'd. If you login to multiple sage notebooks at the same time from the same browser, say on different computers, then the cookies somehow clash and they confuse each other. Fix this completely.

• Add docs about port forwarding to notebook? -- see http://groups.google.com/group/sage-devel/browse_thread/thread/6d94c8655e41ba24#

• Ability to easily highlight then delete any range of cells. Also, to move and group of cells around. The edit mode can sort of do this, but is way too old fashioned, and should basically be not needed. See this jquery demo: http://boagworld.com/demos/sitemap/

• Insert new cell *after* current cell. This has been a missing feature forever!

• Sage Forge -- idea posted on Sage Devel by Matthew Turk

• "It could be really useful to have Sage generate RSS feeds when sharing or publishing worksheets. There is a firefox extension for reading RSS feeds called Sage which could also be developed further to work with Sage software." -- Serge A. Salamanka (on sage-devel)

• See http://routes.groovie.org/ to handle URLS (which is basically all twist.py is doing a lot of). - Django #6897.

• Investigate the new webserver framework that Facebook just released (it's their alternative to twisted!): http://developers.slashdot.org/story/09/09/11/0256233/Facebook-Releases-Open-Source-Web-Server?art_pos=19

• Sage could (and should!) ship copies of the jsmath fonts, and explain clearly what to do to install them whenever they aren't already installed. The current system, which involves visiting the jsmath website in response to subtle hints is not very good.

• Fix TinyMCE issues, as mentioned here: http://groups.google.com/group/sage-devel/browse_frm/thread/4cd1cc188470c723/c2477e1bdba68028?q=tinymce+bug+group:sage-devel#c2477e1bdba68028 and http://groups.google.com/group/sage-devel/msg/e2eb3f848a21faa3

• Move preparsing to the worksheet (not server). This (1) solves implicit multiplication problem, and (2) fits well with the "notebook server computers the least possible" philosophy.
• Improve handling of multiple users using the same worksheet.
• Trivial to set implicit multiplication option for the notebook. On a per worksheet basis with a user default. Same for typeset option.
• Reorganize the notebook help page.
• Automatically have worksheets save their state and restore when started later. Something like "vmware resume saved state", "pause".
• Update the titlebar from the notebook: just a little javascript wrapper function to do this.

   set_title_status("almost done!")

• Auto-updating interacts and cells (like notebook hack but official).

• Optimize fulltext search (could be moving everything to database). Use SQLalchemy? Whoosh full-text search (codenode-devel).

• Keith Lang's Top 10 UI Myths.

• Chat system for students/users - Use qwebirc. Example: Web IRC.

• Implement %timeit block in notebook.

• Worksheet tags. (Brian Granger)

• Embed cells, including interacts, in other web pages. Enable live published worksheets. #6855

• Prevent text movement by a pixel on cell frame focus. (Ondrej)
• Completions link for iphone.
• Improve the introspection / completion interface. (boothby: contrary to my previous claims, it may be possible to display a floating div below the cursor -- otherwise, I think that moving the div to the right-hand side of the screen is the right thing to do).

• Notebook input bug - #6729.

• Python debugger in the notebook: basically pylons implements something like this which could be inspiring. At least try it to see their design! Check this out: https://developer.mozilla.org/En/Pylons

• Make it possible to delete published worksheets.
• Slideshow mode.

• Powerful editor with command-line history, syntax highlighting, context-sensitive completions, etc. - BeSpin.

• Syntax highlighting for published worksheets - pygmentize.

• Fix MoinMoin.

• Make the header a true header, so that (for instance) the Help button and worksheet title are always visible. See this thread

• Update some output cells after the new output arrives. For example, smoothly update Jmol via its JS interface.

• Publishing notebook pages to selected users or groups. (This *is* share, but with groups added.) -- No, publish means "read-only" too, which is part of the feature request.

• GUI plugins for math input.

• Vector graphics editing - SVG-edit, RaphaelJS.

• Account tokens:

A preliminary patch for this is available at
http://www.math.leidenuniv.nl/~wpalenst/sage/account_tokens.patch . It is a
patch against 4.1.1 with tickets 4552
('trac_4552-notebook_account_email.patch'), 6843, 6856 applied.

(TODO: add doctests, implement token-generation page to the interface added by
ticket 4135, allow enabling/disabling the token feature from that page too.)

• The 'forgot password' feature (enabled by enabling the 'email' setting of the notebook) currently allows anyone to reset the password of anyone they know the email address of, since it currently directly resets the password and mails the new password to the email address associated with the account. It would probably be better to instead generate a second password for the account, that would only become permanent once it has been used once. Or, alternatively, to email a link with a secure token to a password-change-page.
• Make it possible to use the notebook as an IDE for the instance of sage that's running.
• Make it possible to download zip files of multiple worksheets, even if accounts=True
• Make it possible to download just text or other options - see Trac # 4404
• Make it possible for Sage servers to sync worksheets between each other (i.e., let me sync all of my worksheets on a given server, with a given username and password). This helps tremendously with keeping backup copies. Should each user directory maybe be an hg repository with the worksheet.txt files and the files in DATA checked in? That would maybe make pushing/pulling changes easier.

• Parens matching: Has there been any progress on this? http://www.mail-archive.com/[email protected]/msg03975.html One of the often heard comments from lecturers is that they are wasting time on basic syntax, when students mismatch parenthesis. In the simplest instance, when a closing ] or ) is typed it helps to highlight the correct (nested or not) matching left one. See http://trac.sagemath.org/sage_trac/ticket/3646

• keyboard shortcut to restart worksheet

• Customizable keyboard shortcuts. For example, <ctrl>+<bksp> is (delete previous word) by default in Firefox text-boxes in Ubuntu and Windows (as well as Thunderbird, OpenOffice, Word, etc.). I use this key combination reflexively whenever I see a typo, and am constantly joining together cells that I wish I hadn't.

• A menubar that stays visible at the top of the screen (maybe in an HTML frame?) My worksheets get quite long, and it is annoying to insert something in the middle, scoll up to the top to select 'save' or 'interrupt' or 'restart' or whatever, and then have to find my place in the middle of the sheet again.

Build / Test

• Notebook testing infrastructure - automated, cross-browser and platform, focused on functionality, consistency, concurrency, and scalability - Selenium, Testled, WebDriver. For example, execute 10 sessions of virtual calculus students in a loop. WebTest, which is included in Pylons, is what mhansen recommends.

• Make it so Sage binaries don't use SSE3, etc. -- make them FAT!
• Windows build farm.
• Sagelite - separate the current notebook from Sage. Windows port.
• Windows port.

• VirtualBox Sage.

• andLinux Sage.
• Improve documentation/testing and refactor code.