Differences between revisions 6 and 13 (spanning 7 versions)
⇤ ← Revision 6 as of 2011-06-14 02:05:44 →
Size: 1875
Editor: was
Comment:
← Revision 13 as of 2011-06-17 16:35:25 → ⇥
Size: 3639
Editor: was
Comment:
Deletions are marked like this. Additions are marked like this.
Line 4: Line 4:

== Published Interacts ==

  * See if it is possible to get Mitesh Patel's patch to work: http://trac.sagemath.org/sage_trac/ticket/7908
  * Test server (might be up): http://sage.math.washington.edu:8976/pub/
Line 15: Line 20:
  * (mhansen is doing this) Fix the __worksheet object to not load everything. Scary stuff if you look at notebook.py.   * (mhansen is doing this) Fix the {{{__worksheet}}} object to not load everything. Scary stuff if you look at notebook.py.
Line 22: Line 27:
Line 23: Line 29:
  * http://prep.sagenb.org
  * http://tutorial.sagenb.org
Line 24: Line 32:
  * http://nt.sagenb.org   * http://alpha.sagenb.org
Line 27: Line 35:
  * http://tutorial.sagenb.org
  * http://prep.sagenb.org
  * http://flask.sagenb.org		   * http://nt.sagenb.org
  * http://standalone.sagenb.org
  * http://test.sagenb.org
Line 31: Line 39:
  * http://flask.sagenb.org -- not upgraded
Line 43: Line 52:

  * I spent a day writing this script to *manage* a collection of notebook servers: [[attachment:admin]]
Line 53: Line 65:

== William's TODO List ==
{{{
 [x] official sagenb list of optional packages, and install them all.
 [x] make a "stop" script for servers that works.
 [x] fix any remaining links issues, then archive the fixlinks script
 [x] start servers and test
 [x] re-enable automatic backups
 [x] upgrade my sagenb install to include rado's fix for unicode issue, etc.
 [x] search is broken (mhansen is on it)
 [x] The _sage_ directory gets too big. (mhansen is on it)
 [x] script that computes and summarizes stats about all notebook servers:
      - number of users
      - how many users who evaluated code during the last day, week, etc. -- implement by reading last mod time on all history.pickle files.
      - what code was evaluated recently (read from history)
      - size of each notebook server
      - disk usage of each user
      - how many worksheets each user has
      
    (use a nosqlite db?)
    datetime server username last_datetime num_worksheets disk_usage

 [ ] make it so all pickle db's get saved in a non-corruptable way (atomic)
 [ ] push merged combined changes to wstein repo?
 [ ] MATHJAX (or jsmath image fonts ?!)
 [ ] twisted -- number of threads?



 [ ] ban copying results over if too big.


 [ ] somehow migrate flask.sagenb.org ??? or just wait.

}}}

Upgrade, flask, sage 4.7, and ext4 filesystem

People: W. Stein, J. Grout, M. Hansen

Published Interacts

Database

  • If we stick with the filesystem: NEED a very fast filesystem that supports > 4 million inodes and > 32000 directories per directory, i.e., ext4. This is difficult because boxen and sage.math are both Ubuntu 8.10.

  • Also, want to at a minimum have user and worksheet metadata stored in database so startup is faster.
  • Could run the server in a virtual machine, which would support ext4. But are virtual machines fast enough these days? They weren't last year. This would be worth trying.
  • Or stick with the filesystem (see below).

Filesystem Thoughts

  • (mhansen is doing this) Fix the __worksheet object to not load everything. Scary stuff if you look at notebook.py.

  • (mhansen) Write a script to prune the filesystem, e.g., deleting cell directories that are empty.
  • (mhansen?) Make it so user directories are stored in a hierarchical way. E.g., home/_store/

Current Servers

Backup Strategy

  • ?

Untrusted User Code: must all be executed in a virtual machine

  • ?

Monitoring

Virtual Machines

  • I tried install virtualbox 4 on sage.math and failed, due to the Linux source and Linux kernel being out of sync somehow. Probably just needs an upgrade and reboot.
  • I am scared to upgrade virtualbox on boxen.math.
  • The Virtualbox on boxen.math is somewhat unstable and very old.

William's TODO List

 [x] official sagenb list of optional packages, and install them all.
 [x] make a "stop" script for servers that works.
 [x] fix any remaining links issues, then archive the fixlinks script
 [x] start servers and test
 [x] re-enable automatic backups
 [x] upgrade my sagenb install to include rado's fix for unicode issue, etc. 
 [x] search is broken (mhansen is on it)
 [x]  The _sage_ directory gets too big. (mhansen is on it)
 [x] script that computes and summarizes stats about all notebook servers:
      - number of users
      - how many users who evaluated code during the last day, week, etc. -- implement by reading last mod time on all history.pickle files.
      - what code was evaluated recently (read from history)
      - size of each notebook server
      - disk usage of each user
      - how many worksheets each user has
      
    (use a nosqlite db?)
    datetime server username last_datetime num_worksheets disk_usage

 [ ] make it so all pickle db's get saved in a non-corruptable way (atomic)
 [ ] push merged combined changes to wstein repo?
 [ ] MATHJAX (or jsmath image fonts ?!)
 [ ] twisted -- number of threads?



 [ ] ban copying results over if too big.


 [ ] somehow migrate flask.sagenb.org ???  or just wait.

days31/workinggroups/sagenb (last edited 2011-06-17 16:35:25 by was)