The Sage Notebook Server: A Technical Overview

This document gives a technical overview of the new Sage Notebook server, including how it responds to user requests, delegates tasks, etc. See #6983 for instructions on trying out the new server.

Directory structure

The sagenb spkg has the following directory structure:

• sagenb
  • sagenb
    • data
      • Data files used by SageNB, including jmol, jQuery, etc. A notable subdirectory is:
      • sage
        • html
          • Has the Jinja2 templates used by SageNB.
          • notebook
            • Templates used by notebook.py
          • worksheet
            • Templates used by worksheet.py
          • accounts
            • Templates used in account management.
          • etc.
        • js
          • Javascript used by SageNB
        • css
          • CSS generated from SASS. *Please do not edit this.* Instead, edit the SASS source files and regenerate these. These are stored in the HG repo since SASS is Ruby based, and we don't want to unnecessarily ship Ruby with Sage.
    • notebook
      • The main SageNB code. Notable files under this subdirectory:
      • avatars.py
        • An authentication system based on twisted.auth.
      • cell.py
        • Cell class representing Worksheet cells. This handles the data in each cell, and delegates evaluation to worksheet.py
      • challenge.py
        • Antibot challenges for registration, etc, including reCaptcha.
      • conf.py, user_conf.py, server_conf.py
        • Classes that store user can server specific configuration.
      • config.py
        • Used in keyboard configuration for the different browsers.
      • guard.py
        • The root node that checks authentication. See below in "Serving pages."
      • interact.py
        • Used in @interact.
      • notebook.py
        • Class representing a Notebook. This contains migration code from the old notebook, worksheet creation, user management, etc.
      • worksheet.py
        • Class representing a Worksheet. Evaluation is handled here. See "Cell Evaluation" for more.
      • notebook_object.py
        • An object wrapper for run_notebook.py.
      • run_notebook.py
        • Code used to launch the notebook.
      • template.py
        • Jinja2 templating interface.
      • twist.py

        • This is the server engine. Handles page serving. Also contains various global variables, including notebook, which is the global Notebook instance.

    • storage

      • Generic storage system. This is to be obsoleted by #8757 in favor of a SQLAlchemy based storage system.

    • misc
      • Miscellaneous functions, including Sphinx conversion, etc.
    • interfaces
      • Various interfaces to Sage subprocesses.
    • simple
      • A simple Twisted based API to SageNB. This does not work as of 11 May 2010.
    • testing

      • Selenium based tests for SageNB.

  • sass
    • src
      • SASS files are stored here. These are the source files from which the CSS for SageNB is generated.

        SageNB uses Compass and SASS for its CSS, since they are much easier to develop in.

  • [..] # setuptools related stuff

Serving pages

• All access goes through guard.py.

• locateChild checks authentication through avatars.py.

• If the user is authenticated, locateResource is called to get the appropriate page.

• locateResource, in turn, uses the LoginSystem class in avatars.py to get the appropriate resource. Specifically, LoginSystem.requestAvatar().

• LoginSystem calls the appropriate function in twist.py.

• Which would be AdminTopLevel, UserTopLevel, FailedTopLevel, or AnonymousTopLevel, depending on the user type.

• The *TopLevel resources generate child resources using their childFactory methods, which may delegate to userchildFactory.

• Each resource represents a segment of the URL.

• Thus '/' is the TopLevel resource, and '/home/' is mapped to the Worksheets resource, etc.

• The tree is traversed until a leaf is reached, then the leaf's render method is called.

Cell Evaluation

• Evaluation occurs when Worksheet_eval is called.

• Evaluation only happens if the user has permissions to evaluate the cell.

• The cell's evaluate method is called.

• evaluate adds itself to the worksheet's job queue using Worksheet#enqueue, which also calls Worksheet#start_next_comp.

• If the cell is marked for introspection, set input to be evaluated to the part before introspection is requested. Otherwise, set input to the whole cell.

• Get a reference to an instance of Sage for computation using Worksheet#sage. Currently, WorksheetProcess_ExpectImplementation from interfaces/expect.py is used.

• Executes input with that instance.

• check_comp is called periodically to see whether computation is done or whether any output has been printed. Any files in the evaluation temporary directory are copied to the respective cell directory and linked to in the online notebook. If computation is done, perform next computation in queue.

• Updates cell content.