=What happens when some code is evaluated= 

== Example: with discussion, simplified ==

Here's an example of the sequence of events when a person evaluates the following input:

      print 2
      sleep(10)
      print 3
      plot(sin)
      print "hello"
      graph_editor()


The cast is as follows:

 * USER -- a human or a program controlling a web browser (selenium) or other user interface
 * CLIENT -- a program, possibly in javascript that displays something
   to the USER
 * SERVER -- a program that handles requests from the CLIENT,
   typically a web server such as flask + mod_wsgi + apache.
 * DATABASE -- stores data
 * DEVICE -- queries the DATABASE for work that needs to be performed,
   does that work, and updates the database in response

 1. The USER types the above into an input object and submits this input.

 2. The CLIENT (e.g., javascript) instantly adds some confirmation that the input is being sent, e.g., a spinning wheel, a green bar, or something.   This CLIENT widget will timeout with an error if no output appears after 15 seconds (say).  

 3. The CLIENT sends a message to the SERVER using this URL schema: 
           /home/wstein/myws/5/evaluate  
    The request does contain the input to the cell (POST).  Here worksheet_filename='myws', cell_id=5.

 4. (Alternatively, if the input was not changed -- e.g., in evaluate all.) the CLIENT sends a message to the SERVER using this URL schema: 
           /home/wstein/myws/5/evaluate
    The request does *NOT* contain the input to the cell.  The same URL, but the absense of the cell input text field means don't change it. 

 5. The SERVER receives the above request (let's just assume it is the evaluate one).  
    First it checks (via a DATABASE query) if a session_id has been assigned for this worksheet document.  (It hasn't.)

    The SERVER does a query about device status and load, runs a very
    fancy algorithm to conclude that DEVICE 1 (not DEVICE 0) is the
    way to go.

    The SERVER upserts the following cell document in the DATABASE:
               
             {cell_id:5, worksheet_filename:'myws', 
              input:"print 2\nsleep(10)\nprint 3\ngraph_editor()", 
              status:"needs_work", device:1, user_id='wstein'}

   After inserting this into the database, it returns a message to the CLIENT as follows:

             {cell_id:5, status:'needs_work'}

   The CLIENT receives the message and changes the cell 5 output to "working", and adds 5 to the list of needs_work cells.

 6. DEVICE 1 does a query for all cells that have status "needs_work"
    and for which device:1.  It gets back an iterator with one
    document in it, namely the above inserted document (from step 5).

    It then:

     - Allocates a fresh Python process with id 1974 for evaluation of code in the worksheet: 'wstein/myws'
       We have an in-memory table that maps wstein/myws to 1974.

     - Does a DATABASE query to change the cell:

             {status:"working..."}

     - Sends a message to the Python process with id 1974 to evaluate "print 2\nsleep(10)\nprint 3\ngraph_editor()". 
    
 7. The CLIENT queries the SERVER    

           /home/wstein/myws/5/update

    The SERVER does the following:

        - Responds to the CLIENT with nothing much, since nothing happened yet.

                   {cell_id:5, status:'working'}        (JSON)

     The CLIENT does *not* get the response message, due to a flakie network.

 8. Meanwhile, DEVICE 1 checks on its message queue and finds the
    following output for process 1974: sys.stdout:"2".  It then does
    this:

        - Update DATABASE cell 5 document:

                  {... output:{stdout_0:{type:'text', order:0, content:'2', state:'open'}} ...}

 9. Next, again the CLIENT queries for updates on cell 5 via the URL: 

           /home/wstein/myws/5/update

    The SERVER queries the DATABASE for info about cell 5 gets 

           ... {stdout_0:{type:'text', order:0, content:'2', state:'open'}} ...

    and returns the JSON message:

                  {cell_id:5, status:'working', output:{stdout_0:{type:'text', order:0, content:'2', state:'open'
  1. Now DEVICE 1 notices that more output has appeared from process 1974, namely "3\n" and it's closed this stream, since it is about to produce a plot, so it updates the DATABASE:
    • {... output:{stdout_0:{type:'text', order:0, content:'2\n3\n', state:'closed'}} ...}
    Incidentally, if there were any files 'foo.png' and 'bar.png' (say) created as a side effect (check modification times), then we would add them to the database as well. This is actually done entirely by the worker process.
    • {... output:{stdout_0:{type:'text', order:0,
    The code the device asks the worker process to execute will look like the following:
    • try:
      • block_api.new_block() print 2 sleep(10) print 3 plot(sin) print "hello" graph_editor()
      finally:
      • block_api.close()
    Note that the plot(sin)... function will actually call block_api.new_block(). At this moment, the block_api object will know its in a block, and check the filesystem for all new files created until now.

}}}