Databases

[:days4/projects/: Other SAGE Days 4 Projects]

SAGE [http://www.initd.org/pub/software/pysqlite/doc/usage-guide.html interface] to sqlite: [:sqlite-tutorial: creating your own database using sqlite in sage]

Robert Miller, Emily Kirkman, Tom Boothby, David Joyner, Yi Qiang

Unresolved issues

  1. Should a generic SAGE Database object try to reproduce much advanced sqlite-fu? This came up during the first-run at coding such a class. If a generic SAGE Database were written to handle only more basic functions (such as only searching one table in a database at a time), it would be easier to maintain: its main purpose would be to expose an easy database structure to those not interested in learning the nuances of sqlite.
    • Note- this would not be a problem for developers, who could expand their own classes as they please.
    • This also wouldn't affect the global computation results database, which would be highly tweaked.
  2. lists as entries
    • storing a whole list in one entry in the database: this is feasible via Emily's rawking regexp skills, IF we can find a delimiter guaranteed not to be in pickle strings. It is optimal for speed and memory.
    • storing a list as an array of rows (experience tells us that this is inefficient)
  3. Whether it is worth it to have separate query objects on the same database (I can think of two advantages)
    • ability to do query arithmetic, perhaps in set theoretic language
    • ability to have different queries to check on an expanding database, for updates of certain subsets of information...
  4. Licensing: \exists DBGPL?

  5. Noticing identical database entries

Quick Notes

  1. ZODB may be very good for object persistence
    • caveats: sometimes you need to tell ZODB that you have changed some data or else it won't remember...
    • ZODB never shrinks a file, unless you call a vacuum-like command
    • pickling is dangerous without a plain text backup, since pickling can fail when you bridge versions...
      • therefore, it is best to have a plain human-readable text file, and a function that will read in the text and convert the data to current pickled cucumbers...
    • real-world data is great for recursive databasing, but algebraic objects can be too complicated...
    • it's like a dictionary...
  2. sqlite best for querying, not for quick reading and writing, i.e. best for serving
  3. Zope to serve webpages? [http://modular.fas.harvard.edu:8080/mfd/index.html e.g.]

SAGE Enhancement Proposal