Differences between revisions 32 and 79 (spanning 47 versions)
Revision 32 as of 2015-09-21 16:19:09
Size: 4846
Editor: chapoton
Comment: patchbot
Revision 79 as of 2020-09-01 11:45:05
Size: 8906
Editor: tmonteil
Comment: update info about askbot machines, remove combinat.sagemath.org machine that seems not to exist anymore (was used for mercurial queues)
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
This page aims to keep information public about Sage's infrastructure. Question mark means that the information is not confirmed, please do not hesitate to edit if you know more. This page aims to keep information public about Sage's infrastructure.
Question mark means that the information is not confirmed, please edit if you know more... and '''note the date of your update''' so that current and stale information can be distinguished.
Line 4: Line 5:

<<TableOfContents()>>
Line 13: Line 16:
  * purpose: webserver wher users can get support
  * hosted at: Ohio State University (Niles Johnson's university)
  * purpose: webserver where users can get support
  * hosted at: [[https://www.osu.edu/|Ohio State University]]
Line 16: Line 19:
  * there are two machines : ask-sagemath and ask-sagemath-db
Line 17: Line 21:
  * admins: niles, tmonteil, vedlecroix, slelievre, kcrisman, mhansen   * admin: niles, tmonteil
Line 19: Line 23:
  * local tech contact: David Alden, Josh Carroll
  * url: https://ask.sagemath.org
Line 20: Line 26:
== backup ==
  * purpose: keep the data of various services in a separate place
  * hosted at: [[https://www.math.u-psud.fr/?lang=fr|Mathematics lab]], Université Paris Sud (Orsay)
  * technology used: debian, rsync, cron
  * resources needed: 1TB disk
  * admin: tmonteil, slelievre
  * local sage dev: slelievre
  * local tech contact: Mathilde Rousseau
Line 21: Line 35:

== build.sagemath.org ==
  * purpose: distribute and gathers automatic binary building on volunteer's machines
  * hosted at: ?UW?
  * technology used: ubuntu, nginx: see [[buildbot]] for more details
  * resources needed: ?CPU ?RAM ?DISK
  * admins: ??
  * contact: ??


== doc.sagemath.org planet.sagemath.org www.sagemath.org ==
== doc.sagemath.org, planet.sagemath.org, www.sagemath.org ==
Line 36: Line 40:
  * admins: everyone who is on the github group for sagemath   * admin: everyone who is on the github group for sagemath
Line 38: Line 42:
  * url: https://doc.sagemath.org https://planet.sagemath.org https://www.sagemath.org
Line 39: Line 44:
== files.sagemath.org, fileserver.sagemath.org, old.files.sagemath.org ==
  * purpose: main files, spkgs, and backup of "other" files
  * what's at files.sagemath.org is ''exactly'' rsync.sagemath.org::sage
  * old.files.sagemath.org has the old stable releases
  * are the old development tarballs are still archived somewhere?
  * hosted at: UW. Other places are the mirrors of it.
  * contact: schilly, vbraun
  * admin: ohanar?
  * url: http://files.sagemath.org http://fileserver.sagemath.org http://old.files.sagemath.org
Line 40: Line 54:
== git.sagemath.org and trac.sagemath.org == == git.sagemath.org, trac.sagemath.org ==
Line 42: Line 56:
  * hosted at: ?UW?   * hosted at: google ?
Line 45: Line 59:
  * admins: ?
  * contact: ?
  * admin: people with root access: kclawson, ohanar, vbraun, mderickx, robertwb, wstein, dimpase
  * contact: no real organisation yet, go to sage-devel
  * url: https://git.sagemath.org https://trac.sagemath.org
Line 48: Line 63:
== GitHub organization for SageMath ==
  * url: https://github.com/sagemath
  * admin: schilly, vbraun

== GitLab organization for SageMath ==

  * url: https://gitlab.com/sagemath
  * admin: embray, jrueth

== build.sagemath.org ==
  * purpose: distribute and gathers automatic binary building on volunteer's machines
  * hosted at: University of Washington
  * technology used: ubuntu, nginx: see [[buildbot]] for more details
  * resources needed: ?CPU ?RAM ?DISK
  * admin: ??
  * contact: ??
  * url: http://build.sagemath.org
Line 51: Line 83:
  * hosted at: ?UW?
  * technology used: centos, custom flask, mongodb, see [[buildbot]] for more details
  * resources needed: Minimal CPU, RAM. ~25GB disk.
  * admins: robertwb, chapoton
  * contact: robertwb, chapoton

== PPA ==
  * purpose: easy-to-install Ubuntu .deb packages
  * hosted at: launchpad.net
  * admins: pipedream
  * contact: https://launchpad.net/~aims/+archive/ubuntu/sagemath
  * hosted at: [[https://irma.math.unistra.fr/|IRMA]], [[http://www.unistra.fr|Université de Strasbourg]]
  * technology used: ubuntu, nginx, uwsgi, flask, mongodb, see [[patchbot]] for more details
  * application source code: https://github.com/sagemath/sage-patchbot
  * resources needed: Minimal CPU, RAM. ~50GB disk.
  * admin: chapoton, tmonteil
  * root access: chapoton, tmonteil
  * contact: chapoton, tmonteil
  * local sage dev: chapoton
  * local tech contact: Alain Sartout
  * url: https://patchbot.sagemath.org
Line 66: Line 96:
  * hosted at: UW   * hosted at: University of Washington
Line 70: Line 100:
     * RAM 2-3 gb (mostly for caching when calculating hashes)      * RAM 2-3 GB (mostly for caching when calculating hashes)
Line 72: Line 102:
  * admins: schilly   * admin: schilly
Line 74: Line 104:
  * url: http://rsync.sagemath.org
Line 75: Line 106:
== sageb0t ==
  * purpose: turn pull requests on !GitHub into trac tickets
  * contact: robertwb
Line 79: Line 113:
    * Goethe-Universität (Frankfurt, Germany)
    * Universidad Autónoma de Madrid (Madrid, Spain)
    * [[Goethe-Universität|www.uni-frankfurt.de]] (Frankfurt, Germany)
    * [[https://www.uam.es|Universidad Autónoma de Madrid]] (Madrid, Spain)
Line 87: Line 121:
  * admins: novoselt   * admin: novoselt
Line 89: Line 123:
  * available urls: http://aleph.sagemath.org, https://sagecell.sagemath.org
Line 93: Line 127:
  * state: end of life, replaced by [[http://cloud.sagemath.com]]
  * volunteers: dimpase, tmonteil, vdelecroix
  * state: end of life, replaced by [[https://cocalc.com/|CoCalc]]
  * volunteers to provide worksheets back to the users: dimpase, tmonteil, vdelecroix
Line 98: Line 131:
  * purpose: the wiki you are currently reading!
  * hosted at: ?UW?
  * purpose: the wiki you are reading!
  * hosted at: moved to cloud.google.com ?
Line 102: Line 135:
  * admins: ?
  * contact: ?
  * possible issue in migration: it share the same db of users acounts than trac.
  * admin: people with root access: kclawson, wstein, ohanar, mmarco, dimpase
  * contact: no real organisation yet, go to sage-devel
  * possible issue in migration: the database of user accounts is synced from trac when this latter is modified (incron)
Line 106: Line 139:
== sagemath files ==
  * purpose: main files, spkgs, and backup of "other" files
  * what's at files.sagemath.org is ''exactly'' rsync.sagemath.org::sage
  * hosted at: UW. Other places are the mirrors of it.
  * contact: schilly, vbraun
== zulip.sagemath.org ==
  * purpose: chat system
  * hosted at: cloud.google.com
  * resources needed: a dedicated VM with 2-4GB of RAM
  * admin: roed, jrueth (saraedum) - as of 2020-05.
Line 112: Line 145:
== DockerHub organization ==
  * purpose: distribute SageMath Docker images
  * hosted at: https://hub.docker.com/u/sagemath/
  * admin: embray, jrueth
  * other: slelievre also has push access
Line 115: Line 153:
schilly: who wishes this and did define those purposes based on what discussion?   * schilly: who wishes this and did define those purposes based on what discussion?
  * tmonteil: this is a ''wishlist'', not something collectively decided. Note however that almost nothing in the current framework was collectively discussed either.
Line 120: Line 159:
     * tmonteil: backup of the data of the services we maintain: ask, trac, git, wiki,... according to [https://groups.google.com/forum/#!msg/sage-devel/ed_ya-d-k_E/5csVA6wsCwAJ this] [http://comments.gmane.org/gmane.comp.mathematics.sage.devel/84894 post] there is currently no regular backup at all for those services (except ask, that benefits for a semi-automatic backup system), this is why i proposed to have a server dedicated to that, located in a different place than the other services.
     * we got a VM at [[https://www.math.u-psud.fr/|Orsay math department]] with 1TB of disk, we are currently setting it up, and use it for more and more services.
Line 124: Line 165:
    * tmonteil: i hope this long time history could be easily fetched in some open format, or does this mean that we locked ourselves there? (being locked would imply that this tool is among the worst)
    * tmonteil: better for who? for google and the NSA to which we currently provide all our visitors IPs? There is a serious privacy issue with such analytics saas tools.
Line 126: Line 169:
    * tmonteil: note that free software alternative do exist
Line 130: Line 174:
     * tmonteil: googleID is an example of a service that was stopped by google, with strong consequences on our side (some people not able to connect anymore (e.g. on ask.sagemath.org), or not able to retrieve their work (e.g. on sagenb.org)). What will happen when google will decide that googlegroups will be shut down? Would we be able to discuss with them about this issue?

This page aims to keep information public about Sage's infrastructure. Question mark means that the information is not confirmed, please edit if you know more... and note the date of your update so that current and stale information can be distinguished.

Please mark which services still need to find a new home, with the system requirements and the deadline.

Current services

DNS

  • registrant: was
  • admin: was, schilly
  • tech contact: was, schilly

ask.sagemath.org

  • purpose: webserver where users can get support
  • hosted at: Ohio State University

  • technology used: ubuntu, nginx, uwsgi, postgesql, askbot
  • there are two machines : ask-sagemath and ask-sagemath-db
  • resources needed: database on a separate VM, 2GB RAM, 10GB disk
  • admin: niles, tmonteil
  • contact: [email protected]

  • local tech contact: David Alden, Josh Carroll
  • url: https://ask.sagemath.org

backup

  • purpose: keep the data of various services in a separate place
  • hosted at: Mathematics lab, Université Paris Sud (Orsay)

  • technology used: debian, rsync, cron
  • resources needed: 1TB disk
  • admin: tmonteil, slelievre
  • local sage dev: slelievre
  • local tech contact: Mathilde Rousseau

doc.sagemath.org, planet.sagemath.org, www.sagemath.org

files.sagemath.org, fileserver.sagemath.org, old.files.sagemath.org

git.sagemath.org, trac.sagemath.org

  • purpose: development tools, they share the same host
  • hosted at: google ?
  • technology used: ubuntu, apache, git, trac
  • resources needed: ?CPU ?RAM ?DISK
  • admin: people with root access: kclawson, ohanar, vbraun, mderickx, robertwb, wstein, dimpase
  • contact: no real organisation yet, go to sage-devel
  • url: https://git.sagemath.org https://trac.sagemath.org

GitHub organization for SageMath

GitLab organization for SageMath

build.sagemath.org

  • purpose: distribute and gathers automatic binary building on volunteer's machines
  • hosted at: University of Washington
  • technology used: ubuntu, nginx: see buildbot for more details

  • resources needed: ?CPU ?RAM ?DISK
  • admin: ??
  • contact: ??
  • url: http://build.sagemath.org

patchbot.sagemath.org

rsync.sagemath.org

  • purpose: the seed for mirrors, see MirrorNetwork for more details

  • hosted at: University of Washington
  • technology used: rsync
  • resources needed:
    • CPU 1 core
    • RAM 2-3 GB (mostly for caching when calculating hashes)
    • DISK enough for all sage files (50+ GB)
  • admin: schilly
  • contact: schilly
  • url: http://rsync.sagemath.org

sageb0t

  • purpose: turn pull requests on GitHub into trac tickets

  • contact: robertwb

sagecell.sagemath.org

  • purpose: allow embedding sage computations within a webpage
  • hosted at:
  • technology used: production installation requires a dedicated server (either physical or virtual)
  • resources needed:
    • RAM: 32GB recommended for smooth operation, 16GB may become enough in the future
    • CPU: the more the better for handling spikes in load and allowing parallel interacts, but any will do if necessary
    • DISK: must have BTRFS at least for /var/lib/lxc, SSD is preferable, 100GB should be sufficient for the foreseeable future
  • admin: novoselt
  • contact: novoselt
  • available urls: http://aleph.sagemath.org, https://sagecell.sagemath.org

sagenb.org

  • purpose: public notebook,
  • state: end of life, replaced by CoCalc

  • volunteers to provide worksheets back to the users: dimpase, tmonteil, vdelecroix

wiki.sagemath.org

  • purpose: the wiki you are reading!
  • hosted at: moved to cloud.google.com ?
  • technology used: ubuntu, apache, moinmoin
  • resources needed: ?CPU ?RAM ?DISK
  • admin: people with root access: kclawson, wstein, ohanar, mmarco, dimpase
  • contact: no real organisation yet, go to sage-devel
  • possible issue in migration: the database of user accounts is synced from trac when this latter is modified (incron)

zulip.sagemath.org

  • purpose: chat system
  • hosted at: cloud.google.com
  • resources needed: a dedicated VM with 2-4GB of RAM
  • admin: roed, jrueth (saraedum) - as of 2020-05.

DockerHub organization

Planned services (wishlist)

  • schilly: who wishes this and did define those purposes based on what discussion?
  • tmonteil: this is a wishlist, not something collectively decided. Note however that almost nothing in the current framework was collectively discussed either.

backup

stats

  • purpose: stop feeding google-analytics database
    • schilly: strong objection, we have a long time history stored there and it's far better than any other tools
    • tmonteil: i hope this long time history could be easily fetched in some open format, or does this mean that we locked ourselves there? (being locked would imply that this tool is among the worst)
    • tmonteil: better for who? for google and the NSA to which we currently provide all our visitors IPs? There is a serious privacy issue with such analytics saas tools.
  • public stats are here http://www.histats.com/viewstats/?SID=1579950&f=2

    • novoselt: horrific page - I thought such blinking adds are a thing of the past, had to close it before finding any data
    • tmonteil: note that free software alternative do exist

lists

  • purpose: do not depend on googlegroups (see what happened with googleID on ask and sagenb)
    • schilly strong objection: what has this to do with google id?
    • tmonteil: googleID is an example of a service that was stopped by google, with strong consequences on our side (some people not able to connect anymore (e.g. on ask.sagemath.org), or not able to retrieve their work (e.g. on sagenb.org)). What will happen when google will decide that googlegroups will be shut down? Would we be able to discuss with them about this issue?