Diff for "days7/DebianPackagingSprint"

Differences between revisions 1 and 22 (spanning 21 versions)

Debian release to support

sbuild: lenny x86-64: can create 32 & 64 bit arch specific packages, ~12 in total:

Ubuntu LTS
last two Ubuntu releases
>=etch

Make check

use schroot to run "make check"

Problematic Packages

ATLAS (atlas-sage-x.y or version bump)
Python (python-sage-x.y or version bump)
jmol - potential conflict with gcj in stable (testing and above fine)
Maxima: copies mgnuplot script to libexec (Maxima is already in Debian)

Changelog

needs to be kept in sync with SPKG.txt (automated mechanism)
dch (Debian changelog)

Initially

webserver for static only content
set up apt-get repo on $SAGE.MATH
To set up repo: reprpro + gpg key for signing
domain: http://debian.sagemath.org registered, still points at sagemath.org, needs vhost to be set up

Problem to be solved

Install location: /usr/share/sage/ - "sage" script in bin does cd and
- then execute the appropriate sage scipt in /usr/share/sage/
location of tmp, i.e. TMP,/tmpor/var/tmp,notSAGE_ROOT/tmp
SAGE_ROOT/ipython - why is it there. What is the content (config files?)
databases in $SAGE_ROOT/data -> /use/share/sage/data
spkg directory: gone or moved to sage-dev.deb
make distclean: remove "*.c", "*.so", "*.os" in /devel/sage/
content of examples directory: What to do with the code?
LIB -> lib - Singular related, move to lib/Singular
text files from $SAGE_ROOT: /usr/share/doc/sage
create official Sage GPG key to sign packages

Package names

Comply with Debian policy when possible
libraries (for example NTL):
- ntl_5.4.1-1.dsc (the source source package), tarball with debian directory
- libntl0_5.4.1-1_i386.deb (initial NTL version, i.e. "0", contains shared library)
- libntl-dev_5.4.1-1_i386.deb (headers & static libraries)
binary (eclib)
- eclib_1.0_i386.deb (binaries)
- libec0_1.0_i386.deb (shared libraries)
- libec-dev_1.0_i386.deb (headers & static libraries)
Singular
- To be determined

Setting up the build system (In Detail)

based on build system for Debian-Athena
Start Debian installer
Partitioning method: "Guided - use entire disk and set up LVM" - accept defaults and write LVM init to disk
delete original main LVM volume, create new VG, say with 10GB
set up accounts for root and users as usual
update to lenny: edit /etc/apt/source and replace "etch" with "lenny"
"aptitude update"
"aptitude dist-upgrade" (wait a while )
download Tim's script collection from the main Deb packaging page (tarballs here for now), untar ... [the code is being partially merged in the sage-scripts repo, some of the other might end up on a wiki page somewhere else once this is over]
file "packages" contains list of packages needed for the build system
aptitude install cat packages installs the packages listed in packages. Also install cdbs.
file "approx.conf.tail": append file to approx.conf - make sure to set a fast mirror, i.e. not the MIT repo
restart approx service via "/etc/init.d/approx restart"
edit make-chroot and set $VG to volume group name or export VG
Now create change roots for various targets using make-chroot: Parameters 1isSUITE(i.e.lenny),2 is ARCH (i.e. i386)
If things go wrong remove lvm group created
download Tim's other script collection from the main Deb packaging page (tarballs for now), untar into /usr/local/bin ...
export DEBATHENA_APT=(path to SAGE apt repository)
adduser sage sbuild (where sage is a user you want to be able to do builds)

Using the chroot environments

Create GPG key: gpg --gen-key (follow instructions)
export key in ascii format: gpg --export -armor > sage-archive.asc
mount-defaults: uncomment /dev/pts, tmpfs mounts since there are currently problems with schroot 1.1.6-1. P potential solution: uncomment /dev/pts, /dev/shm mount points
starting schroot env: example gutsy:
- schroot -c gutsy-i386-sbuild

To actually do builds

get SAGE .spkg and unpack into DIR
copy debian directory into $DIR/src
build source package using debuild -S from inside package directory ($DIR/src)
build binary packages using "da sbuildhack [resulting .dsc file]" from $DIR
upload into SAGE repository using "daupload-release [resulting .changes file]" from $DIR
use clean-schroots whenever you run out of disk space due to stale build chroots (often get leaked when you do ^C during a build)
autotools-dev, m4 (should be added to the dependencies - if they aren't)
using sbuildhack - setup: adduser $USER sbuild [make sure user is in right group]
/etc/sbuild/sbuild.conf:
- set mailto, maintainer, etc...
- uncomment '$sbuild_mode = "user";'

Example

apt-get source valgrind [downloads sources, patches, dsc file]
sbuildhack lenny-i386 valgrind-3.3.0-1.dsc
Building a Debian package via make from $SAGE_LOCAL:
- set env variable DEBIAN_RELEASE to "lenny-i386" [building deb in schroot env]
- set env variable USEDEB to "yes": first try to sudo apt-get install $SPKG.deb
- sage-spkg: check if DEBIAN_RELEASE is set: if spkg-debian exists in $SPKG_ROOT execute it, otherwise do default sbuildhack - see http://trac.sagemath.org/sage_trac/ticket/2098 for patch

Structure for dist-specific build systems in general

Can expect upstream sources to be available at $SPKG_ROOT/src
Can store distribution-specific files in $S P K G_{R} O O T / d i s t /$ NAME

Possibly useful links

Debian alternatives system: see http://www.debian-administration.org/articles/91, http://manpages.debian.net/cgi-bin/man.cgi?query=update-alternatives&apropos=0&sektion=0&manpath=Debian+Sid&format=html&locale=en

-  ⇤ ← Revision 1 as of 2008-02-07 05:51:52 → 
  Size: 1458
  Editor: MichaelAbshoff
  Comment: initial version
+   ← Revision 22 as of 2009-03-26 01:09:18 → ⇥
  Size: 6141
  Editor: MichaelAbshoff
  Comment: fix links
-Deletions are marked like this.
+Additions are marked like this.
 Line 3:
-sbuild: lenny x86-64: can create 32 & 64 bit arch specific packages, 14 in total:
+sbuild: lenny x86-64: can create 32 & 64 bit arch specific packages, ~12 in total:
 Line 5:
- * Untuntu LTS
 * last two ubuntu releases
+ * Ubuntu LTS
 * last two Ubuntu releases
 Line 16:
- * jmol - potential conflict with gcj in stable (testing an above fine)
+ * jmol - potential conflict with gcj in stable (testing and above fine)
 Line 28:
- * domain: mabye "debian.sagemath.org"
+ * domain: http://debian.sagemath.org registered, still points at sagemath.org, needs vhost to be set up
 Line 42:
+ * create official Sage GPG key to sign packages

== Package names ==
 
 * Comply with Debian policy when possible
 * libraries (for example NTL):
  * ntl_5.4.1-1.dsc (the source source package), tarball with debian directory
  * libntl0_5.4.1-1_i386.deb (initial NTL version, i.e. "0", contains shared library) 
  * libntl-dev_5.4.1-1_i386.deb (headers & static libraries)
 * binary (eclib)
  * eclib_1.0_i386.deb (binaries)
  * libec0_1.0_i386.deb (shared libraries)
  * libec-dev_1.0_i386.deb (headers & static libraries)
 * Singular
  * To be determined

== Setting up the build system (In Detail) ==

 * based on build system for [[http://debathena.mit.edu|Debian-Athena]]
 * Start Debian installer
 * Partitioning method: "Guided - use entire disk and set up LVM" - accept defaults and write LVM init to disk
 * delete original main LVM volume, create new VG, say with 10GB
 * set up accounts for root and users as usual
 * update to lenny: edit /etc/apt/source and replace "etch" with "lenny"
 * "aptitude update"
 * "aptitude dist-upgrade" (wait a while :) )
 * download Tim's script collection from the main Deb packaging page [[http://wiki.sagemath.org/days7/DebianPackagingSprint?action=AttachFile|(tarballs here for now)]], untar ... [the code is being partially merged in the sage-scripts repo, some of the other might end up on a wiki page somewhere else once this is over]
 * file "packages" contains list of packages needed for the build system
 * aptitude install `cat packages` installs the packages listed in packages.  Also install cdbs.
 * file "approx.conf.tail": append file to approx.conf - make sure to set a fast mirror, i.e. not the MIT repo
 * restart approx service via "/etc/init.d/approx restart"
 * edit make-chroot and set  $V G t o v o l u m e g r o u p n a m e o r e x p o r t V G * N o w c r e a t e c h a n g e r o o t s f o r v a r i o u s t a r g e t s u s i n g m a k e - c h r o o t : P a r a m e t e r s$ 1 is SUITE (i.e. lenny),  $2 i s A R C H (i . e . i 386) * I f t h i n g s g o w r o n g r e m o v e l v m g r o u p c r e a t e d : (* d o w n l o a d T i m^{'} s o t h e r s c r i p t c o l l e c t i o n f r o m t h e m a i n D e b p a c k a g i n g p a g e [[h t t p : / / w w w . s a g e m a t h . o r g : 9001 / d a y s 7 / D e b i a n P a c k a g i n g S p r i n t ? a c t i o n = A t t a c h F i l e | (t a r b a l l s f o r n o w)]], u n t a r i n t o / u s r / l o c a l / b i n . . . * e x p o r t D E B A T H E N A_{A} P T = (p a t h t o S A G E a p t r e p o s i t o r y) * a d d u s e r s a g e s b u i l d (w h e r e s a g e i s a u s e r y o u w a n t t o b e a b l e t o d o b u i l d s) == U s i n g t h e c h r o o t e n v i r o n m e n t s == * C r e a t e G P G k e y : g p g - - g e n - k e y (f o l l o w i n s t r u c t i o n s) * e x p o r t k e y i n a s c i i f o r m a t : g p g - - e x p o r t - a r m o r > s a g e - a r c h i v e . a s c * m o u n t - d e f a u l t s : u n c o m m e n t / d e v / p t s, t m p f s m o u n t s s i n c e t h e r e a r e c u r r e n t l y p r o b l e m s w i t h s c h r o o t 1.1 .6 - 1. P p o t e n t i a l s o l u t i o n : u n c o m m e n t / d e v / p t s, / d e v / s h m m o u n t p o i n t s * s t a r t i n g s c h r o o t e n v : e x a m p l e g u t s y : s c h r o o t - c g u t s y - i 386 - s b u i l d == T o a c t u a l l y d o b u i l d s == * g e t S A G E . s p k g a n d u n p a c k i n t o D I R * c o p y d e b i a n d i r e c t o r y i n t o$ DIR/src
 * build source package using debuild -S from inside package directory ( $D I R / s r c) * b u i l d b i n a r y p a c k a g e s u s i n g " d a s b u i l d h a c k [r e s u l t i n g . d s c f i l e] " f r o m$ DIR
 * upload into SAGE repository using "daupload-release [resulting .changes file]" from  $D I R * u s e c l e a n - s c h r o o t s w h e n e v e r y o u r u n o u t o f d i s k s p a c e d u e t o s t a l e b u i l d c h r o o t s (o f t e n g e t l e a k e d w h e n y o u d o^{C} d u r i n g a b u i l d) * a u t o t o o l s - d e v, m 4 (s h o u l d b e a d d e d t o t h e d e p e n d e n c i e s - i f t h e y a r e n^{'} t) * u s i n g s b u i l d h a c k - s e t u p : a d d u s e r$ USER sbuild [make sure user is in right group]
 * {{{/etc/sbuild/sbuild.conf}}}:
      * set mailto, maintainer, etc...
      * uncomment '$sbuild_mode = "user";'

=== Example ===

 * apt-get source valgrind [downloads sources, patches, dsc file]
 * sbuildhack lenny-i386 valgrind-3.3.0-1.dsc

 * Building a Debian package via {{{make}}} from {{{$SAGE_LOCAL}}}:
   * set env variable {{{DEBIAN_RELEASE}}} to "lenny-i386" [building deb in schroot env]
   * set env variable {{{USEDEB}}} to "yes": first try to sudo apt-get install {{{$SPKG.deb}}}
   * sage-spkg: check if {{{DEBIAN_RELEASE}}} is set: if spkg-debian exists in {{{$SPKG_ROOT}}} execute it, otherwise do default sbuildhack - see http://trac.sagemath.org/sage_trac/ticket/2098 for patch


== Structure for dist-specific build systems in general ==

 * Can expect upstream sources to be available at {{{$SPKG_ROOT/src}}}
 * Can store distribution-specific files in {{{ $S P K G_{R} O O T / d i s t /$ NAME}}}

== Possibly useful links ==

 * Debian alternatives system: see [[http://www.debian-administration.org/articles/91]], [[http://manpages.debian.net/cgi-bin/man.cgi?query=update-alternatives&apropos=0&sektion=0&manpath=Debian+Sid&format=html&locale=en]]