Differences between revisions 5 and 10 (spanning 5 versions)
Revision 5 as of 2009-07-14 22:50:23
Size: 28826
Editor: PeterJeremy
Comment: Document atlas build issue
Revision 10 as of 2009-08-07 11:16:50
Size: 28128
Editor: PeterJeremy
Comment: Continue documenting the necessary patches.
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= Sage FreeBSD 7.x/amd64 and 8-current/amd64 build notes for Sage 4.1 =

Note that this page is under construction and currently incomplete
= Sage FreeBSD 7.x and 8.x build notes for Sage 4.1 =

Note that this page is under construction and currently incomplete. In particular, it still includes references to sage-3.4 in areas that have not been converted to sage-4.1.
Line 17: Line 17:
 * Install ports/converters/libiconv - This may be optional but is definitely used if found
 
 * Install ports/converters/libiconv
Line 26: Line 26:
 * Unpack sage-4.1.rc1.tar
 * `cd sage-4.1.rc1`
 * Unpack sage-4.1.tar
 * `cd sage-4.1`
Line 37: Line 37:
  . `SAGE_FORTRAN_LIB=/usr/local/lib/gcc-4.3.4/libgfortran.so gmake`   . `SAGE_FORTRAN=/usr/local/bin/gfortran43 SAGE_FORTRAN_LIB=/usr/local/lib/gcc43/libgfortran.so gmake`
Line 129: Line 129:
 * Building the `libclapack.so` should result in a `libclapack.so` being created. This appears to be a copy-and-paste error in atlas and has been reported as https://sourceforge.net/tracker/?func=detail&atid=379483&aid=2728935&group_id=23725
 * numpy on FreeBSD expects atlas to build threaded libraries (eg libatlas_r) [and possibly static, this isn't clear]. The current atlas build is not creating the expected libraries. This is still under investigation.
 * Treat shared libraries the same as Linux - otherwise they aren't correctly detected by (eg) numpy. (sage-specific)
 * Building the `libclapack.so` should result in a `libclapack.so` being created. This appears to be a copy-and-paste error in atlas and has been reported as https://sourceforge.net/tracker/?func=detail&atid=379483&aid=2728935&group_id=23725 (Note that this is not relevant to sage as libclapack is not built or used).
 * numpy on FreeBSD expects atlas to build different and (according to the FreeBSD numpy port, static) threaded libraries: libatlas_r instead of libatlas and libalapack_r instead of liblapack. libatlas_r is built as part of the threaded version of atlas. libalapack_r is a FreeBSD special which explicitly combines all of liblapack from the atlas lapack build with the threaded atlas version of liblapack.
Line 136: Line 135:
  libraries f77blas,cblas,atlas_r not found in /home/pjeremy/sage-3.4/local/lib   libraries f77blas,cblas,atlas_r not found in /usr/home/peter/sage-4.1/local/lib
Line 139: Line 138:
/home/pjeremy/sage-3.4/spkg/build/numpy-1.2.0.p0/src/numpy/distutils/system_info.py:1340: UserWarning:  /usr/home/peter/sage-4.1/spkg/build/numpy-1.3.0.p0/src/numpy/distutils/system_info.py:1383: UserWarning:
Line 147: Line 146:
  libraries f77blas,cblas,atlas_r not found in /home/pjeremy/sage-3.4/local/lib
  libraries lapack_atlas not found in /home/pjeremy/sage-3.4/local/lib
  libraries f77blas,cblas,atlas_r not found in /usr/home/peter/sage-4.1/local/lib
  libraries lapack_atlas not found in /usr/home/peter/sage-4.1/local/lib
Line 152: Line 151:
/home/pjeremy/sage-3.4/spkg/build/numpy-1.2.0.p0/src/numpy/distutils/system_info.py:1247: UserWarning:  /usr/home/peter/sage-4.1/spkg/build/numpy-1.3.0.p0/src/numpy/distutils/system_info.py:1290: UserWarning:
Line 158: Line 157:
...
/home/pjeremy/sage-3.4/local/bin/gfortran -Wall build/temp.freebsd-7.0-STABLE-amd64-2.5/numpy/linalg/lapack_litemodule.o build/temp.freebsd-7.0-STABLE-amd64-2.5/numpy/linalg/python_xerbla.o -L/home/pjeremy/sage-3.4/local/lib -L/usr/local/lib/gcc-4.3.4/gcc/x86_64-portbld-freebsd7.0/4.3.4 -llapack -lblas -lgfortran -o build/lib.freebsd-7.0-STABLE-amd64-2.5/numpy/linalg/lapack_lite.so
build/temp.freebsd-7.0-STABLE-amd64-2.5/numpy/linalg/lapack_litemodule.o(.text+0x41): In function `check_object':
numpy/linalg/lapack_litemodule.c:103: undefined reference to `PyType_IsSubtype'
build/temp.freebsd-7.0-STABLE-amd64-2.5/numpy/linalg/lapack_litemodule.o(.text+0x7b):numpy/linalg/lapack_litemodule.c:114: undefined reference to `PyErr_Format'
build/temp.freebsd-7.0-STABLE-amd64-2.5/numpy/linalg/lapack_litemodule.o(.text+0xb7):numpy/linalg/lapack_litemodule.c:109: undefined reference to `PyErr_Format'
Line 256: Line 249:
{{{#!wiki comment
=== matplotlib-0.98.5.3rc0-svn6910.p3 ===

=== matplotlib-0.98.5.3rc0-svn6910.p4 ===
Line 292: Line 285:
}}}
Line 310: Line 303:
gcc -o libpari-gmp.so.2.3.3 -shared -O3 -Wall -fno-strict-aliasing -fomit-frame-pointer -Wl,-shared,-soname=libpari-gmp.so.2 mp.o mpinl.o Flx.o Qfb.o RgX.o alglin1.o alglin2.o arith1.o arith2.o base1.o base2.o base3.o base4.o base5.o bibli1.o bibli2.o buch1.o buch2.o buch3.o buch4.o galconj.o gen1.o gen2.o gen3.o ifactor1.o perm.o polarit1.o polarit2.o polarit3.o rootpol.o subcyclo.o subgroup.o trans1.o trans2.o trans3.o anal.o compat.o default.o errmsg.o es.o init.o intnum.o members.o sumiter.o aprcl.o elldata.o elliptic.o galois.o groupid.o kummer.o mpqs.o nffactor.o part.o stark.o subfield.o thue.o -lc -lm -L/home/pjeremy/sage-3.4/local/lib -lgmp  gcc -o libpari-gmp.so.2.3.3 -shared -O3 -Wall -fno-strict-aliasing -fomit-frame-pointer -Wl,-shared,-soname=libpari-gmp.so.2 mp.o mpinl.o Flx.o Qfb.o RgX.o alglin1.o alglin2.o arith1.o arith2.o base1.o base2.o base3.o base4.o base5.o bibli1.o bibli2.o buch1.o buch2.o buch3.o buch4.o galconj.o gen1.o gen2.o gen3.o ifactor1.o perm.o polarit1.o polarit2.o polarit3.o rootpol.o subcyclo.o subgroup.o trans1.o trans2.o trans3.o anal.o compat.o default.o errmsg.o es.o init.o intnum.o members.o sumiter.o aprcl.o elldata.o elliptic.o galois.o groupid.o kummer.o mpqs.o nffactor.o part.o stark.o subfield.o thue.o -lc -lm -L/home/pjeremy/sage-3.4/local/lib -lgmp
Line 324: Line 317:
{{{#!wiki comment
=== r-2.6.1.p22 ===
Pass CFLAGS, CPPFLAGS and LDFLAGS from the environment into the build process.

=== r-2.6.1.p23 ===
Pass CFLAGS, CPPFLAGS and LDFLAGS from the environment into the build process.  This also corrects a typo in CPPFLAGS.
Line 333: Line 326:
}}}
Line 339: Line 332:
gcc -shared -o libreadline.so.5 readline.so vi_mode.so funmap.so keymaps.so parens.so search.so rltty.so complete.so bind.so isearch.so display.so signals.so util.so kill.so undo.so macro.so input.so callback.so terminal.so text.so nls.so misc.so xmalloc.so history.so histexpand.so histfile.so histsearch.so shell.so mbutil.so tilde.so compat.so  gcc -shared -o libreadline.so.5 readline.so vi_mode.so funmap.so keymaps.so parens.so search.so rltty.so complete.so bind.so isearch.so display.so signals.so util.so kill.so undo.so macro.so input.so callback.so terminal.so text.so nls.so misc.so xmalloc.so history.so histexpand.so histfile.so histsearch.so shell.so mbutil.so tilde.so compat.so
Line 341: Line 334:
gcc -shared -o libhistory.so.5 history.so histexpand.so histfile.so histsearch.so shell.so mbutil.so xmalloc.so  gcc -shared -o libhistory.so.5 history.so histexpand.so histfile.so histsearch.so shell.so mbutil.so xmalloc.so

Sage FreeBSD 7.x and 8.x build notes for Sage 4.1

Note that this page is under construction and currently incomplete. In particular, it still includes references to sage-3.4 in areas that have not been converted to sage-4.1.

Contents

Overall build environment differences

  • sh and /bin/sh are a POSIX shell, rather than bash.

  • make is the BSD make, not GNU make

Preparatory work

  • Install ports/shells/bash or ports/shells/bash3 (I used 4.0.24)
  • Install ports/devel/gmake
  • Install ports/lang/gcc43 - FreeBSD no longer ships with a Fortran compiler by default.
  • Install ports/devel/autoconf262
  • Install ports/converters/libiconv

It's likely there are other dependencies, I haven't tried building sage in a clean (tinderbox) environment.

Building Sage

The Sage build chain automatically unpacks and builds each spkg, however in order to build on FreeBSD, a number of spkgs need patching. Rather than manually unpack, patch and repack each spkg, I have instead modified base/sage-spkg to automatically apply a local patch if it exists.

On FreeBSD 7.x

  • Unpack sage-4.1.tar
  • cd sage-4.1

  • mkdir -p local/bin spkg/patches

  • Create symlinks to mask name differences.
    • ln -s /usr/local/bin/gmake local/bin/make

    • ln -s /usr/local/bin/bash local/bin/sh

    • ln -s /usr/local/bin/gcc43 local/bin/gcc

    • ln -s /usr/local/bin/g++43 local/bin/g++

    • ln -s /usr/local/bin/gfortran43 local/bin/gfortran

  • Copy the patches listed below into spkg/patches

  • Build Sage
    • SAGE_FORTRAN=/usr/local/bin/gfortran43 SAGE_FORTRAN_LIB=/usr/local/lib/gcc43/libgfortran.so gmake

Current Status

Notes on spkgs and attached patches

A number of the following patches change make to ${MAKE}. This the recommended way to recursively invoke make to ensure that the subordinate make is the same as the parent make (and also ensures that the two make instances will communicate on things like '-jX').

base.patch

base.txt

  • The patch to base/sage-spkg enables the local patching that the rest of the patches rely on. Note that this patch is not intended to be merged into sage but provides a convenient mechanism to apply local patches without requiring that the spkg files are locally re-rolled.

atlas-3.8.3.p5

Various fixes to make it compile on FreeBSD

  • FreeBSD uses an '_fbsd' suffix on the ELF format supported by ld - prevents ld: unrecognised emulation mode: elf_x86_64 error during atlas build. Reported upstream as https://sourceforge.net/tracker/?func=detail&atid=379483&aid=2728930&group_id=23725

  • Building the libclapack.so should result in a libclapack.so being created. This appears to be a copy-and-paste error in atlas and has been reported as https://sourceforge.net/tracker/?func=detail&atid=379483&aid=2728935&group_id=23725 (Note that this is not relevant to sage as libclapack is not built or used).

  • numpy on FreeBSD expects atlas to build different and (according to the FreeBSD numpy port, static) threaded libraries: libatlas_r instead of libatlas and libalapack_r instead of liblapack. libatlas_r is built as part of the threaded version of atlas. libalapack_r is a FreeBSD special which explicitly combines all of liblapack from the atlas lapack build with the threaded atlas version of liblapack.

numpy error from last point above:

atlas_blas_info:
  libraries f77blas,cblas,atlas_r not found in /usr/home/peter/sage-4.1/local/lib
  NOT AVAILABLE

/usr/home/peter/sage-4.1/spkg/build/numpy-1.3.0.p0/src/numpy/distutils/system_info.py:1383: UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
...
atlas_info:
  libraries f77blas,cblas,atlas_r not found in /usr/home/peter/sage-4.1/local/lib
  libraries lapack_atlas not found in /usr/home/peter/sage-4.1/local/lib
numpy.distutils.system_info.atlas_info
  NOT AVAILABLE

/usr/home/peter/sage-4.1/spkg/build/numpy-1.3.0.p0/src/numpy/distutils/system_info.py:1290: UserWarning:
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  warnings.warn(AtlasNotFoundError.__doc__)
...

matplotlib-0.98.5.3rc0-svn6910.p4

Add hooks to ensure that the local Sage versions of dependencies are used by FreeBSD. Otherwise you get:

BUILDING MATPLOTLIB
            matplotlib: 0.98.6svn
                python: 2.5.2 (r252:60911, Apr  1 2009, 08:50:05)  [GCC
                        4.3.4 20090326 (prerelease)]
              platform: freebsd7

REQUIRED DEPENDENCIES
                 numpy: no
                        * You must install numpy 1.1 or later to build
                        * matplotlib.
Error building matplotlib package.
}#}}

Add support for FreeBSD later than 6.x.  Otherwise you get :
{{{
REQUIRED DEPENDENCIES
Traceback (most recent call last):
  File "setup.py", line 119, in <module>
    if not check_for_numpy():
  File "/home/pjeremy/sage-3.4/spkg/build/matplotlib-0.98.5.3rc0-svn6910.p3/src/setupext.py", line 499, in check_for_numpy
    add_base_flags(module)
  File "/home/pjeremy/sage-3.4/spkg/build/matplotlib-0.98.5.3rc0-svn6910.p3/src/setupext.py", line 320, in add_base_flags
    [os.path.join(p, 'include') for p in basedir[sys.platform] ])
KeyError: 'freebsd7'
Error building matplotlib package.
}#}}

gcc4.3 (though not gcc4.2) appears to define putchar() in a way that breaks the putchar() definitions inside ttconv.  I couldn't quickly identify what the underlying cause was so I took the easy way out and undef'd the offending putchar() macro.

Individual character bounding boxes in AFM files do not have to be integral so convert each bounding box to a list of floats, rather than a list of ints.  This corrects a problem where most of the tests would fail with `ValueError: invalid literal for int() with base 10: '539.621'`.

All three sub-patches are in [[http://trac.sagemath.org/sage_trac/ticket/5873|#5873]]

{{{#!wiki comment
=== mpfr-2.4.1 ===
Portability fixes so the configure script works with a POSIX shell.

r-2.6.1.p23

Pass CFLAGS, CPPFLAGS and LDFLAGS from the environment into the build process. This also corrects a typo in CPPFLAGS.

Note that FreeBSD needs the path to libiconv to be explicitly specified. In theory, --with-libiconv-prefix should work but configure script is broken and ignores that path when looking for libiconv. Hard-wire /usr/local/include and /usr/local/lib via xxFLAGS.

Note that the X11 detection in spkg-install will not work anywhere other than Linux. This needs future work.

These patches are all local to Sage.

sage_scripts-4.1.rc1

Patch sage-spkg to apply local patches. This patch also disables deletion of the spkg/build/FOO temporary directories - which was useful during porting. This latter patch can be safely removed.