Clear message

Introduction

Sage 2.8.5 and later supports a number of valgrind tools:

• memcheck: memory leak detector
• massif: heap and stack profiler

• cachegrind: cache profiler, i.e. L1 & L2 hit and misses

• callgrind: general profiler

Valgrinding Sage is difficult because it runs on top of Python which poses a number of challenges compared to pure C/C++ code. Run

sage -optional

to see the list of optional packages. You should install the valgrind.spkg with the highest revision number.

Valgrind Version

The latest official valgrind 3.2.3 does not work. You need to build from 3.3.0svn trunk. There is an experimental Sage valgrind.spkg that works and is used by many of the Sage developers.

Known Issues With Sage Components

• GAP cannot be valgrinded. It isn't due to the use of sbrk, but GAP somehow fails to extend its own workspace when run under valgrind
• libpari makes valgrind terminate itself when it expands its own allocated heap. There is no known workaround at this time. You should see an error message like:

valgrind: m_mallocfree.c:209 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 100000048, hi = 280100984.
Probably caused by overrunning/underrunning a heap block's bounds.

Using callgrind with Sage

callgrind is a powerful profiler that is part of valgrind (but it slows down your program a lot). Just do sage -callgrind to run Sage under callgrind; this will produce files callgrind.out.nnnn in $SAGE_ROOT/local/bin that can be interpreted by callgrind_annotate or by kcachegrind. (A single run of Sage will create multiple files, one for each process involved. That is, one for Sage, one for Singular, etc...)

To profile only part of a Sage run, edit $SAGE_ROOT/local/bin/sage-callgrind and add the flag --instr-atstart=no to the valgrind command line (before python -i). Then run sage -callgrind, and do whatever setup until you're ready to start profiling. In another window, run callgrind-control -i on, then press Enter in your Sage window. Now do whatever you want profiled. When that's completed, go back to your other window, run callgrind-control -i off, then press Enter in your Sage window again.

The default settings for callgrind do not interact well with Cython-generated code--it will assume that each call to PyEval_CallObjectWithKeywords is equally costly. To get callgrind to differentiate calls by their caller, add --seperate-callers=10 to $SAGE_ROOT/local/bin/sage-callgrind.

Preliminary Presentation

More to come at SD6. But: [attachment:SageDays5_preliminary_valgrind_talk.pdf]

Interesting Experimnetal Valgrind Tools/Components

• client requests: see http://valgrind.org/docs/manual/mc-manual.html#mc-manual.clientreqs

• omega: Is getting integrate as experimental tool into 3.3.0svn

• chronicle-recorder: Uses a patched 3.2.3 release - see http://code.google.com/p/chronicle-recorder/ - for a python interface see http://www.visophyte.org/blog/2007/07/26/chroniquery-chronicle-recorder-and-python-boogie-down/

ToDo

• Add lots more info how to build and use valgrind in general, mention experimental valgrind.spkg
• external links
• info about --without-pymalloc and --pydebug flags.
• examples on how to read log, find pyx code corresponding to C/C++ code