.. Last Change: Tue Nov 06 05:00 PM 2007 J
.. vim:syntax=rest

numpy.distutils.scons is basically an extension of numpy.distutils to use scons
for building extensions instead of distutils. Some of the advantages:

    - fine grained control of compilation environement (compiler options,
      etc...)
    - ability to build shared, static libraries in a cross platform manner
    - ability to build standard C python extension and Ctypes-based extensions
      in a cross platform manner
    - support library for high level system capabilities cheks, including
      availability of MKL, ATLAS, etc. and more generic framework to build your
      own, cross platform checks (think autoconf, but using python instead of
      M4)

How does it work ?
==================

The basic scheme is easy: 

    - an add_sconscript function has been added to
      numpy.distutils.Configuration. When called, this registers the sconscript
      given as an argument.
    - a scons command has been added as a distutils command. When called
      through python setup.py scons, scons is called for every SConscript
      registered by add_sconscript function (numpy/distutils/command/scons.py)
    - In the SConscript files, basic environments are obtained from the factory
      numscons.GetNumpyEnvironment. Those objects are true scons
      Environment, but correctly initialized with basic tools (C and fortran
      compilers, etc...), python and numpy specific builders (to build python
      extension, etc...), and some information necessary for scons <-> distutils
      interoperability.
    - As in standard scons, code is built using builders. However, for
      interoperability with distutils, special builders are provided to put any
      generated code in seperate directories, install built code in directories
      where distutils will find them (for install command), etc...

While not the most efficient (scons is launched as a new process for every
package registering a SConscript), this enables truly independant packages.
Basic build parameters (compiler, build directories, etc...) are passed to
scons through command lines argument, but generally, you don't need to
understand how exactly to use it.

Pro of per module scons call:
    - independant configuration and build
    - simplicity

Cons:
    - communication between scons and distutils is cumbersome (since they are
      in different processes). Passing info from distutils -> scons is doable,
      since scons is always called at the same point from distutils, but the
      contrary is more difficult. 

Distutils <-> scons interoperability
====================================

Once you get an Environment instance from GetNumpyEnvironment, you can use any
facility provided by scons. However, following the distutils tradition, some
special facilities are provided to avoid cluttering the source tree, and
putting any generated files (object code, libraries, etc...) inside a separate
build directory. Special builders which take care of the gory details are
provided. They are prefixed by Numpy to differentiate them from the standard
scons builders. For example, NumpySharedLibrary will builds a shared library
with the same api than scons' SharedLibrary, but will take care of setting
build directories, and install the built code in directories where distutils
will find them for later installation.

For now, builders to build Ctypes extensions, shared libraries and python
extensions are provided. I don't think static builders are necessary, since the
code is unusable for any python extension, hence not really making sense for
our usage.

Global design
=============

For now, numscons consists in mainly 5 parts: 
    - the core part: numpyenv.py, default.py, utils.py, libinfo.py and
      libinfo_scons.py
    - custom builders: custom_builder.py, extension.py, extension_scons.py
    - fortran support: fortran.py and fortran_scons.py.     
    - custom checkers: checker submodule
    - additional tools: tools submodule

Only the core is really necessary for basic scons use (that is code which does
not need BLAS/LAPACK or fortran support). Most of the other parts are necessary
for advanced only projects, which mix fortran/C code, etc...

Core
----

This makes interop with distutils possible. Some small parts are also parts of
numpy.distutils. Parts which communicate directly with distutils are kept to a
bare minimum, to avoid problems when setuptools will be better supported.  The
core main function is GetNumpyEnvironment, which is what user will use in
SConscripts to get environments.

A function NumpyCheckLib is also provided, to check libraries ala autoconf
(that is by trying to link a give library with a given list of symbols). The
function is designed to behave exactly like CheckLib, except that is also
supports being overridable by site.cfg and env var (ala system_info).

custom builders
---------------

For now, only building C based and Ctypes-based is supported (SWIG and Pyrex
will be next). Only the function whose names are prepended by Numpy should be
used (those take care of build dir configuration and so on). Those are
effectively scons builders.

When you implement a new generic builder, please follow the separation: a part
which has nothing to do with numpy, and a part based on the former to take care
of build dir configuration. This enables upstream integration of some code.

fortran support
---------------

This consists mainly in checkers usable from scons for mangling, link options
for Fortran/C interop, etc...

custom_checkers
---------------

This submodule defines some code for custom checkers. This is where performance
libraries checks are implemented, as well as meta-checkers. The idea is that
perflib defines functions to check for one performance library (say ATLAS,
MKL), and that checkers in checkers/custom_checkers.py are 'meta' checkers
based on perflib checkers. This makes meta-checkers easy to build on existing code.

Most of the code for perflib checkers is generic enough so that writing a new
one is easy (generally, a few lines of code). Those checkers can also
optionally find version information if you implement the version_check
function.

Tools
-----

For now, it implements a gfortran tool (not yet supported by upstream scons),
and a substinfile tool which is useful to generated a configuration file from a
python dictionary (think config.h.in -> config.h).
