Packaging (numpy.distutils)

NumPy provides enhanced distutils functionality to make it easier to build and install sub-packages, auto-generate code, and extension modules that use Fortran-compiled libraries. To use features of numpy distutils, use the setup command from numpy.distutils.core. A useful Configuration class is also provided in numpy.distutils.misc_util that can make it easier to construct keyword arguments to pass to the setup function (by passing the dictionary obtained from the todict() method of the class). More information is available in the NumPy Distutils Users Guide in <site-packages>/numpy/doc/DISTUTILS.txt.

Modules in numpy.distutils

misc_util

Configuration
get_numpy_include_dirs ()
get_numarray_include_dirs
dict_append (d, **kws)
appendpath (prefix, path)
allpath (name)	Convert a /-separated pathname to one using the OS’s path separator.
dot_join (*args)
generate_config_py (target)	Generate config.py file containing system_info information used during building the package.
get_cmd (cmdname[, _cache])
terminal_has_colors ()
red_text (s)
green_text (s)
yellow_text (s)
blue_text (s)
cyan_text (s)
cyg2win32 (path)
all_strings (lst)	Return True if all items in lst are string objects.
has_f_sources (sources)	Return True if sources contains Fortran files
has_cxx_sources (sources)	Return True if sources contains C++ files
filter_sources (sources)	Return four lists of filenames containing C, C++, Fortran, and Fortran 90 module sources, respectively.
get_dependencies (sources)
is_local_src_dir (directory)	Return true if directory is local directory.
get_ext_source_files (ext)
get_script_files (scripts)

class numpy.distutils.misc_util.Configuration(package_name=None, parent_name=None, top_path=None, package_path=None, **attrs)

Construct a configuration instance for the given package name. If parent_name is not None, then construct the package as a sub-package of the parent_name package. If top_path and package_path are None then they are assumed equal to the path of the file this instance was created in. The setup.py files in the numpy distribution are good examples of how to use the Configuration instance.

todict()

Return a dictionary compatible with the keyword arguments of distutils setup function. Thus, this method may be used as setup(**config.todict()).

get_distribution()

Return the distutils distribution object for self.

get_subpackage(subpackage_name, subpackage_path=None)

Return a Configuration instance for the sub-package given. If subpackage_path is None then the path is assumed to be the local path plus the subpackage_name. If a setup.py file is not found in the subpackage_path, then a default configuration is used.

add_subpackage(subpackage_name, subpackage_path=None)

Add a sub-package to the current Configuration instance. This is useful in a setup.py script for adding sub-packages to a package. The sub-package is contained in subpackage_path / subpackage_name and this directory may contain a setup.py script or else a default setup (suitable for Python-code-only subpackages) is assumed. If the subpackage_path is None, then it is assumed to be located in the local path / subpackage_name.

self.add_data_files(*files)

Add files to the list of data_files to be included with the package. The form of each element of the files sequence is very flexible allowing many combinations of where to get the files from the package and where they should ultimately be installed on the system. The most basic usage is for an element of the files argument sequence to be a simple filename. This will cause that file from the local path to be installed to the installation path of the self.name package (package path). The file argument can also be a relative path in which case the entire relative path will be installed into the package directory. Finally, the file can be an absolute path name in which case the file will be found at the absolute path name but installed to the package path.

This basic behavior can be augmented by passing a 2-tuple in as the file argument. The first element of the tuple should specify the relative path (under the package install directory) where the remaining sequence of files should be installed to (it has nothing to do with the file-names in the source distribution). The second element of the tuple is the sequence of files that should be installed. The files in this sequence can be filenames, relative paths, or absolute paths. For absolute paths the file will be installed in the top-level package installation directory (regardless of the first argument). Filenames and relative path names will be installed in the package install directory under the path name given as the first element of the tuple. An example may clarify:

self.add_data_files('foo.dat',
('fun', ['gun.dat', 'nun/pun.dat', '/tmp/sun.dat']),
'bar/cat.dat',
'/full/path/to/can.dat')

will install these data files to:

<package install directory>/
 foo.dat
 fun/
   gun.dat
   nun/
     pun.dat
 sun.dat
 bar/
   car.dat
 can.dat

where <package install directory> is the package (or sub-package) directory such as ‘/usr/lib/python2.4/site-packages/mypackage’ (‘C: \Python2.4 \Lib \site-packages \mypackage’) or ‘/usr/lib/python2.4/site- packages/mypackage/mysubpackage’ (‘C: \Python2.4 \Lib \site-packages \mypackage \mysubpackage’).

An additional feature is that the path to a data-file can actually be a function that takes no arguments and returns the actual path(s) to the data-files. This is useful when the data files are generated while building the package.

add_data_dir(data_path)

Recursively add files under data_path to the list of data_files to be installed (and distributed). The data_path can be either a relative path-name, or an absolute path-name, or a 2-tuple where the first argument shows where in the install directory the data directory should be installed to. For example suppose the source directory contains fun/foo.dat and fun/bar/car.dat:

self.add_data_dir('fun')
self.add_data_dir(('sun', 'fun'))
self.add_data_dir(('gun', '/full/path/to/fun'))

Will install data-files to the locations:

<package install directory>/
  fun/
    foo.dat
    bar/
      car.dat
  sun/
    foo.dat
    bar/
      car.dat
  gun/
    foo.dat
    car.dat

add_include_dirs(*paths)

Add the given sequence of paths to the beginning of the include_dirs list. This list will be visible to all extension modules of the current package.

add_headers(*files)

Add the given sequence of files to the beginning of the headers list. By default, headers will be installed under <python- include>/<self.name.replace(‘.’,’/’)>/ directory. If an item of files is a tuple, then its first argument specifies the actual installation location relative to the <python-include> path.

add_extension(name, sources, **kw)

Create and add an Extension instance to the ext_modules list. The first argument defines the name of the extension module that will be installed under the self.name package. The second argument is a list of sources. This method also takes the following optional keyword arguments that are passed on to the Extension constructor: include_dirs, define_macros, undef_macros, library_dirs, libraries, runtime_library_dirs, extra_objects, swig_opts, depends, language, f2py_options, module_dirs, and extra_info.

The self.paths(...) method is applied to all lists that may contain paths. The extra_info is a dictionary or a list of dictionaries whose content will be appended to the keyword arguments. The depends list contains paths to files or directories that the sources of the extension module depend on. If any path in the depends list is newer than the extension module, then the module will be rebuilt.

The list of sources may contain functions (called source generators) which must take an extension instance and a build directory as inputs and return a source file or list of source files or None. If None is returned then no sources are generated. If the Extension instance has no sources after processing all source generators, then no extension module is built.

add_library(name, sources, **build_info)

Add a library to the list of libraries. Allowed keyword arguments are depends, macros, include_dirs, extra_compiler_args, and f2py_options. The name is the name of the library to be built and sources is a list of sources (or source generating functions) to add to the library.

add_scripts(*files)

Add the sequence of files to the beginning of the scripts list. Scripts will be installed under the <prefix>/bin/ directory.

paths(*paths)

Applies glob.glob(...) to each path in the sequence (if needed) and pre-pends the local_path if needed. Because this is called on all source lists, this allows wildcard characters to be specified in lists of sources for extension modules and libraries and scripts and allows path-names be relative to the source directory.

get_config_cmd()

Returns the numpy.distutils config command instance.

get_build_temp_dir()

Return a path to a temporary directory where temporary files should be placed.

have_f77c()

True if a Fortran 77 compiler is available (because a simple Fortran 77 code was able to be compiled successfully).

have_f90c()

True if a Fortran 90 compiler is available (because a simple Fortran 90 code was able to be compiled successfully)

get_version()

Return a version string of the current package or None if the version information could not be detected. This method scans files named __version__.py, <packagename>_version.py, version.py, and __svn_version__.py for string variables version, __version__, and <packagename>_version, until a version number is found.

make_svn_version_py()

Appends a data function to the data_files list that will generate __svn_version__.py file to the current package directory. This file will be removed from the source directory when Python exits (so that it can be re-generated next time the package is built). This is intended for working with source directories that are in an SVN repository.

make_config_py()

Generate a package __config__.py file containing system information used during the building of the package. This file is installed to the package installation directory.

get_info(*names)

Return information (from system_info.get_info) for all of the names in the argument list in a single dictionary.

Other modules

system_info.get_info (name[, notfound_action])	notfound_action: 0 - do nothing 1 - display warning message 2 - raise error
system_info.get_standard_file (fname)	Returns a list of files named ‘fname’ from 1) System-wide directory (directory-location of this module) 2) Users HOME directory (os.environ[‘HOME’]) 3) Local directory
cpuinfo.cpu
log.set_verbosity (v[, force])
exec_command	exec_command

Conversion of .src files

NumPy distutils supports automatic conversion of source files named <somefile>.src. This facility can be used to maintain very similar code blocks requiring only simple changes between blocks. During the build phase of setup, if a template file named <somefile>.src is encountered, a new file named <somefile> is constructed from the template and placed in the build directory to be used instead. Two forms of template conversion are supported. The first form occurs for files named named <file>.ext.src where ext is a recognized Fortran extension (f, f90, f95, f77, for, ftn, pyf). The second form is used for all other cases.

Fortran files

This template converter will replicate all function and subroutine blocks in the file with names that contain ‘<...>’ according to the rules in ‘<...>’. The number of comma-separated words in ‘<...>’ determines the number of times the block is repeated. What these words are indicates what that repeat rule, ‘<...>’, should be replaced with in each block. All of the repeat rules in a block must contain the same number of comma-separated words indicating the number of times that block should be repeated. If the word in the repeat rule needs a comma, leftarrow, or rightarrow, then prepend it with a backslash ‘ ‘. If a word in the repeat rule matches ‘ \<index>’ then it will be replaced with the <index>-th word in the same repeat specification. There are two forms for the repeat rule: named and short.

Named repeat rule

A named repeat rule is useful when the same set of repeats must be used several times in a block. It is specified using <rule1=item1, item2, item3,..., itemN>, where N is the number of times the block should be repeated. On each repeat of the block, the entire expression, ‘<...>’ will be replaced first with item1, and then with item2, and so forth until N repeats are accomplished. Once a named repeat specification has been introduced, the same repeat rule may be used in the current block by referring only to the name (i.e. <rule1>.

Short repeat rule

A short repeat rule looks like <item1, item2, item3, ..., itemN>. The rule specifies that the entire expression, ‘<...>’ should be replaced first with item1, and then with item2, and so forth until N repeats are accomplished.

Pre-defined names

The following predefined named repeat rules are available:

• <prefix=s,d,c,z>
• <_c=s,d,c,z>
• <_t=real, double precision, complex, double complex>
• <ftype=real, double precision, complex, double complex>
• <ctype=float, double, complex_float, complex_double>
• <ftypereal=float, double precision, \0, \1>
• <ctypereal=float, double, \0, \1>

Other files

Non-Fortran files use a separate syntax for defining template blocks that should be repeated using a variable expansion similar to the named repeat rules of the Fortran-specific repeats. The template rules for these files are:

1. “/**begin repeat “on a line by itself marks the beginning of a segment that should be repeated.
2. Named variable expansions are defined using #name=item1, item2, item3, ..., itemN# and placed on successive lines. These variables are replaced in each repeat block with corresponding word. All named variables in the same repeat block must define the same number of words.
3. In specifying the repeat rule for a named variable, item*N is short- hand for item, item, ..., item repeated N times. In addition, parenthesis in combination with *N can be used for grouping several items that should be repeated. Thus, #name=(item1, item2)*4# is equivalent to #name=item1, item2, item1, item2, item1, item2, item1, item2#
4. “*/ “on a line by itself marks the end of the the variable expansion naming. The next line is the first line that will be repeated using the named rules.
5. Inside the block to be repeated, the variables that should be expanded are specified as @name@.
6. “/**end repeat**/ “on a line by itself marks the previous line as the last line of the block to be repeated.