These macros all access the PyArrayObject structure members. The input argument, obj, can be any PyObject * that is directly interpretable as a PyArrayObject * (any instance of the PyArray_Type and its sub-types).
These functions and macros provide easy access to elements of the ndarray from C. These work for all arrays. You may need to take care when accessing the data in the array, however, if it is not in machine byte-order, misaligned, or not writeable. In other words, be sure to respect the state of the flags unless you know what you are doing, or have previously guaranteed an array that is writeable, aligned, and in machine byte-order using PyArray_FromAny. If you wish to handle all types of arrays, the copyswap function for each type is useful for handling misbehaved arrays. Some platforms (e.g. Solaris) do not like misaligned data and will crash if you de-reference a misaligned pointer. Other platforms (e.g. x86 Linux) will just work more slowly with misaligned data.
Warning
If data is passed to PyArray_NewFromDescr or PyArray_New, this memory must not be deallocated until the new array is deleted. If this data came from another Python object, this can be accomplished using Py_INCREF on that object and setting the base member of the new array to point to that object. If strides are passed in they must be consistent with the dimensions, the itemsize, and the data of the array.
This is the main function used to obtain an array from any nested sequence, or object that exposes the array interface, op. The parameters allow specification of the required type, the minimum (min_depth) and maximum (max_depth) number of dimensions acceptable, and other requirements for the array. The dtype argument needs to be a PyArray_Descr structure indicating the desired data-type (including required byteorder). The dtype argument may be NULL, indicating that any data-type (and byteorder) is acceptable. If you want to use NULL for the dtype and ensure the array is notswapped then use PyArray_CheckFromAny. A value of 0 for either of the depth parameters causes the parameter to be ignored. Any of the following array flags can be added (e.g. using |) to get the requirements argument. If your code can handle general (e.g. strided, byte-swapped, or unaligned arrays) then requirements may be 0. Also, if op is not already an array (or does not expose the array interface), then a new array will be created (and filled from op using the sequence protocol). The new array will have NPY_DEFAULT as its flags member. The context argument is passed to the __array__ method of op and is only used if the array is constructed that way.
For the typenum macros, the argument is an integer representing an enumerated array data type. For the array type checking macros the argument must be a PyObject * that can be directly interpreted as a PyArrayObject *.
Convert a sequence of Python objects contained in op to an array of ndarrays each having the same data type. The type is selected based on the typenumber (larger type number is chosen over a smaller one) ignoring objects that are only scalars. The length of the sequence is returned in n, and an n -length array of PyArrayObject pointers is the return value (or NULL if an error occurs). The returned array must be freed by the caller of this routine (using PyDataMem_FREE ) and all the array objects in it DECREF ‘d or a memory-leak will occur. The example template-code below shows a typically usage:
mps = PyArray_ConvertToCommonType(obj, &n);
if (mps==NULL) return NULL;
{code}
<before return>
for (i=0; i<n; i++) Py_DECREF(mps[i]);
PyDataMem_FREE(mps);
{return}
Register a data-type as a new user-defined data type for arrays. The type must have most of its entries filled in. This is not always checked and errors can produce segfaults. In particular, the typeobj member of the dtype structure must be filled with a Python type that has a fixed-size element-size that corresponds to the elsize member of dtype. Also the f member must have the required functions: nonzero, copyswap, copyswapn, getitem, setitem, and cast (some of the cast functions may be NULL if no support is desired). To avoid confusion, you should choose a unique character typecode but this is not enforced and not relied on internally.
A user-defined type number is returned that uniquely identifies the type. A pointer to the new structure can then be obtained from PyArray_DescrFromType using the returned type number. A -1 is returned if an error occurs. If this dtype has already been registered (checked only by the address of the pointer), then return the previously-assigned type-number.
An ndarray can have a data segment that is not a simple contiguous chunk of well-behaved memory you can manipulate. It may not be aligned with word boundaries (very important on some platforms). It might have its data in a different byte-order than the machine recognizes. It might not be writeable. It might be in Fortan-contiguous order. The array flags are used to indicate what can be said about data associated with an array.
The data area can be written to.
Notice that the above 3 flags are are defined so that a new, well- behaved array has these flags defined as true.
These constants are used in PyArray_FromAny (and its macro forms) to specify desired properties of the new array.
For all of these macros arr must be an instance of a (subclass of) PyArray_Type, but no checking is done.
Warning
It is important to keep the flags updated (using PyArray_UpdateFlags can help) whenever a manipulation with an array is performed that might cause them to change. Later calculations in NumPy that rely on the state of these flags do not repeat the calculation to update them.
Warning
matrix objects are always 2-dimensional. Therefore, PyArray_Squeeze has no effect on arrays of matrix sub-class.
Equivalent to ndarray.choose (self, op, ret, clipmode). Create a new array by selecting elements from the sequence of arrays in op based on the integer values in self. The arrays must all be broadcastable to the same shape and the entries in self should be between 0 and len(op). The output is placed in ret unless it is NULL in which case a new output is created. The clipmode argument determines behavior for when entries in self are not between 0 and len(op).
Given a sequence of arrays (sort_keys) of the same shape, return an array of indices (similar to PyArray_ArgSort (...)) that would sort the arrays lexicographically. A lexicographic sort specifies that when two keys are found to be equal, the order is based on comparison of subsequent keys. A merge sort (which leaves equal entries unmoved) is required to be defined for the types. The sort is accomplished by sorting the indices first using the first sort_key and then using the second sort_key and so forth. This is equivalent to the lexsort(sort_keys, axis) Python command. Because of the way the merge-sort works, be sure to understand the order the sort_keys must be in (reversed from the order you would use when comparing two elements).
If these arrays are all collected in a record array, then PyArray_Sort (...) can also be used to sort the array directly.
Tip
Pass in NPY_MAXDIMS for axis in order to achieve the same effect that is obtained by passing in axis = None in Python (treating the array as a 1-d array).
Note
The rtype argument specifies the data-type the reduction should take place over. This is important if the data-type of the array is not “large” enough to handle the output. By default, all integer data-types are made at least as large as NPY_LONG for the “add” and “multiply” ufuncs (which form the basis for mean, sum, cumsum, prod, and cumprod functions).
Sometimes it is useful to access a multidimensional array as a C-style multi-dimensional array so that algorithms can be implemented using C’s a[i][j][k] syntax. This routine returns a pointer, ptr, that simulates this kind of C-style array, for 1-, 2-, and 3-d ndarrays.
Parameters: |
|
---|
Note
The simulation of a C-style array is not complete for 2-d and 3-d arrays. For example, the simulated arrays of pointers cannot be passed to subroutines expecting specific, statically-defined 2-d and 3-d arrays. To pass to functions requiring those kind of inputs, you must statically define the required array and copy data.
An array iterator is a simple way to access the elements of an N-dimensional array quickly and efficiently. Section 2 provides more description and examples of this useful approach to looping over an array.
Warning
Data-type objects must be reference counted so be aware of the action on the data-type reference of different C-API calls. The standard rule is that when a data-type object is returned it is a new reference. Functions that take PyArray_Descr * objects and return arrays steal references to the data-type their inputs unless otherwise noted. Therefore, you must own a reference to any data-type object used as input to such a function.
All of these functions can be used in PyArg_ParseTuple (...) with the “O&” format specifier to automatically convert any Python object to the required C-object. All of these functions return NPY_SUCCEED if successful and NPY_FAIL if not. The first argument to all of these function is a Python object. The second argument is the address of the C-type to convert the Python object to.
Warning
Be sure to understand what steps you should take to manage the memory when using these conversion functions. These functions can require freeing memory, and/or altering the reference counts of specific objects based on your use.
Convert all kinds of Python objects (including arrays and array scalars) to a standard integer. On error, -1 is returned and an exception set. You may find useful the macro:
#define error_converting(x) (((x) == -1) && PyErr_Occurred()
In order to make use of the C-API from another extension module, the import_array () command must be used. If the extension module is self-contained in a single .c file, then that is all that needs to be done. If, however, the extension module involves multiple files where the C-API is needed then some additional steps must be taken.
Using these #defines you can use the C-API in multiple files for a single extension module. In each file you must define PY_ARRAY_UNIQUE_SYMBOL to some name that will hold the C-API (e.g. myextension_ARRAY_API). This must be done before including the numpy/arrayobject.h file. In the module intialization routine you call import_array (). In addition, in the files that do not have the module initialization sub_routine define NO_IMPORT_ARRAY prior to including numpy/arrayobject.h.
Suppose I have two files coolmodule.c and coolhelper.c which need to be compiled and linked into a single extension module. Suppose coolmodule.c contains the required initcool module initialization function (with the import_array() function called). Then, coolmodule.c would have at the top:
#define PY_ARRAY_UNIQUE_SYMBOL cool_ARRAY_API
#include numpy/arrayobject.h
On the other hand, coolhelper.c would contain at the top:
#define PY_ARRAY_UNIQUE_SYMBOL cool_ARRAY_API
#define NO_IMPORT_ARRAY
#include numpy/arrayobject.h
NumPy stores an internal table of Python callable objects that are used to implement arithmetic operations for arrays as well as certain array calculation methods. This function allows the user to replace any or all of these Python objects with their own versions. The keys of the dictionary, dict, are the named functions to replace and the paired value is the Python callable object to use. Care should be taken that the function used to replace an internal array operation does not itself call back to that internal array operation (unless you have designed the function to handle that), or an unchecked infinite recursion can result (possibly causing program crash). The key names that represent operations that can be replaced are:
add, subtract, multiply, divide, remainder, power, square, reciprocal, ones_like, sqrt, negative, absolute, invert, left_shift, right_shift, bitwise_and, bitwise_xor, bitwise_or, less, less_equal, equal, not_equal, greater, greater_equal, floor_divide, true_divide, logical_or, logical_and, floor, ceil, maximum, minimum, rint.
These functions are included here because they are used at least once in the array object’s methods. The function returns -1 (without setting a Python Error) if one of the objects being assigned is not callable.
These macros are only meaningful if NPY_ALLOW_THREADS evaluates True during compilation of the extension module. Otherwise, these macros are equivalent to whitespace. Python uses a single Global Interpreter Lock (GIL) for each Python process so that only a single thread may excecute at a time (even on multi-cpu machines). When calling out to a compiled function that may take time to compute (and does not have side-effects for other threads like updated global variables), the GIL should be released so that other Python threads can run while the time-consuming calculations are performed. This can be accomplished using two groups of macros. Typically, if one macro in a group is used in a code block, all of them must be used in the same code block. Currently, NPY_ALLOW_THREADS is defined to the python-defined WITH_THREADS constant unless the environment variable NPY_NOSMP is set in which case NPY_ALLOW_THREADS is defined to be 0.
This group is used to call code that may take some time but does not use any Python C-API calls. Thus, the GIL should be released during its calculation.
- NPY_BEGIN_ALLOW_THREADS¶
- Equivalent to Py_BEGIN_ALLOW_THREADS except it uses NPY_ALLOW_THREADS to determine if the macro if replaced with white-space or not.
- NPY_END_ALLOW_THREADS¶
- Equivalent to Py_END_ALLOW_THREADS except it uses NPY_ALLOW_THREADS to determine if the macro if replaced with white-space or not.
- NPY_BEGIN_THREADS_DEF¶
- Place in the variable declaration area. This macro sets up the variable needed for storing the Python state.
- NPY_BEGIN_THREADS¶
- Place right before code that does not need the Python interpreter (no Python C-API calls). This macro saves the Python state and releases the GIL.
- NPY_END_THREADS¶
- Place right after code that does not need the Python interpreter. This macro acquires the GIL and restores the Python state from the saved variable.
- NPY_BEGIN_THREADS_DESCR(PyArray_Descr *dtype)¶
- Useful to release the GIL only if dtype does not contain arbitrary Python objects which may need the Python interpreter during execution of the loop. Equivalent to
- NPY_END_THREADS_DESCR(PyArray_Descr *dtype)¶
- Useful to regain the GIL in situations where it was released using the BEGIN form of this macro.
This group is used to re-acquire the Python GIL after it has been released. For example, suppose the GIL has been released (using the previous calls), and then some path in the code (perhaps in a different subroutine) requires use of the Python C-API, then these macros are useful to acquire the GIL. These macros accomplish essentially a reverse of the previous three (acquire the LOCK saving what state it had) and then re-release it with the saved state.
- NPY_ALLOW_C_API_DEF¶
- Place in the variable declaration area to set up the necessary variable.
- NPY_ALLOW_C_API¶
- Place before code that needs to call the Python C-API (when it is known that the GIL has already been released).
- NPY_DISABLE_C_API¶
- Place after code that needs to call the Python C-API (to re-release the GIL).
Tip
Never use semicolons after the threading support macros.
A special variable-type which can take on the values NPY_{KIND} where {KIND} is
QUICKSORT, HEAPSORT, MERGESORT
A special variable type indicating the number of “kinds” of scalars distinguished in determining scalar-coercion rules. This variable can take on the values NPY_{KIND} where {KIND} can be
NOSCALAR, BOOL_SCALAR, INTPOS_SCALAR, INTNEG_SCALAR, FLOAT_SCALAR, COMPLEX_SCALAR, OBJECT_SCALAR
A variable type indicating the order that an array should be interpreted in. The value of a variable of this type can be NPY_{ORDER} where {ORDER} is
ANYORDER, CORDER, FORTRANORDER
A variable type indicating the kind of clipping that should be applied in certain functions. The value of a variable of this type can be NPY_{MODE} where {MODE} is
CLIP, WRAP, RAISE