SciPy

Release Notes

NumPy 1.9.0 Release Notes

This release supports Python 2.6 -2.7 and 3.2 - 3.3.

Highlights

Dropped Support

The oldnumeric and numarray modules have been removed.

Future Changes

Compatibility notes

Special scalar float values don’t cause upcast to double anymore

In previous numpy versions operations involving floating point scalars containing special values NaN, Inf and -Inf caused the result type to be at least float64. As the special values can be represented in the smallest available floating point type the upcast not performed anymore.

For example the dtype of:

np.array([1.], dtype=np.float32) * float(‘nan’)

now remains float32 instead of being cast to float64. Operations involving non-special values have not been changed.

Percentile output changes

If given more than one percentile to compute numpy.percentile returns an array instead of a list. A single percentile still returns a scalar. The array is equivalent to converting the the list returned in older versions to an array via np.array.

If the overwrite_input option is used the input is only partially instead of fully sorted.

npy_3kcompat.h header change

The unused simple_capsule_dtor function has been removed from npy_3kcompat.h. Note that this header is not meant to be used outside of numpy; other projects should be using their own copy of this file when needed.

New Features

Percentile supports more interpolation options

np.percentile now has the interpolation keyword argument to specify in which way points should be interpolated if the percentiles fall between two values. See the documentation for the available options.

Ufunc and Dot Overrides

For better compatibility with external objects you can now override universal functions (ufuncs), numpy.core._dotblas.dot, and numpy.core.multiarray.dot (the numpy.dot functions). By defining a __numpy_ufunc__ method.

Dtype parameter added to np.linspace and np.logspace

The returned data type from the linspace and logspace functions can now be specificed using the dtype parameter.

Improvements

Percentile implemented in terms of np.partition

np.percentile has been implemented interms of np.partition which only partially sorts the data via a selection algorithm. This improves the time complexcity from O(nlog(n)) to O(n).

Performance improvement for np.array

The performance of converting lists containing arrays to arrays using np.array has been improved. It is now equivalent in speed to np.vstack(list).

Changes

Argmin and argmax out argument

The out argument to np.argmin and np.argmax and their equivalent C-API functions is now checked to match the desired output shape exactly. If the check fails a ValueError instead of TypeError is raised.

Einsum

Remove unnecessary broadcasting notation restrictions. np.einsum(‘ijk,j->ijk’, A, B) can also be written as np.einsum(‘ij...,j->ij...’, A, B) (ellipsis is no longer required on ‘j’)

C-API

None

Deprecations

Non-integer scalars for sequence repetition

Using non-integer numpy scalars to repeat python sequences is deprecated. For example np.float_(2) * [1] will be an error in the future.

C-API

None

New Features

NumPy 1.8.0 Release Notes

This release supports Python 2.6 -2.7 and 3.2 - 3.3.

Highlights

  • New, no 2to3, Python 2 and Python 3 are supported by a common code base.
  • New, gufuncs for linear algebra, enabling operations on stacked arrays.
  • New, inplace fancy indexing for ufuncs with the .at method.
  • New, partition function, partial sorting via selection for fast median.
  • New, nanmean, nanvar, and nanstd functions skipping NaNs.
  • New, full and full_like functions to create value initialized arrays.
  • New, PyUFunc_RegisterLoopForDescr, better ufunc support for user dtypes.
  • Numerous performance improvements in many areas.

Dropped Support

Support for Python versions 2.4 and 2.5 has been dropped,

Support for SCons has been removed.

Future Changes

The Datetime64 type remains experimental in this release. In 1.9 there will probably be some changes to make it more useable.

The diagonal method currently returns a new array and raises a FutureWarning. In 1.9 it will return a readonly view.

Multiple field selection from a array of structured type currently returns a new array and raises a FutureWarning. In 1.9 it will return a readonly view.

The numpy/oldnumeric and numpy/numarray compatibility modules will be removed in 1.9.

Compatibility notes

The doc/sphinxext content has been moved into its own github repository, and is included in numpy as a submodule. See the instructions in doc/HOWTO_BUILD_DOCS.rst.txt for how to access the content.

The hash function of numpy.void scalars has been changed. Previously the pointer to the data was hashed as an integer. Now, the hash function uses the tuple-hash algorithm to combine the hash functions of the elements of the scalar, but only if the scalar is read-only.

Numpy has switched its build system to using ‘separate compilation’ by default. In previous releases this was supported, but not default. This should produce the same results as the old system, but if you’re trying to do something complicated like link numpy statically or using an unusual compiler, then it’s possible you will encounter problems. If so, please file a bug and as a temporary workaround you can re-enable the old build system by exporting the shell variable NPY_SEPARATE_COMPILATION=0.

For the AdvancedNew iterator the oa_ndim flag should now be -1 to indicate that no op_axes and itershape are passed in. The oa_ndim == 0 case, now indicates a 0-D iteration and op_axes being NULL and the old usage is deprecated. This does not effect the NpyIter_New or NpyIter_MultiNew functions.

The functions nanargmin and nanargmax now return np.iinfo[‘intp’].min for the index in all-NaN slices. Previously the functions would raise a ValueError for array returns and NaN for scalar returns.

NPY_RELAXED_STRIDES_CHECKING

There is a new compile time environment variable NPY_RELAXED_STRIDES_CHECKING. If this variable is set to 1, then numpy will consider more arrays to be C- or F-contiguous – for example, it becomes possible to have a column vector which is considered both C- and F-contiguous simultaneously. The new definition is more accurate, allows for faster code that makes fewer unnecessary copies, and simplifies numpy’s code internally. However, it may also break third-party libraries that make too-strong assumptions about the stride values of C- and F-contiguous arrays. (It is also currently known that this breaks Cython code using memoryviews, which will be fixed in Cython.) THIS WILL BECOME THE DEFAULT IN A FUTURE RELEASE, SO PLEASE TEST YOUR CODE NOW AGAINST NUMPY BUILT WITH:

NPY_RELAXED_STRIDES_CHECKING=1 python setup.py install

You can check whether NPY_RELAXED_STRIDES_CHECKING is in effect by running:

np.ones((10, 1), order="C").flags.f_contiguous

This will be True if relaxed strides checking is enabled, and False otherwise. The typical problem we’ve seen so far is C code that works with C-contiguous arrays, and assumes that the itemsize can be accessed by looking at the last element in the PyArray_STRIDES(arr) array. When relaxed strides are in effect, this is not true (and in fact, it never was true in some corner cases). Instead, use PyArray_ITEMSIZE(arr).

For more information check the “Internal memory layout of an ndarray” section in the documentation.

Binary operations with non-arrays as second argument

Binary operations of the form <array-or-subclass> * <non-array-subclass> where <non-array-subclass> declares an __array_priority__ higher than that of <array-or-subclass> will now unconditionally return NotImplemented, giving <non-array-subclass> a chance to handle the operation. Previously, NotImplemented would only be returned if <non-array-subclass> actually implemented the reversed operation, and after a (potentially expensive) array conversion of <non-array-subclass> had been attempted. (bug, pull request)

Function median used with overwrite_input only partially sorts array

If median is used with overwrite_input option the input array will now only be partially sorted instead of fully sorted.

Fix to financial.npv

The npv function had a bug. Contrary to what the documentation stated, it summed from indexes 1 to M instead of from 0 to M - 1. The fix changes the returned value. The mirr function called the npv function, but worked around the problem, so that was also fixed and the return value of the mirr function remains unchanged.

Runtime warnings when comparing NaN numbers

Comparing NaN floating point numbers now raises the invalid runtime warning. If a NaN is expected the warning can be ignored using np.errstate. E.g.:

with np.errstate(invalid='ignore'):
    operation()

New Features

Support for linear algebra on stacked arrays

The gufunc machinery is now used for np.linalg, allowing operations on stacked arrays and vectors. For example:

>>> a
array([[[ 1.,  1.],
        [ 0.,  1.]],

       [[ 1.,  1.],
        [ 0.,  1.]]])

>>> np.linalg.inv(a)
array([[[ 1., -1.],
        [ 0.,  1.]],

       [[ 1., -1.],
        [ 0.,  1.]]])

In place fancy indexing for ufuncs

The function at has been added to ufunc objects to allow in place ufuncs with no buffering when fancy indexing is used. For example, the following will increment the first and second items in the array, and will increment the third item twice: numpy.add.at(arr, [0, 1, 2, 2], 1)

This is what many have mistakenly thought arr[[0, 1, 2, 2]] += 1 would do, but that does not work as the incremented value of arr[2] is simply copied into the third slot in arr twice, not incremented twice.

New functions partition and argpartition

New functions to partially sort arrays via a selection algorithm.

A partition by index k moves the k smallest element to the front of an array. All elements before k are then smaller or equal than the value in position k and all elements following k are then greater or equal than the value in position k. The ordering of the values within these bounds is undefined. A sequence of indices can be provided to sort all of them into their sorted position at once iterative partitioning. This can be used to efficiently obtain order statistics like median or percentiles of samples. partition has a linear time complexity of O(n) while a full sort has O(n log(n)).

New functions nanmean, nanvar and nanstd

New nan aware statistical functions are added. In these functions the results are what would be obtained if nan values were ommited from all computations.

New functions full and full_like

New convenience functions to create arrays filled with a specific value; complementary to the existing zeros and zeros_like functions.

IO compatibility with large files

Large NPZ files >2GB can be loaded on 64-bit systems.

Building against OpenBLAS

It is now possible to build numpy against OpenBLAS by editing site.cfg.

New constant

Euler’s constant is now exposed in numpy as euler_gamma.

New modes for qr

New modes ‘complete’, ‘reduced’, and ‘raw’ have been added to the qr factorization and the old ‘full’ and ‘economic’ modes are deprecated. The ‘reduced’ mode replaces the old ‘full’ mode and is the default as was the ‘full’ mode, so backward compatibility can be maintained by not specifying the mode.

The ‘complete’ mode returns a full dimensional factorization, which can be useful for obtaining a basis for the orthogonal complement of the range space. The ‘raw’ mode returns arrays that contain the Householder reflectors and scaling factors that can be used in the future to apply q without needing to convert to a matrix. The ‘economic’ mode is simply deprecated, there isn’t much use for it and it isn’t any more efficient than the ‘raw’ mode.

New invert argument to in1d

The function in1d now accepts a invert argument which, when True, causes the returned array to be inverted.

Advanced indexing using np.newaxis

It is now possible to use np.newaxis/None together with index arrays instead of only in simple indices. This means that array[np.newaxis, [0, 1]] will now work as expected and select the first two rows while prepending a new axis to the array.

C-API

New ufuncs can now be registered with builtin input types and a custom output type. Before this change, NumPy wouldn’t be able to find the right ufunc loop function when the ufunc was called from Python, because the ufunc loop signature matching logic wasn’t looking at the output operand type. Now the correct ufunc loop is found, as long as the user provides an output argument with the correct output type.

runtests.py

A simple test runner script runtests.py was added. It also builds Numpy via setup.py build and can be used to run tests easily during development.

Improvements

IO performance improvements

Performance in reading large files was improved by chunking (see also IO compatibility).

Performance improvements to pad

The pad function has a new implementation, greatly improving performance for all inputs except mode= (retained for backwards compatibility). Scaling with dimensionality is dramatically improved for rank >= 4.

Performance improvements to isnan, isinf, isfinite and byteswap

isnan, isinf, isfinite and byteswap have been improved to take advantage of compiler builtins to avoid expensive calls to libc. This improves performance of these operations by about a factor of two on gnu libc systems.

Performance improvements via SSE2 vectorization

Several functions have been optimized to make use of SSE2 CPU SIMD instructions.

  • Float32 and float64:
    • base math (add, subtract, divide, multiply)
    • sqrt
    • minimum/maximum
    • absolute
  • Bool:
    • logical_or
    • logical_and
    • logical_not

This improves performance of these operations up to 4x/2x for float32/float64 and up to 10x for bool depending on the location of the data in the CPU caches. The performance gain is greatest for in-place operations.

In order to use the improved functions the SSE2 instruction set must be enabled at compile time. It is enabled by default on x86_64 systems. On x86_32 with a capable CPU it must be enabled by passing the appropriate flag to the CFLAGS build variable (-msse2 with gcc).

Performance improvements to median

median is now implemented in terms of partition instead of sort which reduces its time complexity from O(n log(n)) to O(n). If used with the overwrite_input option the array will now only be partially sorted instead of fully sorted.

Overrideable operand flags in ufunc C-API

When creating a ufunc, the default ufunc operand flags can be overridden via the new op_flags attribute of the ufunc object. For example, to set the operand flag for the first input to read/write:

PyObject *ufunc = PyUFunc_FromFuncAndData(...); ufunc->op_flags[0] = NPY_ITER_READWRITE;

This allows a ufunc to perform an operation in place. Also, global nditer flags can be overridden via the new iter_flags attribute of the ufunc object. For example, to set the reduce flag for a ufunc:

ufunc->iter_flags = NPY_ITER_REDUCE_OK;

Changes

General

The function np.take now allows 0-d arrays as indices.

The separate compilation mode is now enabled by default.

Several changes to np.insert and np.delete:

  • Previously, negative indices and indices that pointed past the end of the array were simply ignored. Now, this will raise a Future or Deprecation Warning. In the future they will be treated like normal indexing treats them – negative indices will wrap around, and out-of-bound indices will generate an error.
  • Previously, boolean indices were treated as if they were integers (always referring to either the 0th or 1st item in the array). In the future, they will be treated as masks. In this release, they raise a FutureWarning warning of this coming change.
  • In Numpy 1.7. np.insert already allowed the syntax np.insert(arr, 3, [1,2,3]) to insert multiple items at a single position. In Numpy 1.8. this is also possible for np.insert(arr, [3], [1, 2, 3]).

Padded regions from np.pad are now correctly rounded, not truncated.

C-API Array Additions

Four new functions have been added to the array C-API.

  • PyArray_Partition
  • PyArray_ArgPartition
  • PyArray_SelectkindConverter
  • PyDataMem_NEW_ZEROED

C-API Ufunc Additions

One new function has been added to the ufunc C-API that allows to register an inner loop for user types using the descr.

  • PyUFunc_RegisterLoopForDescr

Deprecations

The ‘full’ and ‘economic’ modes of qr factorization are deprecated.

General

The use of non-integer for indices and most integer arguments has been deprecated. Previously float indices and function arguments such as axes or shapes were truncated to integers without warning. For example arr.reshape(3., -1) or arr[0.] will trigger a deprecation warning in NumPy 1.8., and in some future version of NumPy they will raise an error.

Authors

This release contains work by the following people who contributed at least one patch to this release. The names are in alphabetical order by first name:

  • 87
  • Adam Ginsburg +
  • Adam Griffiths +
  • Alexander Belopolsky +
  • Alex Barth +
  • Alex Ford +
  • Andreas Hilboll +
  • Andreas Kloeckner +
  • Andreas Schwab +
  • Andrew Horton +
  • argriffing +
  • Arink Verma +
  • Bago Amirbekian +
  • Bartosz Telenczuk +
  • bebert218 +
  • Benjamin Root +
  • Bill Spotz +
  • Bradley M. Froehle
  • Carwyn Pelley +
  • Charles Harris
  • Chris
  • Christian Brueffer +
  • Christoph Dann +
  • Christoph Gohlke
  • Dan Hipschman +
  • Daniel +
  • Dan Miller +
  • daveydave400 +
  • David Cournapeau
  • David Warde-Farley
  • Denis Laxalde
  • dmuellner +
  • Edward Catmur +
  • Egor Zindy +
  • endolith
  • Eric Firing
  • Eric Fode
  • Eric Moore +
  • Eric Price +
  • Fazlul Shahriar +
  • Félix Hartmann +
  • Fernando Perez
  • Frank B +
  • Frank Breitling +
  • Frederic
  • Gabriel
  • GaelVaroquaux
  • Guillaume Gay +
  • Han Genuit
  • HaroldMills +
  • hklemm +
  • jamestwebber +
  • Jason Madden +
  • Jay Bourque
  • jeromekelleher +
  • Jesús Gómez +
  • jmozmoz +
  • jnothman +
  • Johannes Schönberger +
  • John Benediktsson +
  • John Salvatier +
  • John Stechschulte +
  • Jonathan Waltman +
  • Joon Ro +
  • Jos de Kloe +
  • Joseph Martinot-Lagarde +
  • Josh Warner (Mac) +
  • Jostein Bø Fløystad +
  • Juan Luis Cano Rodríguez +
  • Julian Taylor +
  • Julien Phalip +
  • K.-Michael Aye +
  • Kumar Appaiah +
  • Lars Buitinck
  • Leon Weber +
  • Luis Pedro Coelho
  • Marcin Juszkiewicz
  • Mark Wiebe
  • Marten van Kerkwijk +
  • Martin Baeuml +
  • Martin Spacek
  • Martin Teichmann +
  • Matt Davis +
  • Matthew Brett
  • Maximilian Albert +
  • m-d-w +
  • Michael Droettboom
  • mwtoews +
  • Nathaniel J. Smith
  • Nicolas Scheffer +
  • Nils Werner +
  • ochoadavid +
  • Ondřej Čertík
  • ovillellas +
  • Paul Ivanov
  • Pauli Virtanen
  • peterjc
  • Ralf Gommers
  • Raul Cota +
  • Richard Hattersley +
  • Robert Costa +
  • Robert Kern
  • Rob Ruana +
  • Ronan Lamy
  • Sandro Tosi
  • Sascha Peilicke +
  • Sebastian Berg
  • Skipper Seabold
  • Stefan van der Walt
  • Steve +
  • Takafumi Arakaki +
  • Thomas Robitaille +
  • Tomas Tomecek +
  • Travis E. Oliphant
  • Valentin Haenel
  • Vladimir Rutsky +
  • Warren Weckesser
  • Yaroslav Halchenko
  • Yury V. Zaytsev +

A total of 119 people contributed to this release. People with a “+” by their names contributed a patch for the first time.

NumPy 1.7.2 Release Notes

This is a bugfix only release in the 1.7.x series. It supports Python 2.4 - 2.7 and 3.1 - 3.3 and is the last series that supports Python 2.4 - 2.5.

Issues fixed

  • gh-3153: Do not reuse nditer buffers when not filled enough
  • gh-3192: f2py crashes with UnboundLocalError exception
  • gh-442: Concatenate with axis=None now requires equal number of array elements
  • gh-2485: Fix for astype(‘S’) string truncate issue
  • gh-3312: bug in count_nonzero
  • gh-2684: numpy.ma.average casts complex to float under certain conditions
  • gh-2403: masked array with named components does not behave as expected
  • gh-2495: np.ma.compress treated inputs in wrong order
  • gh-576: add __len__ method to ma.mvoid
  • gh-3364: reduce performance regression of mmap slicing
  • gh-3421: fix non-swapping strided copies in GetStridedCopySwap
  • gh-3373: fix small leak in datetime metadata initialization
  • gh-2791: add platform specific python include directories to search paths
  • gh-3168: fix undefined function and add integer divisions
  • gh-3301: memmap does not work with TemporaryFile in python3
  • gh-3057: distutils.misc_util.get_shared_lib_extension returns wrong debug extension
  • gh-3472: add module extensions to load_library search list
  • gh-3324: Make comparison function (gt, ge, ...) respect __array_priority__
  • gh-3497: np.insert behaves incorrectly with argument ‘axis=-1’
  • gh-3541: make preprocessor tests consistent in halffloat.c
  • gh-3458: array_ass_boolean_subscript() writes ‘non-existent’ data to array
  • gh-2892: Regression in ufunc.reduceat with zero-sized index array
  • gh-3608: Regression when filling struct from tuple
  • gh-3701: add support for Python 3.4 ast.NameConstant
  • gh-3712: do not assume that GIL is enabled in xerbla
  • gh-3712: fix LAPACK error handling in lapack_litemodule
  • gh-3728: f2py fix decref on wrong object
  • gh-3743: Hash changed signature in Python 3.3
  • gh-3793: scalar int hashing broken on 64 bit python3
  • gh-3160: SandboxViolation easyinstalling 1.7.0 on Mac OS X 10.8.3
  • gh-3871: npy_math.h has invalid isinf for Solaris with SUNWspro12.2
  • gh-2561: Disable check for oldstyle classes in python3
  • gh-3900: Ensure NotImplemented is passed on in MaskedArray ufunc’s
  • gh-2052: del scalar subscript causes segfault
  • gh-3832: fix a few uninitialized uses and memleaks
  • gh-3971: f2py changed string.lowercase to string.ascii_lowercase for python3
  • gh-3480: numpy.random.binomial raised ValueError for n == 0
  • gh-3992: hypot(inf, 0) shouldn’t raise a warning, hypot(inf, inf) wrong result
  • gh-4018: Segmentation fault dealing with very large arrays
  • gh-4094: fix NaT handling in _strided_to_strided_string_to_datetime
  • gh-4051: fix uninitialized use in _strided_to_strided_string_to_datetime
  • gh-4123: lexsort segfault
  • gh-4141: Fix a few issues that show up with python 3.4b1

NumPy 1.7.1 Release Notes

This is a bugfix only release in the 1.7.x series. It supports Python 2.4 - 2.7 and 3.1 - 3.3 and is the last series that supports Python 2.4 - 2.5.

Issues fixed

  • gh-2973: Fix 1 is printed during numpy.test()
  • gh-2983: BUG: gh-2969: Backport memory leak fix 80b3a34.
  • gh-3007: Backport gh-3006
  • gh-2984: Backport fix complex polynomial fit
  • gh-2982: BUG: Make nansum work with booleans.
  • gh-2985: Backport large sort fixes
  • gh-3039: Backport object take
  • gh-3105: Backport nditer fix op axes initialization
  • gh-3108: BUG: npy-pkg-config ini files were missing after Bento build.
  • gh-3124: BUG: PyArray_LexSort allocates too much temporary memory.
  • gh-3131: BUG: Exported f2py_size symbol prevents linking multiple f2py modules.
  • gh-3117: Backport gh-2992
  • gh-3135: DOC: Add mention of PyArray_SetBaseObject stealing a reference
  • gh-3134: DOC: Fix typo in fft docs (the indexing variable is ‘m’, not ‘n’).
  • gh-3136: Backport #3128

NumPy 1.7.0 Release Notes

This release includes several new features as well as numerous bug fixes and refactorings. It supports Python 2.4 - 2.7 and 3.1 - 3.3 and is the last release that supports Python 2.4 - 2.5.

Highlights

  • where= parameter to ufuncs (allows the use of boolean arrays to choose where a computation should be done)
  • vectorize improvements (added ‘excluded’ and ‘cache’ keyword, general cleanup and bug fixes)
  • numpy.random.choice (random sample generating function)

Compatibility notes

In a future version of numpy, the functions np.diag, np.diagonal, and the diagonal method of ndarrays will return a view onto the original array, instead of producing a copy as they do now. This makes a difference if you write to the array returned by any of these functions. To facilitate this transition, numpy 1.7 produces a FutureWarning if it detects that you may be attempting to write to such an array. See the documentation for np.diagonal for details.

Similar to np.diagonal above, in a future version of numpy, indexing a record array by a list of field names will return a view onto the original array, instead of producing a copy as they do now. As with np.diagonal, numpy 1.7 produces a FutureWarning if it detects that you may be attempting to write to such an array. See the documentation for array indexing for details.

In a future version of numpy, the default casting rule for UFunc out= parameters will be changed from ‘unsafe’ to ‘same_kind’. (This also applies to in-place operations like a += b, which is equivalent to np.add(a, b, out=a).) Most usages which violate the ‘same_kind’ rule are likely bugs, so this change may expose previously undetected errors in projects that depend on NumPy. In this version of numpy, such usages will continue to succeed, but will raise a DeprecationWarning.

Full-array boolean indexing has been optimized to use a different, optimized code path. This code path should produce the same results, but any feedback about changes to your code would be appreciated.

Attempting to write to a read-only array (one with arr.flags.writeable set to False) used to raise either a RuntimeError, ValueError, or TypeError inconsistently, depending on which code path was taken. It now consistently raises a ValueError.

The <ufunc>.reduce functions evaluate some reductions in a different order than in previous versions of NumPy, generally providing higher performance. Because of the nature of floating-point arithmetic, this may subtly change some results, just as linking NumPy to a different BLAS implementations such as MKL can.

If upgrading from 1.5, then generally in 1.6 and 1.7 there have been substantial code added and some code paths altered, particularly in the areas of type resolution and buffered iteration over universal functions. This might have an impact on your code particularly if you relied on accidental behavior in the past.

New features

Reduction UFuncs Generalize axis= Parameter

Any ufunc.reduce function call, as well as other reductions like sum, prod, any, all, max and min support the ability to choose a subset of the axes to reduce over. Previously, one could say axis=None to mean all the axes or axis=# to pick a single axis. Now, one can also say axis=(#,#) to pick a list of axes for reduction.

Reduction UFuncs New keepdims= Parameter

There is a new keepdims= parameter, which if set to True, doesn’t throw away the reduction axes but instead sets them to have size one. When this option is set, the reduction result will broadcast correctly to the original operand which was reduced.

Datetime support

Note

The datetime API is experimental in 1.7.0, and may undergo changes in future versions of NumPy.

There have been a lot of fixes and enhancements to datetime64 compared to NumPy 1.6:

  • the parser is quite strict about only accepting ISO 8601 dates, with a few convenience extensions
  • converts between units correctly
  • datetime arithmetic works correctly
  • business day functionality (allows the datetime to be used in contexts where only certain days of the week are valid)

The notes in doc/source/reference/arrays.datetime.rst (also available in the online docs at arrays.datetime.html) should be consulted for more details.

Custom formatter for printing arrays

See the new formatter parameter of the numpy.set_printoptions function.

New function numpy.random.choice

A generic sampling function has been added which will generate samples from a given array-like. The samples can be with or without replacement, and with uniform or given non-uniform probabilities.

New function isclose

Returns a boolean array where two arrays are element-wise equal within a tolerance. Both relative and absolute tolerance can be specified.

Preliminary multi-dimensional support in the polynomial package

Axis keywords have been added to the integration and differentiation functions and a tensor keyword was added to the evaluation functions. These additions allow multi-dimensional coefficient arrays to be used in those functions. New functions for evaluating 2-D and 3-D coefficient arrays on grids or sets of points were added together with 2-D and 3-D pseudo-Vandermonde matrices that can be used for fitting.

Ability to pad rank-n arrays

A pad module containing functions for padding n-dimensional arrays has been added. The various private padding functions are exposed as options to a public ‘pad’ function. Example:

pad(a, 5, mode='mean')

Current modes are constant, edge, linear_ramp, maximum, mean, median, minimum, reflect, symmetric, wrap, and <function>.

New argument to searchsorted

The function searchsorted now accepts a ‘sorter’ argument that is a permutation array that sorts the array to search.

Build system

Added experimental support for the AArch64 architecture.

C API

New function PyArray_RequireWriteable provides a consistent interface for checking array writeability – any C code which works with arrays whose WRITEABLE flag is not known to be True a priori, should make sure to call this function before writing.

NumPy C Style Guide added (doc/C_STYLE_GUIDE.rst.txt).

Changes

General

The function np.concatenate tries to match the layout of its input arrays. Previously, the layout did not follow any particular reason, and depended in an undesirable way on the particular axis chosen for concatenation. A bug was also fixed which silently allowed out of bounds axis arguments.

The ufuncs logical_or, logical_and, and logical_not now follow Python’s behavior with object arrays, instead of trying to call methods on the objects. For example the expression (3 and ‘test’) produces the string ‘test’, and now np.logical_and(np.array(3, ‘O’), np.array(‘test’, ‘O’)) produces ‘test’ as well.

The .base attribute on ndarrays, which is used on views to ensure that the underlying array owning the memory is not deallocated prematurely, now collapses out references when you have a view-of-a-view. For example:

a = np.arange(10)
b = a[1:]
c = b[1:]

In numpy 1.6, c.base is b, and c.base.base is a. In numpy 1.7, c.base is a.

To increase backwards compatibility for software which relies on the old behaviour of .base, we only ‘skip over’ objects which have exactly the same type as the newly created view. This makes a difference if you use ndarray subclasses. For example, if we have a mix of ndarray and matrix objects which are all views on the same original ndarray:

a = np.arange(10)
b = np.asmatrix(a)
c = b[0, 1:]
d = c[0, 1:]

then d.base will be b. This is because d is a matrix object, and so the collapsing process only continues so long as it encounters other matrix objects. It considers c, b, and a in that order, and b is the last entry in that list which is a matrix object.

Casting Rules

Casting rules have undergone some changes in corner cases, due to the NA-related work. In particular for combinations of scalar+scalar:

  • the longlong type (q) now stays longlong for operations with any other number (? b h i l q p B H I), previously it was cast as int_ (l). The ulonglong type (Q) now stays as ulonglong instead of uint (L).
  • the timedelta64 type (m) can now be mixed with any integer type (b h i l q p B H I L Q P), previously it raised TypeError.

For array + scalar, the above rules just broadcast except the case when the array and scalars are unsigned/signed integers, then the result gets converted to the array type (of possibly larger size) as illustrated by the following examples:

>>> (np.zeros((2,), dtype=np.uint8) + np.int16(257)).dtype
dtype('uint16')
>>> (np.zeros((2,), dtype=np.int8) + np.uint16(257)).dtype
dtype('int16')
>>> (np.zeros((2,), dtype=np.int16) + np.uint32(2**17)).dtype
dtype('int32')

Whether the size gets increased depends on the size of the scalar, for example:

>>> (np.zeros((2,), dtype=np.uint8) + np.int16(255)).dtype
dtype('uint8')
>>> (np.zeros((2,), dtype=np.uint8) + np.int16(256)).dtype
dtype('uint16')

Also a complex128 scalar + float32 array is cast to complex64.

In NumPy 1.7 the datetime64 type (M) must be constructed by explicitly specifying the type as the second argument (e.g. np.datetime64(2000, 'Y')).

Deprecations

General

Specifying a custom string formatter with a _format array attribute is deprecated. The new formatter keyword in numpy.set_printoptions or numpy.array2string can be used instead.

The deprecated imports in the polynomial package have been removed.

concatenate now raises DepractionWarning for 1D arrays if axis != 0. Versions of numpy < 1.7.0 ignored axis argument value for 1D arrays. We allow this for now, but in due course we will raise an error.

C-API

Direct access to the fields of PyArrayObject* has been deprecated. Direct access has been recommended against for many releases. Expect similar deprecations for PyArray_Descr* and other core objects in the future as preparation for NumPy 2.0.

The macros in old_defines.h are deprecated and will be removed in the next major release (>= 2.0). The sed script tools/replace_old_macros.sed can be used to replace these macros with the newer versions.

You can test your code against the deprecated C API by #defining NPY_NO_DEPRECATED_API to the target version number, for example NPY_1_7_API_VERSION, before including any NumPy headers.

The NPY_CHAR member of the NPY_TYPES enum is deprecated and will be removed in NumPy 1.8. See the discussion at gh-2801 for more details.

NumPy 1.6.2 Release Notes

This is a bugfix release in the 1.6.x series. Due to the delay of the NumPy 1.7.0 release, this release contains far more fixes than a regular NumPy bugfix release. It also includes a number of documentation and build improvements.

Issues fixed

numpy.core

  • #2063: make unique() return consistent index
  • #1138: allow creating arrays from empty buffers or empty slices
  • #1446: correct note about correspondence vstack and concatenate
  • #1149: make argmin() work for datetime
  • #1672: fix allclose() to work for scalar inf
  • #1747: make np.median() work for 0-D arrays
  • #1776: make complex division by zero to yield inf properly
  • #1675: add scalar support for the format() function
  • #1905: explicitly check for NaNs in allclose()
  • #1952: allow floating ddof in std() and var()
  • #1948: fix regression for indexing chararrays with empty list
  • #2017: fix type hashing
  • #2046: deleting array attributes causes segfault
  • #2033: a**2.0 has incorrect type
  • #2045: make attribute/iterator_element deletions not segfault
  • #2021: fix segfault in searchsorted()
  • #2073: fix float16 __array_interface__ bug

numpy.lib

  • #2048: break reference cycle in NpzFile
  • #1573: savetxt() now handles complex arrays
  • #1387: allow bincount() to accept empty arrays
  • #1899: fixed histogramdd() bug with empty inputs
  • #1793: fix failing npyio test under py3k
  • #1936: fix extra nesting for subarray dtypes
  • #1848: make tril/triu return the same dtype as the original array
  • #1918: use Py_TYPE to access ob_type, so it works also on Py3

numpy.distutils

  • #1261: change compile flag on AIX from -O5 to -O3
  • #1377: update HP compiler flags
  • #1383: provide better support for C++ code on HPUX
  • #1857: fix build for py3k + pip
  • BLD: raise a clearer warning in case of building without cleaning up first
  • BLD: follow build_ext coding convention in build_clib
  • BLD: fix up detection of Intel CPU on OS X in system_info.py
  • BLD: add support for the new X11 directory structure on Ubuntu & co.
  • BLD: add ufsparse to the libraries search path.
  • BLD: add ‘pgfortran’ as a valid compiler in the Portland Group
  • BLD: update version match regexp for IBM AIX Fortran compilers.

numpy.random

  • BUG: Use npy_intp instead of long in mtrand

Changes

numpy.f2py

  • ENH: Introduce new options extra_f77_compiler_args and extra_f90_compiler_args
  • BLD: Improve reporting of fcompiler value
  • BUG: Fix f2py test_kind.py test

numpy.poly

  • ENH: Add some tests for polynomial printing
  • ENH: Add companion matrix functions
  • DOC: Rearrange the polynomial documents
  • BUG: Fix up links to classes
  • DOC: Add version added to some of the polynomial package modules
  • DOC: Document xxxfit functions in the polynomial package modules
  • BUG: The polynomial convenience classes let different types interact
  • DOC: Document the use of the polynomial convenience classes
  • DOC: Improve numpy reference documentation of polynomial classes
  • ENH: Improve the computation of polynomials from roots
  • STY: Code cleanup in polynomial [*]fromroots functions
  • DOC: Remove references to cast and NA, which were added in 1.7

NumPy 1.6.1 Release Notes

This is a bugfix only release in the 1.6.x series.

Issues Fixed

  • #1834: einsum fails for specific shapes
  • #1837: einsum throws nan or freezes python for specific array shapes
  • #1838: object <-> structured type arrays regression
  • #1851: regression for SWIG based code in 1.6.0
  • #1863: Buggy results when operating on array copied with astype()
  • #1870: Fix corner case of object array assignment
  • #1843: Py3k: fix error with recarray
  • #1885: nditer: Error in detecting double reduction loop
  • #1874: f2py: fix –include_paths bug
  • #1749: Fix ctypes.load_library()
  • #1895/1896: iter: writeonly operands weren’t always being buffered correctly

NumPy 1.6.0 Release Notes

This release includes several new features as well as numerous bug fixes and improved documentation. It is backward compatible with the 1.5.0 release, and supports Python 2.4 - 2.7 and 3.1 - 3.2.

Highlights

  • Re-introduction of datetime dtype support to deal with dates in arrays.
  • A new 16-bit floating point type.
  • A new iterator, which improves performance of many functions.

New features

New 16-bit floating point type

This release adds support for the IEEE 754-2008 binary16 format, available as the data type numpy.half. Within Python, the type behaves similarly to float or double, and C extensions can add support for it with the exposed half-float API.

New iterator

A new iterator has been added, replacing the functionality of the existing iterator and multi-iterator with a single object and API. This iterator works well with general memory layouts different from C or Fortran contiguous, and handles both standard NumPy and customized broadcasting. The buffering, automatic data type conversion, and optional output parameters, offered by ufuncs but difficult to replicate elsewhere, are now exposed by this iterator.

Legendre, Laguerre, Hermite, HermiteE polynomials in numpy.polynomial

Extend the number of polynomials available in the polynomial package. In addition, a new window attribute has been added to the classes in order to specify the range the domain maps to. This is mostly useful for the Laguerre, Hermite, and HermiteE polynomials whose natural domains are infinite and provides a more intuitive way to get the correct mapping of values without playing unnatural tricks with the domain.

Fortran assumed shape array and size function support in numpy.f2py

F2py now supports wrapping Fortran 90 routines that use assumed shape arrays. Before such routines could be called from Python but the corresponding Fortran routines received assumed shape arrays as zero length arrays which caused unpredicted results. Thanks to Lorenz Hüdepohl for pointing out the correct way to interface routines with assumed shape arrays.

In addition, f2py supports now automatic wrapping of Fortran routines that use two argument size function in dimension specifications.

Other new functions

numpy.ravel_multi_index : Converts a multi-index tuple into an array of flat indices, applying boundary modes to the indices.

numpy.einsum : Evaluate the Einstein summation convention. Using the Einstein summation convention, many common multi-dimensional array operations can be represented in a simple fashion. This function provides a way compute such summations.

numpy.count_nonzero : Counts the number of non-zero elements in an array.

numpy.result_type and numpy.min_scalar_type : These functions expose the underlying type promotion used by the ufuncs and other operations to determine the types of outputs. These improve upon the numpy.common_type and numpy.mintypecode which provide similar functionality but do not match the ufunc implementation.

Changes

default error handling

The default error handling has been change from print to warn for all except for underflow, which remains as ignore.

numpy.distutils

Several new compilers are supported for building Numpy: the Portland Group Fortran compiler on OS X, the PathScale compiler suite and the 64-bit Intel C compiler on Linux.

numpy.testing

The testing framework gained numpy.testing.assert_allclose, which provides a more convenient way to compare floating point arrays than assert_almost_equal, assert_approx_equal and assert_array_almost_equal.

C API

In addition to the APIs for the new iterator and half data type, a number of other additions have been made to the C API. The type promotion mechanism used by ufuncs is exposed via PyArray_PromoteTypes, PyArray_ResultType, and PyArray_MinScalarType. A new enumeration NPY_CASTING has been added which controls what types of casts are permitted. This is used by the new functions PyArray_CanCastArrayTo and PyArray_CanCastTypeTo. A more flexible way to handle conversion of arbitrary python objects into arrays is exposed by PyArray_GetArrayParamsFromObject.

Deprecated features

The “normed” keyword in numpy.histogram is deprecated. Its functionality will be replaced by the new “density” keyword.

Removed features

numpy.fft

The functions refft, refft2, refftn, irefft, irefft2, irefftn, which were aliases for the same functions without the ‘e’ in the name, were removed.

numpy.memmap

The sync() and close() methods of memmap were removed. Use flush() and “del memmap” instead.

numpy.lib

The deprecated functions numpy.unique1d, numpy.setmember1d, numpy.intersect1d_nu and numpy.lib.ufunclike.log2 were removed.

numpy.ma

Several deprecated items were removed from the numpy.ma module:

* ``numpy.ma.MaskedArray`` "raw_data" method
* ``numpy.ma.MaskedArray`` constructor "flag" keyword
* ``numpy.ma.make_mask`` "flag" keyword
* ``numpy.ma.allclose`` "fill_value" keyword

numpy.distutils

The numpy.get_numpy_include function was removed, use numpy.get_include instead.

NumPy 1.5.0 Release Notes

Highlights

Python 3 compatibility

This is the first NumPy release which is compatible with Python 3. Support for Python 3 and Python 2 is done from a single code base. Extensive notes on changes can be found at http://projects.scipy.org/numpy/browser/trunk/doc/Py3K.txt.

Note that the Numpy testing framework relies on nose, which does not have a Python 3 compatible release yet. A working Python 3 branch of nose can be found at http://bitbucket.org/jpellerin/nose3/ however.

Porting of SciPy to Python 3 is expected to be completed soon.

PEP 3118 compatibility

The new buffer protocol described by PEP 3118 is fully supported in this version of Numpy. On Python versions >= 2.6 Numpy arrays expose the buffer interface, and array(), asarray() and other functions accept new-style buffers as input.

New features

Warning on casting complex to real

Numpy now emits a numpy.ComplexWarning when a complex number is cast into a real number. For example:

>>> x = np.array([1,2,3])
>>> x[:2] = np.array([1+2j, 1-2j])
ComplexWarning: Casting complex values to real discards the imaginary part

The cast indeed discards the imaginary part, and this may not be the intended behavior in all cases, hence the warning. This warning can be turned off in the standard way:

>>> import warnings
>>> warnings.simplefilter("ignore", np.ComplexWarning)

Dot method for ndarrays

Ndarrays now have the dot product also as a method, which allows writing chains of matrix products as

>>> a.dot(b).dot(c)

instead of the longer alternative

>>> np.dot(a, np.dot(b, c))

linalg.slogdet function

The slogdet function returns the sign and logarithm of the determinant of a matrix. Because the determinant may involve the product of many small/large values, the result is often more accurate than that obtained by simple multiplication.

new header

The new header file ndarraytypes.h contains the symbols from ndarrayobject.h that do not depend on the PY_ARRAY_UNIQUE_SYMBOL and NO_IMPORT/_ARRAY macros. Broadly, these symbols are types, typedefs, and enumerations; the array function calls are left in ndarrayobject.h. This allows users to include array-related types and enumerations without needing to concern themselves with the macro expansions and their side- effects.

Changes

polynomial.polynomial

  • The polyint and polyder functions now check that the specified number integrations or derivations is a non-negative integer. The number 0 is a valid value for both functions.
  • A degree method has been added to the Polynomial class.
  • A trimdeg method has been added to the Polynomial class. It operates like truncate except that the argument is the desired degree of the result, not the number of coefficients.
  • Polynomial.fit now uses None as the default domain for the fit. The default Polynomial domain can be specified by using [] as the domain value.
  • Weights can be used in both polyfit and Polynomial.fit
  • A linspace method has been added to the Polynomial class to ease plotting.
  • The polymulx function was added.

polynomial.chebyshev

  • The chebint and chebder functions now check that the specified number integrations or derivations is a non-negative integer. The number 0 is a valid value for both functions.
  • A degree method has been added to the Chebyshev class.
  • A trimdeg method has been added to the Chebyshev class. It operates like truncate except that the argument is the desired degree of the result, not the number of coefficients.
  • Chebyshev.fit now uses None as the default domain for the fit. The default Chebyshev domain can be specified by using [] as the domain value.
  • Weights can be used in both chebfit and Chebyshev.fit
  • A linspace method has been added to the Chebyshev class to ease plotting.
  • The chebmulx function was added.
  • Added functions for the Chebyshev points of the first and second kind.

histogram

After a two years transition period, the old behavior of the histogram function has been phased out, and the “new” keyword has been removed.

correlate

The old behavior of correlate was deprecated in 1.4.0, the new behavior (the usual definition for cross-correlation) is now the default.

NumPy 1.4.0 Release Notes

This minor includes numerous bug fixes, as well as a few new features. It is backward compatible with 1.3.0 release.

Highlights

  • New datetime dtype support to deal with dates in arrays
  • Faster import time
  • Extended array wrapping mechanism for ufuncs
  • New Neighborhood iterator (C-level only)
  • C99-like complex functions in npymath

New features

Extended array wrapping mechanism for ufuncs

An __array_prepare__ method has been added to ndarray to provide subclasses greater flexibility to interact with ufuncs and ufunc-like functions. ndarray already provided __array_wrap__, which allowed subclasses to set the array type for the result and populate metadata on the way out of the ufunc (as seen in the implementation of MaskedArray). For some applications it is necessary to provide checks and populate metadata on the way in. __array_prepare__ is therefore called just after the ufunc has initialized the output array but before computing the results and populating it. This way, checks can be made and errors raised before operations which may modify data in place.

Automatic detection of forward incompatibilities

Previously, if an extension was built against a version N of NumPy, and used on a system with NumPy M < N, the import_array was successfull, which could cause crashes because the version M does not have a function in N. Starting from NumPy 1.4.0, this will cause a failure in import_array, so the error will be catched early on.

New iterators

A new neighborhood iterator has been added to the C API. It can be used to iterate over the items in a neighborhood of an array, and can handle boundaries conditions automatically. Zero and one padding are available, as well as arbitrary constant value, mirror and circular padding.

New polynomial support

New modules chebyshev and polynomial have been added. The new polynomial module is not compatible with the current polynomial support in numpy, but is much like the new chebyshev module. The most noticeable difference to most will be that coefficients are specified from low to high power, that the low level functions do not work with the Chebyshev and Polynomial classes as arguements, and that the Chebyshev and Polynomial classes include a domain. Mapping between domains is a linear substitution and the two classes can be converted one to the other, allowing, for instance, a Chebyshev series in one domain to be expanded as a polynomial in another domain. The new classes should generally be used instead of the low level functions, the latter are provided for those who wish to build their own classes.

The new modules are not automatically imported into the numpy namespace, they must be explicitly brought in with an “import numpy.polynomial” statement.

New C API

The following C functions have been added to the C API:

  1. PyArray_GetNDArrayCFeatureVersion: return the API version of the loaded numpy.
  2. PyArray_Correlate2 - like PyArray_Correlate, but implements the usual definition of correlation. Inputs are not swapped, and conjugate is taken for complex arrays.
  3. PyArray_NeighborhoodIterNew - a new iterator to iterate over a neighborhood of a point, with automatic boundaries handling. It is documented in the iterators section of the C-API reference, and you can find some examples in the multiarray_test.c.src file in numpy.core.

New ufuncs

The following ufuncs have been added to the C API:

  1. copysign - return the value of the first argument with the sign copied from the second argument.
  2. nextafter - return the next representable floating point value of the first argument toward the second argument.

New defines

The alpha processor is now defined and available in numpy/npy_cpu.h. The failed detection of the PARISC processor has been fixed. The defines are:

  1. NPY_CPU_HPPA: PARISC
  2. NPY_CPU_ALPHA: Alpha

Testing

  1. deprecated decorator: this decorator may be used to avoid cluttering testing output while testing DeprecationWarning is effectively raised by the decorated test.
  2. assert_array_almost_equal_nulps: new method to compare two arrays of floating point values. With this function, two values are considered close if there are not many representable floating point values in between, thus being more robust than assert_array_almost_equal when the values fluctuate a lot.
  3. assert_array_max_ulp: raise an assertion if there are more than N representable numbers between two floating point values.
  4. assert_warns: raise an AssertionError if a callable does not generate a warning of the appropriate class, without altering the warning state.

Reusing npymath

In 1.3.0, we started putting portable C math routines in npymath library, so that people can use those to write portable extensions. Unfortunately, it was not possible to easily link against this library: in 1.4.0, support has been added to numpy.distutils so that 3rd party can reuse this library. See coremath documentation for more information.

Improved set operations

In previous versions of NumPy some set functions (intersect1d, setxor1d, setdiff1d and setmember1d) could return incorrect results if the input arrays contained duplicate items. These now work correctly for input arrays with duplicates. setmember1d has been renamed to in1d, as with the change to accept arrays with duplicates it is no longer a set operation, and is conceptually similar to an elementwise version of the Python operator ‘in’. All of these functions now accept the boolean keyword assume_unique. This is False by default, but can be set True if the input arrays are known not to contain duplicates, which can increase the functions’ execution speed.

Improvements

  1. numpy import is noticeably faster (from 20 to 30 % depending on the platform and computer)

  2. The sort functions now sort nans to the end.

    • Real sort order is [R, nan]
    • Complex sort order is [R + Rj, R + nanj, nan + Rj, nan + nanj]

    Complex numbers with the same nan placements are sorted according to the non-nan part if it exists.

  3. The type comparison functions have been made consistent with the new sort order of nans. Searchsorted now works with sorted arrays containing nan values.

  4. Complex division has been made more resistent to overflow.

  5. Complex floor division has been made more resistent to overflow.

Deprecations

The following functions are deprecated:

  1. correlate: it takes a new keyword argument old_behavior. When True (the default), it returns the same result as before. When False, compute the conventional correlation, and take the conjugate for complex arrays. The old behavior will be removed in NumPy 1.5, and raises a DeprecationWarning in 1.4.
  2. unique1d: use unique instead. unique1d raises a deprecation warning in 1.4, and will be removed in 1.5.
  3. intersect1d_nu: use intersect1d instead. intersect1d_nu raises a deprecation warning in 1.4, and will be removed in 1.5.
  4. setmember1d: use in1d instead. setmember1d raises a deprecation warning in 1.4, and will be removed in 1.5.

The following raise errors:

  1. When operating on 0-d arrays, numpy.max and other functions accept only axis=0, axis=-1 and axis=None. Using an out-of-bounds axes is an indication of a bug, so Numpy raises an error for these cases now.
  2. Specifying axis > MAX_DIMS is no longer allowed; Numpy raises now an error instead of behaving similarly as for axis=None.

Internal changes

Use C99 complex functions when available

The numpy complex types are now guaranteed to be ABI compatible with C99 complex type, if availble on the platform. Moreoever, the complex ufunc now use the platform C99 functions intead of our own.

split multiarray and umath source code

The source code of multiarray and umath has been split into separate logic compilation units. This should make the source code more amenable for newcomers.

Separate compilation

By default, every file of multiarray (and umath) is merged into one for compilation as was the case before, but if NPY_SEPARATE_COMPILATION env variable is set to a non-negative value, experimental individual compilation of each file is enabled. This makes the compile/debug cycle much faster when working on core numpy.

Separate core math library

New functions which have been added:

  • npy_copysign
  • npy_nextafter
  • npy_cpack
  • npy_creal
  • npy_cimag
  • npy_cabs
  • npy_cexp
  • npy_clog
  • npy_cpow
  • npy_csqr
  • npy_ccos
  • npy_csin

NumPy 1.3.0 Release Notes

This minor includes numerous bug fixes, official python 2.6 support, and several new features such as generalized ufuncs.

Highlights

Python 2.6 support

Python 2.6 is now supported on all previously supported platforms, including windows.

http://www.python.org/dev/peps/pep-0361/

Generalized ufuncs

There is a general need for looping over not only functions on scalars but also over functions on vectors (or arrays), as explained on http://scipy.org/scipy/numpy/wiki/GeneralLoopingFunctions. We propose to realize this concept by generalizing the universal functions (ufuncs), and provide a C implementation that adds ~500 lines to the numpy code base. In current (specialized) ufuncs, the elementary function is limited to element-by-element operations, whereas the generalized version supports “sub-array” by “sub-array” operations. The Perl vector library PDL provides a similar functionality and its terms are re-used in the following.

Each generalized ufunc has information associated with it that states what the “core” dimensionality of the inputs is, as well as the corresponding dimensionality of the outputs (the element-wise ufuncs have zero core dimensions). The list of the core dimensions for all arguments is called the “signature” of a ufunc. For example, the ufunc numpy.add has signature “(),()->()” defining two scalar inputs and one scalar output.

Another example is (see the GeneralLoopingFunctions page) the function inner1d(a,b) with a signature of “(i),(i)->()”. This applies the inner product along the last axis of each input, but keeps the remaining indices intact. For example, where a is of shape (3,5,N) and b is of shape (5,N), this will return an output of shape (3,5). The underlying elementary function is called 3*5 times. In the signature, we specify one core dimension “(i)” for each input and zero core dimensions “()” for the output, since it takes two 1-d arrays and returns a scalar. By using the same name “i”, we specify that the two corresponding dimensions should be of the same size (or one of them is of size 1 and will be broadcasted).

The dimensions beyond the core dimensions are called “loop” dimensions. In the above example, this corresponds to (3,5).

The usual numpy “broadcasting” rules apply, where the signature determines how the dimensions of each input/output object are split into core and loop dimensions:

While an input array has a smaller dimensionality than the corresponding number of core dimensions, 1’s are pre-pended to its shape. The core dimensions are removed from all inputs and the remaining dimensions are broadcasted; defining the loop dimensions. The output is given by the loop dimensions plus the output core dimensions.

Experimental Windows 64 bits support

Numpy can now be built on windows 64 bits (amd64 only, not IA64), with both MS compilers and mingw-w64 compilers:

This is highly experimental: DO NOT USE FOR PRODUCTION USE. See INSTALL.txt, Windows 64 bits section for more information on limitations and how to build it by yourself.

New features

Formatting issues

Float formatting is now handled by numpy instead of the C runtime: this enables locale independent formatting, more robust fromstring and related methods. Special values (inf and nan) are also more consistent across platforms (nan vs IND/NaN, etc...), and more consistent with recent python formatting work (in 2.6 and later).

Nan handling in max/min

The maximum/minimum ufuncs now reliably propagate nans. If one of the arguments is a nan, then nan is retured. This affects np.min/np.max, amin/amax and the array methods max/min. New ufuncs fmax and fmin have been added to deal with non-propagating nans.

Nan handling in sign

The ufunc sign now returns nan for the sign of anan.

New ufuncs

  1. fmax - same as maximum for integer types and non-nan floats. Returns the non-nan argument if one argument is nan and returns nan if both arguments are nan.
  2. fmin - same as minimum for integer types and non-nan floats. Returns the non-nan argument if one argument is nan and returns nan if both arguments are nan.
  3. deg2rad - converts degrees to radians, same as the radians ufunc.
  4. rad2deg - converts radians to degrees, same as the degrees ufunc.
  5. log2 - base 2 logarithm.
  6. exp2 - base 2 exponential.
  7. trunc - truncate floats to nearest integer towards zero.
  8. logaddexp - add numbers stored as logarithms and return the logarithm of the result.
  9. logaddexp2 - add numbers stored as base 2 logarithms and return the base 2 logarithm of the result result.

Masked arrays

Several new features and bug fixes, including:

  • structured arrays should now be fully supported by MaskedArray (r6463, r6324, r6305, r6300, r6294...)
  • Minor bug fixes (r6356, r6352, r6335, r6299, r6298)
  • Improved support for __iter__ (r6326)
  • made baseclass, sharedmask and hardmask accesible to the user (but read-only)
  • doc update

gfortran support on windows

Gfortran can now be used as a fortran compiler for numpy on windows, even when the C compiler is Visual Studio (VS 2005 and above; VS 2003 will NOT work). Gfortran + Visual studio does not work on windows 64 bits (but gcc + gfortran does). It is unclear whether it will be possible to use gfortran and visual studio at all on x64.

Arch option for windows binary

Automatic arch detection can now be bypassed from the command line for the superpack installed:

numpy-1.3.0-superpack-win32.exe /arch=nosse

will install a numpy which works on any x86, even if the running computer supports SSE set.

Deprecated features

Histogram

The semantics of histogram has been modified to fix long-standing issues with outliers handling. The main changes concern

  1. the definition of the bin edges, now including the rightmost edge, and
  2. the handling of upper outliers, now ignored rather than tallied in the rightmost bin.

The previous behavior is still accessible using new=False, but this is deprecated, and will be removed entirely in 1.4.0.

Documentation changes

A lot of documentation has been added. Both user guide and references can be built from sphinx.

New C API

Multiarray API

The following functions have been added to the multiarray C API:

  • PyArray_GetEndianness: to get runtime endianness

Ufunc API

The following functions have been added to the ufunc API:

  • PyUFunc_FromFuncAndDataAndSignature: to declare a more general ufunc (generalized ufunc).

New defines

New public C defines are available for ARCH specific code through numpy/npy_cpu.h:

  • NPY_CPU_X86: x86 arch (32 bits)
  • NPY_CPU_AMD64: amd64 arch (x86_64, NOT Itanium)
  • NPY_CPU_PPC: 32 bits ppc
  • NPY_CPU_PPC64: 64 bits ppc
  • NPY_CPU_SPARC: 32 bits sparc
  • NPY_CPU_SPARC64: 64 bits sparc
  • NPY_CPU_S390: S390
  • NPY_CPU_IA64: ia64
  • NPY_CPU_PARISC: PARISC

New macros for CPU endianness has been added as well (see internal changes below for details):

  • NPY_BYTE_ORDER: integer
  • NPY_LITTLE_ENDIAN/NPY_BIG_ENDIAN defines

Those provide portable alternatives to glibc endian.h macros for platforms without it.

Portable NAN, INFINITY, etc...

npy_math.h now makes available several portable macro to get NAN, INFINITY:

  • NPY_NAN: equivalent to NAN, which is a GNU extension
  • NPY_INFINITY: equivalent to C99 INFINITY
  • NPY_PZERO, NPY_NZERO: positive and negative zero respectively

Corresponding single and extended precision macros are available as well. All references to NAN, or home-grown computation of NAN on the fly have been removed for consistency.

Internal changes

numpy.core math configuration revamp

This should make the porting to new platforms easier, and more robust. In particular, the configuration stage does not need to execute any code on the target platform, which is a first step toward cross-compilation.

http://projects.scipy.org/numpy/browser/trunk/doc/neps/math_config_clean.txt

umath refactor

A lot of code cleanup for umath/ufunc code (charris).

Improvements to build warnings

Numpy can now build with -W -Wall without warnings

http://projects.scipy.org/numpy/browser/trunk/doc/neps/warnfix.txt

Separate core math library

The core math functions (sin, cos, etc... for basic C types) have been put into a separate library; it acts as a compatibility layer, to support most C99 maths functions (real only for now). The library includes platform-specific fixes for various maths functions, such as using those versions should be more robust than using your platform functions directly. The API for existing functions is exactly the same as the C99 math functions API; the only difference is the npy prefix (npy_cos vs cos).

The core library will be made available to any extension in 1.4.0.

CPU arch detection

npy_cpu.h defines numpy specific CPU defines, such as NPY_CPU_X86, etc... Those are portable across OS and toolchains, and set up when the header is parsed, so that they can be safely used even in the case of cross-compilation (the values is not set when numpy is built), or for multi-arch binaries (e.g. fat binaries on Max OS X).

npy_endian.h defines numpy specific endianness defines, modeled on the glibc endian.h. NPY_BYTE_ORDER is equivalent to BYTE_ORDER, and one of NPY_LITTLE_ENDIAN or NPY_BIG_ENDIAN is defined. As for CPU archs, those are set when the header is parsed by the compiler, and as such can be used for cross-compilation and multi-arch binaries.

Table Of Contents

Previous topic

git resources

Next topic

About NumPy