SciPy

Toolchain Roadmap

The use of the SciPy library requires (or optionally depends upon) several other libraries in order to operate, the main dependency being Python and NumPy. It requires a larger collection of libraries and tools in order to build the library, or to build the documentation.

Of course, the tooling and libraries are themselves not static. This document aims to provide a guide as to how SciPy’s use of these dynamic dependencies will proceed over time.

SciPy aims to be compatible with a number of releases of its dependent libraries and tools. Forcing the user base to other components for upgrade for every release would greatly diminish the value of SciPy. However, maintaining backwards compatibility with very old tooling/libraries imposes limitations on which newer functionalities and capabilities can be incorporated. SciPy takes a somewhat conservative approach, maintaining compatibility with several major releases of Python and NumPy on the major platforms. (That may in of itself impose further restrictions. See the C Compilers section for an example.)

  • First and foremost, SciPy is a Python project hence it requires a Python environment.

  • BLAS and LAPACK numerical libraries need to be installed.

  • Compilers for C, C++, Cython and Fortran code are needed.

  • The Python environment needs the NumPy package to be installed.

  • Testing requires the pytest Python package.

  • Building the documentation requires the matplotlib, Sphinx packages, as well as a LaTeX installation.

The tooling used to build CPython has some implications for the tooling used in building SciPy. It also has implications for the examples used in the documentation (e.g. docstrings for functions), as these examples can only use functionality present in all supported configurations.

Building SciPy

Python Versions

SciPy is compatible with several versions of Python, and some specific decisions are still under consideration, especially with regard to future changes. Python 2.7 support was dropped for SciPy releases numbered 1.3 and above but is still available in Release 1.2.x, which is a long-term support release. 1, 2.

Date

Pythons supported

2018

Py2.7, Py3.4+ (SciPy 1.2.x is the last release to support Python 2.7)

2019

Py3.5+ (but Py2.7-specific code not removed)

2020

Py3.6+ (removal of Py2.7-specific code permitted)

NumPy

SciPy depends on NumPy but releases of SciPy are not tied to releases of NumPy. SciPy attempts to be compatible with at least the 4 previous releases of NumPy. In particular, SciPy can not rely on features of just the latest NumPy, but needs to be written using what is common in all of those 4 releases. 1, 3.

The table shows the NumPy versions suitable for each major Python version (for SciPy 1.3.x unless otherwise stated.)

Python

Minimum NumPy version

Maximum NumPy version

2.7 (SciPy 1.2)

1.8.2

1.16.x

3.5

1.13.3

>= 1.16.x

3.6

1.13.3

>= 1.16.x

3.7

1.14.5

>= 1.16.x

C Compilers

SciPy is compatible with most modern C compilers (in particular clang). However CPython on Windows is built with specific versions of the Microsoft Visual C++ compiler 7, 8, 9, as is the corresponding build of SciPy. This has implications for the C language standards that can be supported 6.

CPython

MS Visual C++

C Standard

2.7, 3.0, 3.1, 3.2

9.0

C90

3.3, 3.4

10.0

C90 & some of C99

3.5, 3.6

14.0

C90 & most of C99

3.7

15.7

C90 & most of C99

C and C++ Language Standards

C and C++ language standards for SciPy are generally guidelines rather than official decisions. This is particularly true of attempting to predict adoption timelines for newer standards.

Date

C Standard

<= 2018

C90

2019

C90 for old code, may consider C99 for new

2020

C99

?

C11

?

C17, C18

The use of MSVisual Studio 9.0 (which doesn’t have support C99) to build Python2.7, has meant that C code in SciPy has had to conform to the earlier C90 standard for the language and standard library. With the dropping of Python2.7 for SciPy 1.3.x, the C90 restriction is no longer imposed by compilers. Even though C99 has been a standard for 20 years, experience has shown that not all features are supported equally well across all platforms. The expectation is that C99 code will be become acceptable in 2020.

C18 is a bug fix for C11, so C11 may be skipped entirely.

In practice the C++ feature set that can be used is limited by the availability in the MS VisualStudio versions that SciPy needs to support. C++11 can be used, C++14/17 is going to be impossible for a very long time because of ecosystem support restrictions. See 4.

Note

Developer Note: Some C99 features would be useful for scientific programming, in particular better support of IEEE 754 5. SciPy has a small include file scipy/_lib/_c99compat.h which provides access to a few functions. Use in conjunction with <numpy/npy_math.h>.

Feature

Workaround

isnan(), isinf(), isfinite()

Use sc_isnan(), sc_isinf(), sc_isfinite()

NAN

Use NPY_NAN (it is almost equivalent)

inline functions

Make static functions and place in an include .h file

mid-block variable declarations

Declare variables at the top of the block

Fortran Compilers

Generally, any well-maintained compiler is likely suitable and can be used to build SciPy.

Tool

Version

gfortran

>= 4.8.0

ifort

A recent version

flang

A recent version

Cython Compiler

SciPy always requires a recent Cython compiler.

Tool

Tool Version

SciPy version

Cython

>= 0.29.2

1.2.1

Other Libraries

Any library conforming to the BLAS/LAPACK interface may be used. OpenBLAS, ATLAS, MKL, BLIS and reference Netlib libraries are known to work.

Library

Minimum version

LAPACK

3.4.1

BLAS

A recent version of OpenBLAS, MKL or ATLAS. The Accelerate BLAS is no longer supported.

There are some additional optional dependencies.

Library

Version

URL

mpmath

Recent

http://mpmath.org/

scikit-umfpack

Recent

https://pypi.org/project/scikit-umfpack/

Testing and Benchmarking

Testing and benchmarking require recent versions of:

Tool

Version

URL

pytest

Recent

https://docs.pytest.org/en/latest/

asv (airspeed velocity)

Recent

https://asv.readthedocs.io/

Building the Documentation

Tool

Version

Sphinx

Whatever recent versions work. >= 2.0.

numpydoc

Whatever recent versions work. >= 0.8.0.

matplotlib

Generally suggest >= 2.0

LaTeX

A recent distribution, such as TeX Live 2016

[The numpydoc package is also used, but that is currently packaged in doc/sphinxext.]

Note

Developer Note: The versions of numpy and matplotlib required has implications for the examples in Python docstrings. Examples must be able to be executed both in the environment used to build the documentation, as well as with any supported versions of numpy/matplotlib that a user may use with this release of SciPy.

Packaging

A Recent version of:

Tool

Version

URL

setuptools

Recent

wheel

Recent

https://pythonwheels.com

multibuild

Recent

https://github.com/matthew-brett/multibuild

Making a SciPy release and Distributing contain information on making and distributing a SciPy release.