Toolchain Roadmap

The use of the SciPy library requires (or optionally depends upon) several other libraries in order to operate, the main dependency being Python and NumPy. It requires a larger collection of libraries and tools in order to build the library, or to build the documentation.

Of course, the tooling and libraries are themselves not static. This document aims to provide a guide as to how SciPy’s use of these dynamic dependencies will proceed over time.

SciPy aims to be compatible with a number of releases of its dependent libraries and tools. Forcing the user base to other components for upgrade for every release would greatly diminish the value of SciPy. However, maintaining backwards compatibility with very old tooling/libraries imposes limitations on which newer functionalities and capabilities can be incorporated. SciPy takes a somewhat conservative approach, maintaining compatibility with several major releases of Python and NumPy on the major platforms. (That may in of itself impose further restrictions. See the C Compilers section for an example.)

• First and foremost, SciPy is a Python project hence it requires a Python environment.

• BLAS and LAPACK numerical libraries need to be installed.

• Compilers for C, C++, Cython and Fortran code are needed.

• The Python environment needs the NumPy package to be installed.

• Testing requires the pytest Python package.

• Building the documentation requires the matplotlib, Sphinx packages, as well as a LaTeX installation.

The tooling used to build CPython has some implications for the tooling used in building SciPy. It also has implications for the examples used in the documentation (e.g. docstrings for functions), as these examples can only use functionality present in all supported configurations.

Building SciPy

Python Versions

SciPy is compatible with several versions of Python, and some specific decisions are still under consideration, especially with regard to future changes. Python 2.7 support was dropped for SciPy releases numbered 1.3 and above but is still available in Release 1.2.x, which is a long-term support release. 1, 2.

Date	Pythons supported
2018	Py2.7, Py3.4+ (SciPy 1.2.x is the last release to support Python 2.7)
2019	Py3.5+ (but Py2.7-specific code not removed)
2020	Py3.6+ (removal of Py2.7-specific code permitted)

NumPy

SciPy depends on NumPy but releases of SciPy are not tied to releases of NumPy. SciPy attempts to be compatible with at least the 4 previous releases of NumPy. In particular, SciPy can not rely on features of just the latest NumPy, but needs to be written using what is common in all of those 4 releases. 1, 3.

The table shows the NumPy versions suitable for each major Python version (for SciPy 1.3.x unless otherwise stated.)

Python	Minimum NumPy version	Maximum NumPy version
2.7 (SciPy 1.2)	1.8.2	1.16.x
3.5	1.13.3	>= 1.16.x
3.6	1.13.3	>= 1.16.x
3.7	1.14.5	>= 1.16.x

C Compilers

SciPy is compatible with most modern C compilers (in particular clang). However CPython on Windows is built with specific versions of the Microsoft Visual C++ compiler 7, 8, 9, as is the corresponding build of SciPy. This has implications for the C language standards that can be supported 6.

CPython	MS Visual C++	C Standard
2.7, 3.0, 3.1, 3.2	9.0	C90
3.3, 3.4	10.0	C90 & some of C99
3.5, 3.6	14.0	C90 & most of C99
3.7	15.7	C90 & most of C99

C and C++ Language Standards

C and C++ language standards for SciPy are generally guidelines rather than official decisions. This is particularly true of attempting to predict adoption timelines for newer standards.

Date	C Standard
<= 2018	C90
2019	C90 for old code, may consider C99 for new
2020	C99
?	C11
?	C17, C18

The use of MSVisual Studio 9.0 (which doesn’t have support C99) to build Python2.7, has meant that C code in SciPy has had to conform to the earlier C90 standard for the language and standard library. With the dropping of Python2.7 for SciPy 1.3.x, the C90 restriction is no longer imposed by compilers. Even though C99 has been a standard for 20 years, experience has shown that not all features are supported equally well across all platforms. The expectation is that C99 code will be become acceptable in 2020.

C18 is a bug fix for C11, so C11 may be skipped entirely.

In practice the C++ feature set that can be used is limited by the availability in the MS VisualStudio versions that SciPy needs to support. C++11 can be used, C++14/17 is going to be impossible for a very long time because of ecosystem support restrictions. See 4.

Note

Developer Note: Some C99 features would be useful for scientific programming, in particular better support of IEEE 754 5. SciPy has a small include file scipy/_lib/_c99compat.h which provides access to a few functions. Use in conjunction with <numpy/npy_math.h>.

Feature	Workaround
isnan(), isinf(), isfinite()	Use sc_isnan(), sc_isinf(), sc_isfinite()
NAN	Use NPY_NAN (it is almost equivalent)
inline functions	Make static functions and place in an include .h file
mid-block variable declarations	Declare variables at the top of the block

Fortran Compilers

Generally, any well-maintained compiler is likely suitable and can be used to build SciPy.

Tool	Version
gfortran	>= 4.8.0
ifort	A recent version
flang	A recent version

Cython Compiler

SciPy always requires a recent Cython compiler.

Tool	Tool Version	SciPy version
Cython	>= 0.29.2	1.2.1

Other Libraries

Any library conforming to the BLAS/LAPACK interface may be used. OpenBLAS, ATLAS, MKL, BLIS and reference Netlib libraries are known to work.

Library	Minimum version
LAPACK	3.4.1
BLAS	A recent version of OpenBLAS, MKL or ATLAS. The Accelerate BLAS is no longer supported.

There are some additional optional dependencies.

Library	Version	URL
mpmath	Recent	http://mpmath.org/
scikit-umfpack	Recent	https://pypi.org/project/scikit-umfpack/

Testing and Benchmarking

Testing and benchmarking require recent versions of:

Tool	Version	URL
pytest	Recent	https://docs.pytest.org/en/latest/
asv (airspeed velocity)	Recent	https://asv.readthedocs.io/

Building the Documentation

Tool	Version
Sphinx	Whatever recent versions work. >= 2.0.
numpydoc	Whatever recent versions work. >= 0.8.0.
matplotlib	Generally suggest >= 2.0
LaTeX	A recent distribution, such as TeX Live 2016

[The numpydoc package is also used, but that is currently packaged in doc/sphinxext.]

Note

Developer Note: The versions of numpy and matplotlib required has implications for the examples in Python docstrings. Examples must be able to be executed both in the environment used to build the documentation, as well as with any supported versions of numpy/matplotlib that a user may use with this release of SciPy.

Packaging

A Recent version of:

Tool	Version	URL
setuptools	Recent
wheel	Recent	https://pythonwheels.com
multibuild	Recent	https://github.com/matthew-brett/multibuild

Making a SciPy release and Distributing contain information on making and distributing a SciPy release.

References

1(1,2)

https://docs.scipy.org/doc/scipy/reference/release.1.2.0.html

2

https://python3statement.org

3

https://docs.scipy.org/doc/numpy/release.html

4

https://en.cppreference.com/w/cpp/compiler_support

5

https://en.wikipedia.org/wiki/IEEE_754-1985

6

https://blogs.msdn.microsoft.com/vcblog/2013/07/19/c99-library-support-in-visual-studio-2013/

7

https://pythondev.readthedocs.io/windows.html#python-and-visual-studio-version-matrix

8

https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B#Internal_version_numbering

9

https://wiki.python.org/moin/WindowsCompilers