Toolchain Roadmap¶
The use of the SciPy library requires (or optionally depends upon) several other libraries in order to operate, the main dependencies being Python and NumPy. It requires a larger collection of libraries and tools in order to build the library or to build the documentation.
Of course, the tooling and libraries are themselves not static. This document aims to provide a guide as to how SciPy’s use of these dynamic dependencies will proceed over time.
SciPy aims to be compatible with a number of releases of its dependent libraries and tools. Forcing the user base to other components for upgrade for every release would greatly diminish the value of SciPy. However, maintaining backwards compatibility with very old tooling/libraries imposes limitations on which newer functionalities and capabilities can be incorporated. SciPy takes a somewhat conservative approach, maintaining compatibility with several major releases of Python and NumPy on the major platforms. (That may in and of itself impose further restrictions. See the C Compilers section for an example.)
First and foremost, SciPy is a Python project, hence it requires a Python environment.
BLAS and LAPACK numerical libraries need to be installed.
Compilers for C, C++, Cython, and Fortran code are needed.
The Python environment needs the
NumPy
package to be installed.Testing requires the
pytest
Python package.Building the documentation requires the
matplotlib
, Sphinx packages, as well as a LaTeX installation.
The tooling used to build CPython has some implications for the tooling used in building SciPy. It also has implications for the examples used in the documentation (e.g., docstrings for functions), as these examples can only use functionality present in all supported configurations.
Building SciPy¶
Python Versions¶
SciPy is compatible with several versions of Python. When dropping support for older Python versions, SciPy takes guidance from NEP 29 10. Python 2.7 support was dropped for SciPy releases numbered 1.3 and above but is still available in release 1.2.x, which is a long-term support release 1, 2.
Date |
Pythons supported |
---|---|
2018 |
Py2.7, Py3.4+ (SciPy 1.2.x is the last release to support Python 2.7) |
2019 |
Py3.5+ (but Py2.7-specific code not removed) |
2020 |
Py3.6+ (removal of Py2.7-specific code permitted) |
NumPy¶
SciPy depends on NumPy but releases of SciPy are not tied to releases of NumPy. SciPy attempts to be compatible with at least the 4 previous releases of NumPy. In particular, SciPy cannot rely on features of just the latest NumPy, but needs to be written using what is common in all of those 4 releases. 1, 3.
The table shows the NumPy versions suitable for each major Python version (for SciPy 1.3.x unless otherwise stated).
Python |
Minimum NumPy version |
Maximum NumPy version |
---|---|---|
2.7 (SciPy 1.2) |
1.8.2 |
1.16.x |
3.5 (SciPy 1.4) |
1.13.3 |
1.18.x |
3.6 |
1.14.5 |
>= 1.18.x |
3.7 |
1.14.5 |
>= 1.18.x |
3.8 |
1.17.3 |
>= 1.18.x |
C Compilers¶
SciPy is compatible with most modern C compilers (in particular clang
).
However, CPython on Windows is
built with specific versions of the Microsoft Visual C++ compiler 7, 8, 9,
as is the corresponding build of SciPy. This has implications for the
C language standards that can be supported 6.
CPython |
MS Visual C++ |
C Standard |
---|---|---|
2.7, 3.0, 3.1, 3.2 |
9.0 |
C90 |
3.3, 3.4 |
10.0 |
C90 & some of C99 |
3.5, 3.6 |
14.0 |
C90 & most of C99 |
3.7 |
15.7 |
C90 & most of C99 |
3.8 |
15.7 |
C90 & most of C99 |
C and C++ Language Standards¶
C and C++ language standards for SciPy are generally guidelines rather than official decisions. This is particularly true of attempting to predict adoption timelines for newer standards.
Date |
C Standard |
---|---|
<= 2018 |
C90 |
2019 |
C90 for old code, may consider C99 for new |
2020 |
C99 |
2020 |
C11 |
? |
C14 some C++14 features can be used, typically limited by MSVC support |
? |
C17, C18 |
The use of MS Visual Studio 9.0 (which doesn’t have support C99)
to build Python2.7 has meant that C code in SciPy has had to conform
to the earlier C90 standard for the language and standard library.
With the dropping of Python2.7 for SciPy 1.3.x, the C90 restriction is no
longer imposed by compilers. For GCC version < 5, an explicit -std=c99
may have to be added by the user if C99 features are used in SciPy code.
Note: even though C99 has been a standard for 20 years, experience has shown
that not all features are supported equally well across all platforms.
C18 is a bug fix for C11, so C11 may be skipped entirely.
In practice, the C++ feature set that can be used is limited by the availability in the MS VisualStudio versions that SciPy needs to support. C++11 can be used, C++14/17 is going to be impossible for a very long time because of ecosystem support restrictions. See 4.
Note
Developer Note: Some C99 features would be useful for scientific
programming, in particular better support of IEEE 754 5.
SciPy has a small include file scipy/_lib/_c99compat.h
which
provides access to a few functions. Use in conjunction
with <numpy/npy_math.h>
.
Feature |
Workaround |
---|---|
|
Use |
|
Use |
inline functions |
Make static functions and place in an include .h file |
mid-block variable declarations |
Declare variables at the top of the block |
Fortran Compilers¶
Generally, any well-maintained compiler is likely suitable and can be used to build SciPy.
Tool |
Version |
---|---|
gfortran |
>= 4.8.0 |
ifort |
A recent version |
flang |
A recent version |
Cython Compiler¶
SciPy always requires a recent Cython compiler.
Tool |
Tool Version |
SciPy version |
---|---|---|
Cython |
>= 0.29.13 |
1.4.1 |
Cython |
>= 0.29.18 |
1.5.0 |
Other Libraries¶
Any library conforming to the BLAS/LAPACK interface may be used. OpenBLAS, ATLAS, MKL, BLIS, and reference Netlib libraries are known to work.
Library |
Minimum version |
---|---|
LAPACK |
3.4.1 |
BLAS |
A recent version of OpenBLAS, MKL or ATLAS. The Accelerate BLAS is no longer supported. |
There are some additional optional dependencies.
Library |
Version |
URL |
---|---|---|
mpmath |
Recent |
|
scikit-umfpack |
Recent |
Moreover, Scipy supports interaction with other libraries. The test suite has additional compatibility tests that are run when these are installed:
Tool |
Version |
URL |
---|---|---|
pydata/sparse |
Recent |
Testing and Benchmarking¶
Testing and benchmarking require recent versions of:
Tool |
Version |
URL |
---|---|---|
pytest |
Recent |
|
asv (airspeed velocity) |
Recent |
Building the Documentation¶
Tool |
Version |
---|---|
Sphinx |
Whatever recent versions work. >= 2.0. |
numpydoc |
Whatever recent versions work. >= 0.8.0. |
matplotlib |
Generally suggest >= 2.0. |
LaTeX |
A recent distribution, such as |
[The numpydoc
package is also used, but that is currently
packaged in doc/sphinxext
.]
Note
Developer Note: The versions of numpy
and matplotlib
required have
implications for the examples in Python docstrings.
Examples must be able to be executed both in the environment used to
build the documentation,
as well as with any supported versions of numpy/matplotlib
that
a user may use with this release of SciPy.
Packaging¶
A Recent version of:
Tool |
Version |
URL |
---|---|---|
setuptools |
Recent |
|
wheel |
Recent |
|
multibuild |
Recent |
Making a SciPy release and Distributing contain information on making and distributing a SciPy release.
References¶
- 1(1,2)
https://docs.scipy.org/doc/scipy/reference/release.1.2.0.html
- 2
- 3
- 4
- 5
- 6
https://blogs.msdn.microsoft.com/vcblog/2013/07/19/c99-library-support-in-visual-studio-2013/
- 7
https://pythondev.readthedocs.io/windows.html#python-and-visual-studio-version-matrix
- 8
https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B#Internal_version_numbering
- 9
- 10