Running SciPy Tests Locally

Basic test writing and execution from within the Python interpreter is documented in the NumPy/SciPy Testing Guidelines. This page includes information about running tests from the command line using SciPy’s runtests.py, which permits greater control. Note: Before beginning, ensure that pytest is installed.

To run all tests, navigate to the root SciPy directory at the command line and execute

python runtests.py -v

where -v activates the --verbose option. This builds SciPy (or updates an existing build) and runs the tests.

To run tests on a particular submodule, such as optimize, use the --submodule option:

python runtests.py -v -s optimize

To run a particular test module, such as scipy/optimize/tests/test_linprog.py, use the --test option:

python runtests.py -v -t scipy/optimize/tests/test_linprog.py

To run a test class, such as TestLinprogIPDense from test_linprog.py:

python runtests.py -v -t scipy/optimize/tests/test_linprog.py::TestLinprogIPDense

To run a particular test, such as test_unknown_solver from test_linprog.py:

python runtests.py -v -t scipy/optimize/tests/test_linprog.py::test_unknown_solver

For tests within a class, you need to specify the class name and the test name:

python runtests.py -v -t scipy/optimize/tests/test_linprog.py::TestLinprogIPDense::test_nontrivial_problem

Other useful options include:

• --coverage to generate a test coverage report in scipy/build/coverage/index.html. Note: pytest-cov must be installed.

• --doc to build the docs in scipy/doc/build. By default, docs are built only in the html-scipyorg format, but you can change this by appending the name of the desired format (e.g. --doc latex).

• -n or --no-build to prevent SciPy from updating the build before testing

• -j or --parallel n to engage n cores when building SciPy; e.g. python runtests.py -j 4 engages four cores. As of #10172 this also runs the tests on four cores if pytest-xdist is installed.

• -m or --mode full to run the full test suite, including slow tests. For example, python runtests.py -m full.

• -- to send remaining command line arguments to pytest instead of runtest.py. For instance, while -n sent to pytest.py activates the --no-build option, -n sent to pytest runs the tests on multiple cores; e.g. python runtests.py -- -n 4 runs tests using four cores. Note: pytest-xdist must be installed for testing on multiple cores.

Other options not documented here are listed in the main function of the source code for runtests.py. For much more information about pytest, see the pytest documentation.

Tips:

If you built SciPy from source but are having trouble running tests after a change to the codebase, try deleting the scipy/build directory. This forces runtest.py to completely rebuild SciPy before performing tests.

There is an additional level of very slow tests (several minutes), which are disabled even when calling python runtests.py -m full. They can be enabled by setting the environment variable SCIPY_XSLOW=1 before running the test suite.