Running SciPy Tests Locally#

Basic test writing and execution from within the Python interpreter is documented in the NumPy/SciPy Testing Guidelines. This page includes information about running tests from the command line using SciPy’s runtests.py, which permits greater control. Note: Before beginning, ensure that pytest is installed.

To run all tests, navigate to the root SciPy directory at the command line and execute

python runtests.py -v

where -v activates the --verbose option. This builds SciPy (or updates an existing build) and runs the tests.

To run tests on a particular submodule, such as optimize, use the --submodule option:

python runtests.py -v -s optimize

To run a particular test module, use the --test option:

python runtests.py -v -t scipy.<module>.tests.<test_file>

Example for scipy/optimize/tests/test_linprog.py file tests, run:

python runtests.py -v -t scipy.optimize.tests.test_linprog

To run a test class:

python runtests.py -v -t scipy.<module>.tests.<test_file>::<TestClass>

Example for TestLinprogRSCommon class from test_linprog.py:

python runtests.py -v -t scipy.optimize.tests.test_linprog::TestLinprogRSCommon

To run a particular test:

python runtests.py -v -t scipy.<module>.tests.<test_file>::<test_name>

Example for test_unknown_solvers_and_options from test_linprog.py:

python runtests.py -v -t scipy.optimize.tests.test_linprog::test_unknown_solvers_and_options

For tests within a class, you need to specify the class name and the test name:

python runtests.py -v -t scipy.<module>.tests.<test_file>::<TestClass>::<test_name>

Example:

python runtests.py -v -t scipy.optimize.tests.test_linprog::TestLinprogRSCommon::test_nontrivial_problem_with_guess

Other useful options include:

  • --coverage to generate a test coverage report in scipy/build/coverage/index.html. Note: pytest-cov must be installed.

  • --doc to build the docs in scipy/doc/build. By default, docs are built only in the html format, but you can change this by appending the name of the desired format.

  • --refguide-check to check whether the objects in a Scipy submodule’s __all__ dict correspond to the objects included in the reference guide. It also checks the validity of code samples in docstrings.

  • --bench to run all benchmarks. See Benchmarking SciPy with airspeed velocity.

  • --pep8 to perform pep8 check.

  • --mypy to run mypy on the codebase.

  • -n or --no-build to prevent SciPy from updating the build before testing

  • -j or --parallel n to engage n cores when building SciPy; e.g. python runtests.py -j 4 engages four cores. As of #10172 this also runs the tests on four cores if pytest-xdist is installed.

  • -m or --mode full to run the full test suite, including slow tests. For example, python runtests.py -m full.

  • -- to send remaining command line arguments to pytest instead of runtest.py. For instance, while -n sent to pytest.py activates the --no-build option, -n sent to pytest runs the tests on multiple cores; e.g. python runtests.py -- -n 4 runs tests using four cores. Note: pytest-xdist must be installed for testing on multiple cores.

Other options not documented here are listed in the main function of the source code for runtests.py. For much more information about pytest, see the pytest documentation.

Tips:#

If you built SciPy from source but are having trouble running tests after a change to the codebase, try deleting the scipy/build directory. This forces runtest.py to completely rebuild SciPy before performing tests.

There is an additional level of very slow tests (several minutes), which are disabled even when calling python runtests.py -m full. They can be enabled by setting the environment variable SCIPY_XSLOW=1 before running the test suite.