linprog(method=’interiorpoint’)¶

scipy.optimize.
linprog
(c, method='interiorpoint', callback=None, options={'c0': 0, 'A': None, 'b': None, 'alpha0': 0.99995, 'beta': 0.1, 'maxiter': 1000, 'disp': False, 'tol': 1e08, 'sparse': False, 'lstsq': False, 'sym_pos': True, 'cholesky': None, 'pc': True, 'ip': False, 'permc_spec': 'MMD_AT_PLUS_A'}) Minimize a linear objective function subject to linear equality and nonnegativity constraints using the interior point method of [4]. Linear programming is intended to solve problems of the following form:
Minimize:
c @ x
Subject to:
A @ x == b x >= 0
Parameters:  c : 1D array
Coefficients of the linear objective function to be minimized.
 c0 : float
Constant term in objective function due to fixed (and eliminated) variables. (Purely for display.)
 A : 2D array
2D array such that
A @ x
, gives the values of the equality constraints atx
. b : 1D array
1D array of values representing the right hand side of each equality constraint (row) in
A
.
Returns:  x : 1D array
Solution vector.
 status : int
An integer representing the exit status of the optimization:
0 : Optimization terminated successfully 1 : Iteration limit reached 2 : Problem appears to be infeasible 3 : Problem appears to be unbounded 4 : Serious numerical difficulties encountered
 message : str
A string descriptor of the exit status of the optimization.
 iteration : int
The number of iterations taken to solve the problem.
See also
For documentation for the rest of the parameters, see
scipy.optimize.linprog
Options:  maxiter : int (default = 1000)
The maximum number of iterations of the algorithm.
 disp : bool (default = False)
Set to
True
if indicators of optimization status are to be printed to the console each iteration. tol : float (default = 1e8)
Termination tolerance to be used for all termination criteria; see [4] Section 4.5.
 alpha0 : float (default = 0.99995)
The maximal step size for Mehrota’s predictorcorrector search direction; see \(\beta_{3}\) of [4] Table 8.1.
 beta : float (default = 0.1)
The desired reduction of the path parameter \(\mu\) (see [6]) when Mehrota’s predictorcorrector is not in use (uncommon).
 sparse : bool (default = False)
Set to
True
if the problem is to be treated as sparse after presolve. If eitherA_eq
orA_ub
is a sparse matrix, this option will automatically be setTrue
, and the problem will be treated as sparse even during presolve. If your constraint matrices contain mostly zeros and the problem is not very small (less than about 100 constraints or variables), consider settingTrue
or providingA_eq
andA_ub
as sparse matrices. lstsq : bool (default = False)
Set to
True
if the problem is expected to be very poorly conditioned. This should always be leftFalse
unless severe numerical difficulties are encountered. Leave this at the default unless you receive a warning message suggesting otherwise. sym_pos : bool (default = True)
Leave
True
if the problem is expected to yield a well conditioned symmetric positive definite normal equation matrix (almost always). Leave this at the default unless you receive a warning message suggesting otherwise. cholesky : bool (default = True)
Set to
True
if the normal equations are to be solved by explicit Cholesky decomposition followed by explicit forward/backward substitution. This is typically faster for moderate, dense problems that are numerically wellbehaved. pc : bool (default = True)
Leave
True
if the predictorcorrector method of Mehrota is to be used. This is almost always (if not always) beneficial. ip : bool (default = False)
Set to
True
if the improved initial point suggestion due to [4] Section 4.3 is desired. Whether this is beneficial or not depends on the problem. permc_spec : str (default = ‘MMD_AT_PLUS_A’)
(Has effect only with
sparse = True
,lstsq = False
,sym_pos = True
.) A matrix is factorized in each iteration of the algorithm. This option specifies how to permute the columns of the matrix for sparsity preservation. Acceptable values are:NATURAL
: natural ordering.MMD_ATA
: minimum degree ordering on the structure of A^T A.MMD_AT_PLUS_A
: minimum degree ordering on the structure of A^T+A.COLAMD
: approximate minimum degree column ordering.
This option can impact the convergence of the interior point algorithm; test different values to determine which performs best for your problem. For more information, refer to
scipy.sparse.linalg.splu
.
Notes
This method implements the algorithm outlined in [4] with ideas from [8] and a structure inspired by the simpler methods of [6] and [4].
The primaldual path following method begins with initial ‘guesses’ of the primal and dual variables of the standard form problem and iteratively attempts to solve the (nonlinear) KarushKuhnTucker conditions for the problem with a gradually reduced logarithmic barrier term added to the objective. This particular implementation uses a homogeneous selfdual formulation, which provides certificates of infeasibility or unboundedness where applicable.
The default initial point for the primal and dual variables is that defined in [4] Section 4.4 Equation 8.22. Optionally (by setting initial point option
ip=True
), an alternate (potentially improved) starting point can be calculated according to the additional recommendations of [4] Section 4.4.A search direction is calculated using the predictorcorrector method (single correction) proposed by Mehrota and detailed in [4] Section 4.1. (A potential improvement would be to implement the method of multiple corrections described in [4] Section 4.2.) In practice, this is accomplished by solving the normal equations, [4] Section 5.1 Equations 8.31 and 8.32, derived from the Newton equations [4] Section 5 Equations 8.25 (compare to [4] Section 4 Equations 8.68.8). The advantage of solving the normal equations rather than 8.25 directly is that the matrices involved are symmetric positive definite, so Cholesky decomposition can be used rather than the more expensive LU factorization.
With the default
cholesky=True
, this is accomplished usingscipy.linalg.cho_factor
followed by forward/backward substitutions viascipy.linalg.cho_solve
. Withcholesky=False
andsym_pos=True
, Cholesky decomposition is performed instead byscipy.linalg.solve
. Based on speed tests, this also appears to retain the Cholesky decomposition of the matrix for later use, which is beneficial as the same system is solved four times with different right hand sides in each iteration of the algorithm.In problems with redundancy (e.g. if presolve is turned off with option
presolve=False
) or if the matrices become illconditioned (e.g. as the solution is approached and some decision variables approach zero), Cholesky decomposition can fail. Should this occur, successively more robust solvers (scipy.linalg.solve
withsym_pos=False
thenscipy.linalg.lstsq
) are tried, at the cost of computational efficiency. These solvers can be used from the outset by setting the optionssym_pos=False
andlstsq=True
, respectively.Note that with the option
sparse=True
, the normal equations are solved usingscipy.sparse.linalg.spsolve
. Unfortunately, this uses the more expensive LU decomposition from the outset, but for large, sparse problems, the use of sparse linear algebra techniques improves the solve speed despite the use of LU rather than Cholesky decomposition. A simple improvement would be to use the sparse Cholesky decomposition ofCHOLMOD
viascikitsparse
when available.Other potential improvements for combatting issues associated with dense columns in otherwise sparse problems are outlined in [4] Section 5.3 and [10] Section 4.14.2; the latter also discusses the alleviation of accuracy issues associated with the substitution approach to free variables.
After calculating the search direction, the maximum possible step size that does not activate the nonnegativity constraints is calculated, and the smaller of this step size and unity is applied (as in [4] Section 4.1.) [4] Section 4.3 suggests improvements for choosing the step size.
The new point is tested according to the termination conditions of [4] Section 4.5. The same tolerance, which can be set using the
tol
option, is used for all checks. (A potential improvement would be to expose the different tolerances to be set independently.) If optimality, unboundedness, or infeasibility is detected, the solve procedure terminates; otherwise it repeats.The expected problem formulation differs between the top level
linprog
module and the method specific solvers. The method specific solvers expect a problem in standard form:Minimize:
c @ x
Subject to:
A @ x == b x >= 0
Whereas the top level
linprog
module expects a problem of form:Minimize:
c @ x
Subject to:
A_ub @ x <= b_ub A_eq @ x == b_eq lb <= x <= ub
where
lb = 0
andub = None
unless set inbounds
.The original problem contains equality, upperbound and variable constraints whereas the method specific solver requires equality constraints and variable nonnegativity.
linprog
module converts the original problem to standard form by converting the simple bounds to upper bound constraints, introducing nonnegative slack variables for inequality constraints, and expressing unbounded variables as the difference between two nonnegative variables.References
[4] (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18) Andersen, Erling D., and Knud D. Andersen. “The MOSEK interior point optimizer for linear programming: an implementation of the homogeneous algorithm.” High performance optimization. Springer US, 2000. 197232. [6] (1, 2, 3) Freund, Robert M. “PrimalDual InteriorPoint Methods for Linear Programming based on Newton’s Method.” Unpublished Course Notes, March 2004. Available 2/25/2017 at https://ocw.mit.edu/courses/sloanschoolofmanagement/15084jnonlinearprogrammingspring2004/lecturenotes/lec14_int_pt_mthd.pdf [8] (1, 2) Andersen, Erling D., and Knud D. Andersen. “Presolving in linear programming.” Mathematical Programming 71.2 (1995): 221245. [9] Bertsimas, Dimitris, and J. Tsitsiklis. “Introduction to linear programming.” Athena Scientific 1 (1997): 997. [10] (1, 2) Andersen, Erling D., et al. Implementation of interior point methods for large scale linear programming. HEC/Universite de Geneve, 1996.