Distance matrix computation from a collection of raw observation vectors stored in a rectangular array.
Function | Description |
pdist | pairwise distances between observation vectors. |
cdist | distances between between two collections of observation vectors. |
squareform | converts a square distance matrix to a condensed one and vice versa. |
Predicates for checking the validity of distance matrices, both condensed and redundant. Also contained in this module are functions for computing the number of observations in a distance matrix.
Function | Description |
is_valid_dm | checks for a valid distance matrix. |
is_valid_y | checks for a valid condensed distance matrix. |
num_obs_dm | # of observations in a distance matrix. |
num_obs_y | # of observations in a condensed distance matrix. |
Distance functions between two vectors u and v. Computing distances over a large collection of vectors is inefficient for these functions. Use pdist for this purpose.
Function | Description |
braycurtis | the Bray-Curtis distance. |
canberra | the Canberra distance. |
chebyshev | the Chebyshev distance. |
cityblock | the Manhattan distance. |
correlation | the Correlation distance. |
cosine | the Cosine distance. |
dice | the Dice dissimilarity (boolean). |
euclidean | the Euclidean distance. |
hamming | the Hamming distance (boolean). |
jaccard | the Jaccard distance (boolean). |
kulsinski | the Kulsinski distance (boolean). |
mahalanobis | the Mahalanobis distance. |
matching | the matching dissimilarity (boolean). |
minkowski | the Minkowski distance. |
rogerstanimoto | the Rogers-Tanimoto dissimilarity (boolean). |
russellrao | the Russell-Rao dissimilarity (boolean). |
seuclidean | the normalized Euclidean distance. |
sokalmichener | the Sokal-Michener dissimilarity (boolean). |
sokalsneath | the Sokal-Sneath dissimilarity (boolean). |
sqeuclidean | the squared Euclidean distance. |
yule | the Yule dissimilarity (boolean). |
[Sta07] | “Statistics toolbox.” API Reference Documentation. The MathWorks. http://www.mathworks.com/access/helpdesk/help/toolbox/stats/. Accessed October 1, 2007. |
[Mti07] | “Hierarchical clustering.” API Reference Documentation. The Wolfram Research, Inc. http://reference.wolfram.com/mathematica/HierarchicalClustering/tutorial/HierarchicalClustering.html. Accessed October 1, 2007. |
[Gow69] | Gower, JC and Ross, GJS. “Minimum Spanning Trees and Single Linkage Cluster Analysis.” Applied Statistics. 18(1): pp. 54–64. 1969. |
[War63] | Ward Jr, JH. “Hierarchical grouping to optimize an objective function.” Journal of the American Statistical Association. 58(301): pp. 236–44. 1963. |
[Joh66] | Johnson, SC. “Hierarchical clustering schemes.” Psychometrika. 32(2): pp. 241–54. 1966. |
[Sne62] | Sneath, PH and Sokal, RR. “Numerical taxonomy.” Nature. 193: pp. 855–60. 1962. |
[Bat95] | Batagelj, V. “Comparing resemblance measures.” Journal of Classification. 12: pp. 73–90. 1995. |
[Sok58] | Sokal, RR and Michener, CD. “A statistical method for evaluating systematic relationships.” Scientific Bulletins. 38(22): pp. 1409–38. 1958. |
[Ede79] | Edelbrock, C. “Mixture model tests of hierarchical clustering algorithms: the problem of classifying everybody.” Multivariate Behavioral Research. 14: pp. 367–84. 1979. |
[Jai88] | Jain, A., and Dubes, R., “Algorithms for Clustering Data.” Prentice-Hall. Englewood Cliffs, NJ. 1988. |
[Fis36] | Fisher, RA “The use of multiple measurements in taxonomic problems.” Annals of Eugenics, 7(2): 179-188. 1936 |
Copyright (C) Damian Eads, 2007-2008. New BSD License.
Computes the Bray-Curtis distance between two n-vectors u and v, which is defined as
Parameters: |
|
---|---|
Returns: |
|
Computes the Canberra distance between two n-vectors u and v, which is defined as
Parameters: |
|
---|---|
Returns: |
|
Computes distance between each pair of observation vectors in the Cartesian product of two collections of vectors. XA is a by array while XB is a by array. A by array is returned. An exception is thrown if XA and XB do not have the same number of columns.
A rectangular distance matrix Y is returned. For each and , the metric dist(u=XA[i], v=XB[j]) is computed and stored in the th entry.
The following are common calling conventions:
Y = cdist(XA, XB, 'euclidean')
Computes the distance between points using Euclidean distance (2-norm) as the distance metric between the points. The points are arranged as -dimensional row vectors in the matrix X.
Y = cdist(XA, XB, 'minkowski', p)
Computes the distances using the Minkowski distance (-norm) where .
Y = cdist(XA, XB, 'cityblock')
Computes the city block or Manhattan distance between the points.
Y = cdist(XA, XB, 'seuclidean', V=None)
Computes the standardized Euclidean distance. The standardized Euclidean distance between two n-vectors u and v is
the i’th components of the points. If not passed, it is automatically computed.
Y = cdist(XA, XB, 'sqeuclidean')
Computes the squared Euclidean distance between the vectors.
Y = cdist(XA, XB, 'cosine')
Computes the cosine distance between vectors u and v,
where is the 2-norm of its argument *.
Y = cdist(XA, XB, 'correlation')
Computes the correlation distance between vectors u and v. This is
where is the Manhattan (or 1-norm) of its argument, and is the common dimensionality of the vectors.
Y = cdist(XA, XB, 'hamming')
Computes the normalized Hamming distance, or the proportion of those vector elements between two n-vectors u and v which disagree. To save memory, the matrix X can be of type boolean.
Y = cdist(XA, XB, 'jaccard')
Computes the Jaccard distance between the points. Given two vectors, u and v, the Jaccard distance is the proportion of those elements u[i] and v[i] that disagree where at least one of them is non-zero.
Y = cdist(XA, XB, 'chebyshev')
Computes the Chebyshev distance between the points. The Chebyshev distance between two n-vectors u and v is the maximum norm-1 distance between their respective elements. More precisely, the distance is given by
Computes the Canberra distance between the points. The Canberra distance between two points u and v is
Computes the Bray-Curtis distance between the points. The Bray-Curtis distance between two points u and v is
Computes the Mahalanobis distance between the points. The Mahalanobis distance between two points u and v is where (the VI variable) is the inverse covariance. If VI is not None, VI will be used as the inverse covariance matrix.
Computes the Yule distance between the boolean vectors. (see yule function documentation)
Computes the matching distance between the boolean vectors. (see matching function documentation)
Computes the Dice distance between the boolean vectors. (see dice function documentation)
Computes the Kulsinski distance between the boolean vectors. (see kulsinski function documentation)
Computes the Rogers-Tanimoto distance between the boolean vectors. (see rogerstanimoto function documentation)
Computes the Russell-Rao distance between the boolean vectors. (see russellrao function documentation)
Computes the Sokal-Michener distance between the boolean vectors. (see sokalmichener function documentation)
Computes the Sokal-Sneath distance between the vectors. (see sokalsneath function documentation)
Computes the weighted Minkowski distance between the vectors. (see sokalsneath function documentation)
Computes the distance between all pairs of vectors in X using the user supplied 2-arity function f. For example, Euclidean distance between the vectors could be computed as follows:
dm = cdist(XA, XB, (lambda u, v: np.sqrt(((u-v)*(u-v).T).sum())))Note that you should avoid passing a reference to one of the distance functions defined in this library. For example,:
dm = cdist(XA, XB, sokalsneath)would calculate the pair-wise distances between the vectors in X using the Python function sokalsneath. This would result in sokalsneath being called times, which is inefficient. Instead, the optimized C version is more efficient, and we call it using the following syntax.:
dm = cdist(XA, XB, 'sokalsneath')
Parameters: |
|
---|---|
Returns: |
|
Computes the Chebyshev distance between two n-vectors u and v, which is defined as
Parameters: |
|
---|---|
Returns: |
|
Computes the Manhattan distance between two n-vectors u and v, which is defined as
Parameters: |
|
---|---|
Returns: |
|
Computes the correlation distance between two n-vectors u and v, which is defined as
where is the mean of a vectors elements and n is the common dimensionality of u and v.
Parameters: |
|
---|---|
Returns: |
|
Computes the Cosine distance between two n-vectors u and v, which is defined as
Parameters: |
|
---|---|
Returns: |
|
Computes the Dice dissimilarity between two boolean n-vectors u and v, which is
where is the number of occurrences of and for .
Parameters: |
|
---|---|
Returns: |
|
Computes the Euclidean distance between two n-vectors u and v, which is defined as
Parameters: |
|
---|---|
Returns: |
|
Computes the Hamming distance between two n-vectors u and v, which is simply the proportion of disagreeing components in u and v. If u and v are boolean vectors, the Hamming distance is
where is the number of occurrences of and for .
Parameters: |
|
---|---|
Returns: |
|
Returns True if the variable D passed is a valid distance matrix. Distance matrices must be 2-dimensional numpy arrays containing doubles. They must have a zero-diagonal, and they must be symmetric.
Parameters: |
|
---|---|
Returns: | Returns True if the variable D passed is a valid distance matrix. Small numerical differences in D and D.T and non-zeroness of the diagonal are ignored if they are within the tolerance specified by tol. |
Returns True if the variable y passed is a valid condensed distance matrix. Condensed distance matrices must be 1-dimensional numpy arrays containing doubles. Their length must be a binomial coefficient for some positive integer n.
Parameters: |
|
---|
Computes the Jaccard-Needham dissimilarity between two boolean n-vectors u and v, which is
where is the number of occurrences of and for .
Parameters: |
|
---|---|
Returns: |
|
Computes the Kulsinski dissimilarity between two boolean n-vectors u and v, which is defined as
where is the number of occurrences of and for .
Parameters: |
|
---|---|
Returns: |
|
Computes the Mahalanobis distance between two n-vectors u and v, which is defiend as
where VI is the inverse covariance matrix .
Parameters: |
|
---|---|
Returns: |
|
Computes the Matching dissimilarity between two boolean n-vectors u and v, which is defined as
where is the number of occurrences of and for .
Parameters: |
|
---|---|
Returns: |
|
Computes the Minkowski distance between two vectors u and v, defined as
Parameters: |
|
---|---|
Returns: |
|
Returns the number of original observations that correspond to a square, redudant distance matrix D.
Parameters: |
|
---|---|
Returns: | The number of observations in the redundant distance matrix. |
Returns the number of original observations that correspond to a condensed distance matrix Y.
Parameters: |
|
---|---|
Returns: |
|
Computes the pairwise distances between m original observations in n-dimensional space. Returns a condensed distance matrix Y. For each and (where ), the metric dist(u=X[i], v=X[j]) is computed and stored in the :math:`ij`th entry.
See squareform for information on how to calculate the index of this entry or to convert the condensed distance matrix to a redundant square matrix.
The following are common calling conventions.
Y = pdist(X, 'euclidean')
Computes the distance between m points using Euclidean distance (2-norm) as the distance metric between the points. The points are arranged as m n-dimensional row vectors in the matrix X.
Y = pdist(X, 'minkowski', p)
Computes the distances using the Minkowski distance (p-norm) where .
Y = pdist(X, 'cityblock')
Computes the city block or Manhattan distance between the points.
Y = pdist(X, 'seuclidean', V=None)
Computes the standardized Euclidean distance. The standardized Euclidean distance between two n-vectors u and v is
the i’th components of the points. If not passed, it is automatically computed.
Y = pdist(X, 'sqeuclidean')
Computes the squared Euclidean distance between the vectors.
Y = pdist(X, 'cosine')
Computes the cosine distance between vectors u and v,
Y = pdist(X, 'correlation')
Computes the correlation distance between vectors u and v. This is
where is the mean of the elements of vector v.
Y = pdist(X, 'hamming')
Computes the normalized Hamming distance, or the proportion of those vector elements between two n-vectors u and v which disagree. To save memory, the matrix X can be of type boolean.
Y = pdist(X, 'jaccard')
Computes the Jaccard distance between the points. Given two vectors, u and v, the Jaccard distance is the proportion of those elements u[i] and v[i] that disagree where at least one of them is non-zero.
Y = pdist(X, 'chebyshev')
Computes the Chebyshev distance between the points. The Chebyshev distance between two n-vectors u and v is the maximum norm-1 distance between their respective elements. More precisely, the distance is given by
Computes the Canberra distance between the points. The Canberra distance between two points u and v is
Computes the Bray-Curtis distance between the points. The Bray-Curtis distance between two points u and v is
Computes the Mahalanobis distance between the points. The Mahalanobis distance between two points u and v is where (the VI variable) is the inverse covariance. If VI is not None, VI will be used as the inverse covariance matrix.
Computes the Yule distance between each pair of boolean vectors. (see yule function documentation)
Computes the matching distance between each pair of boolean vectors. (see matching function documentation)
Computes the Dice distance between each pair of boolean vectors. (see dice function documentation)
Computes the Kulsinski distance between each pair of boolean vectors. (see kulsinski function documentation)
Computes the Rogers-Tanimoto distance between each pair of boolean vectors. (see rogerstanimoto function documentation)
Computes the Russell-Rao distance between each pair of boolean vectors. (see russellrao function documentation)
Computes the Sokal-Michener distance between each pair of boolean vectors. (see sokalmichener function documentation)
Computes the Sokal-Sneath distance between each pair of boolean vectors. (see sokalsneath function documentation)
Computes the weighted Minkowski distance between each pair of vectors. (see wminkowski function documentation)
Computes the distance between all pairs of vectors in X using the user supplied 2-arity function f. For example, Euclidean distance between the vectors could be computed as follows:
dm = pdist(X, (lambda u, v: np.sqrt(((u-v)*(u-v).T).sum())))Note that you should avoid passing a reference to one of the distance functions defined in this library. For example,:
dm = pdist(X, sokalsneath)would calculate the pair-wise distances between the vectors in X using the Python function sokalsneath. This would result in sokalsneath being called times, which is inefficient. Instead, the optimized C version is more efficient, and we call it using the following syntax.:
dm = pdist(X, 'sokalsneath')
Parameters: |
|
---|---|
Returns: |
|
Seealso: |
|
Computes the Rogers-Tanimoto dissimilarity between two boolean n-vectors u and v, which is defined as
where is the number of occurrences of and for and .
Parameters: |
|
---|---|
Returns: |
|
Computes the Russell-Rao dissimilarity between two boolean n-vectors u and v, which is defined as
where is the number of occurrences of and for .
Parameters: |
|
---|---|
Returns: |
|
Returns the standardized Euclidean distance between two n-vectors u and v. V is an m-dimensional vector of component variances. It is usually computed among a larger collection vectors.
Parameters: |
|
---|---|
Returns: |
|
Computes the Sokal-Michener dissimilarity between two boolean vectors u and v, which is defined as
where is the number of occurrences of and for , and .
Parameters: |
|
---|---|
Returns: |
|
Computes the Sokal-Sneath dissimilarity between two boolean vectors u and v,
where is the number of occurrences of and for and .
Parameters: |
|
---|---|
Returns: |
|
Computes the squared Euclidean distance between two n-vectors u and v, which is defined as
Parameters: |
|
---|---|
Returns: |
|
Converts a vector-form distance vector to a square-form distance matrix, and vice-versa.
Parameters: |
|
---|---|
Returns: |
|
Computes the weighted Minkowski distance between two vectors u and v, defined as
Parameters: |
|
---|---|
Returns: |
|
Computes the Yule dissimilarity between two boolean n-vectors u and v, which is defined as
where is the number of occurrences of and for and .
Parameters: |
|
---|---|
Returns: |
|