1 From distances to a map

From travel times to a map

MDS sees only the travel times and draws a ring in driving order. With a long wait at the construction site, the ring becomes a line.

What MDS sees: the distance matrix \(\mathbf D\)

The entry \(d_{st}\) is the travel time between places \(s\) and \(t\). The outlined trips pass the construction site.

Can we get a map back from \(\mathbf D\)?

Here \(\mathbf D\) holds the 1,128 distances between the 48 state centres. MDS gets the shape right, but upside down.

Every turn fits the same \(\mathbf D\)

Turning the map changes no distance.

So \(\mathbf D\) cannot tell which way is north.

It fixes the map only up to a turn and a flip.

In three dimensions: three angles

None of the three turns changes a distance. So three facts are needed to pin the plane down.

2 How MDS draws the map

From distances to inner products

  • The inner product multiplies the coordinates and adds them up, as in \(b_{13}=-8-2=-10\).
  • Distances and inner products are linked, \(d_{st}^2=b_{ss}+b_{tt}-2b_{st}\).
  • So the inner products \(\mathbf B\) follow from \(\mathbf D\) alone.

MDS returns the PCA scores

The eigenvectors of \(\mathbf B\) give the map. However the map is turned (left), its PCA scores stay put (right). They are exactly the map that MDS returns.

The roommates need no first step

Random pairing gives the roommates no order.

So their GPA axes lie at 45ยฐ to the axes of the cloud, and the map alone gives the peer effect, 0.20.

With an order, as in Cholesky, the map must first be turned into place.

3 Other distances: profiles and text

The states as profiles

Each state is a point in six dimensions, one per indicator. Mahalanobis distances remove the correlations, so the cloud turns round.

Distances from text

\(d_{st}\) is how many words apart \(s\) and \(t\) appear in a text, on average.

Coke and Pepsi are 3.2 words apart. Coke and runway are 22.6.

The embedding

MDS turns these distances into coordinates, the embedding.

Each theme forms its own group. MDS was never told the themes.

One-hot encoding, by contrast, puts every two words the same distance apart.

4 When no map fits

Four places on a loop: no map fits

A square gets either the sides or the diagonals right, never both. No number of dimensions helps.

Negative eigenvalues

A negative eigenvalue means that no map reproduces the table exactly.

The construction site in eigenvalues

As the wait grows, one eigenvalue takes over. From 27.9 minutes on, a line fits exactly.

The Shepard diagram

Each point is one pair of observations. On a perfect map, all points lie on the diagonal.

Non-metric MDS

It keeps only the order of the distances. For the loop road, that order fits a circle exactly.

Takeaways

  • MDS turns a distance matrix into a map.
  • The map is fixed only up to a turn and a flip. MDS returns the PCA scores.
  • Embeddings are such maps for words.
  • Negative eigenvalues mean that no map fits exactly.

MDS draws the map that a table of distances describes.