t-SNE visualization of CNN codes

I took 50,000 ILSVRC 2012 validation images, extracted the 4096-dimensional fc7 CNN (Convolutional Neural Network) features using Caffe and then used Barnes-Hut t-SNE to compute a 2-dimensional embedding that respects the high-dimensional (L2) distances. In other words, t-SNE arranges images that have a similar CNN (fc7) code nearby in the embedding.

Embeddings where images are displayed exactly at their embedded location:


And below, embeddings where every position is filled with its nearest neighbor. Note that since the actual embedding is roughly circular, this leads to a visualization where the corners are a little "stretched" out and over-represented:


It's impossible to precisely embed 4096-dimensional space in 2 dimensions so in this final visualization, I take the 4000x4000 image and also draw the "seams", which measure the actual (L2) distance between the full 4096-dimensional codes of neighboring images in the grid. A bright red edge means the distance is high, and a black edge means the distance is low, in the original space.