Skip to main content

t-SNE implementation in scikit-learn

t-Distributed Stochastic Neighbor Embedding (t-SNE) is a powerful tool for visualizing high-dimensional data. It reduces the dimensions of a dataset while preserving the relationships between points, making it easier to detect patterns and clusters. In this article, we will implement t-SNE using scikit-learn, a widely-used Python library for machine learning.

1. Introduction​

t-SNE is commonly used for visualizing datasets with a large number of features or high-dimensional data. It is particularly useful in fields such as bioinformatics, natural language processing, and image recognition, where understanding the structure of the data can lead to valuable insights.

In this example, we will apply t-SNE to the famous MNIST dataset, which contains images of handwritten digits. We will visualize how t-SNE reduces the dimensions of the dataset, allowing us to see the structure and clustering of different digits.

2. Importing necessary libraries​

We start by importing the necessary libraries, including scikit-learn for the t-SNE implementation and matplotlib for visualization.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_digits
from sklearn.manifold import TSNE
from sklearn.preprocessing import StandardScaler

3. Loading and preprocessing the dataset​

We will use the MNIST dataset, which is available in scikit-learn. The dataset contains 8x8 pixel images of handwritten digits, along with their corresponding labels.

# Load the digits dataset
digits = load_digits()
X = digits.data
y = digits.target

# Standardize the data
scaler = StandardScaler()
X_scaled = scaler.fit_transform(X)

Explanation:​

  • load_digits: Loads the MNIST dataset.
  • StandardScaler: Standardizes the dataset by removing the mean and scaling to unit variance, which helps improve the performance of t-SNE.

4. Applying t-SNE​

Next, we apply t-SNE to reduce the dimensions of the dataset from 64 (8x8 pixels) to 2 dimensions for visualization.

# Apply t-SNE
tsne = TSNE(n_components=2, random_state=42)
X_tsne = tsne.fit_transform(X_scaled)

Explanation:​

  • TSNE: Initializes the t-SNE model.
  • n_components=2: Specifies that we want to reduce the data to 2 dimensions.
  • random_state=42: Sets a random seed for reproducibility.
  • fit_transform: Fits the model to the data and transforms it into the lower-dimensional space.

5. Visualizing the results​

We can now visualize the 2D representation of the dataset. Each point in the plot corresponds to an image in the dataset, and we will color the points according to their digit label.

# Plot the t-SNE result
plt.figure(figsize=(10, 7))
scatter = plt.scatter(X_tsne[:, 0], X_tsne[:, 1], c=y, cmap='viridis', s=50, alpha=0.7)
plt.colorbar(scatter, label='Digit Label')
plt.title('t-SNE Visualization of MNIST Digits')
plt.xlabel('t-SNE Dimension 1')
plt.ylabel('t-SNE Dimension 2')
plt.grid(True)
plt.show()

Explanation:​

  • scatter: Creates a scatter plot of the t-SNE results.
  • c=y: Colors the points based on their digit label.
  • cmap='viridis': Specifies the colormap for coloring the points.
  • alpha=0.7: Sets the transparency of the points to make overlapping points more visible.
  • plt.colorbar: Adds a colorbar to the plot for reference.

6. Interpreting the results​

The resulting plot shows the 2D representation of the MNIST digits. Points that are close to each other in the plot correspond to images that are similar in the original high-dimensional space. Clusters of points often represent digits that are similar in shape, indicating that t-SNE has successfully captured the structure of the data.

7. Conclusion​

t-SNE is a powerful technique for visualizing high-dimensional data in 2D or 3D. In this article, we demonstrated how to apply t-SNE using scikit-learn and visualize the results using the MNIST dataset. By carefully tuning the parameters of t-SNE and preprocessing the data, you can gain valuable insights into the structure of your datasets.