Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visualization

Понимание принципов работы методов снижения размерности: эмпирический подход к интерпретации t-SNE, UMAP, TriMAP и PaCMAP для визуализации данных
Cynthia D Rudin, Haiyang Huang, Yaron Shaposhnik, Yingfan Wang
2020-12-08

PaCMAPUMAPdata visualizationdimension reductiont-SNE
Dimension reduction (DR) techniques such as t-SNE, UMAP, and TriMAP have demonstrated impressive visualization performance on many real world datasets. One tension that has always faced these methods is the trade-off between preservation of global structure and preservation of local structure: these methods can either handle one or the other, but not both. In this work, our main goal is to understand what aspects of DR methods are important for preserving both local and global structure: it is difficult to design a better method without a true understanding of the choices we make in our algorithms and their empirical impact on the lower-dimensional embeddings they produce. Towards the goal of local structure preservation, we provide several useful design principles for DR loss functions based on our new understanding of the mechanisms behind successful DR methods. Towards the goal of global structure preservation, our analysis illuminates that the choice of which components to preserve is important. We leverage these insights to design a new algorithm for DR, called Pairwise Controlled Manifold Approximation Projection (PaCMAP), which preserves both local and global structure. Our work provides several unexpected insights into what design choices both to make and avoid when constructing DR algorithms.
1
It identifies design principles for dimensional-reduction loss functions that improve preservation of local structure.
2
The analysis shows that selecting which data components to preserve is critical for maintaining global structure.
3
The authors introduce PaCMAP, a dimensional-reduction algorithm designed to preserve both local and global structure.
4
The study empirically analyzes how t-SNE, UMAP, and TriMAP choices affect local and global structure preservation in low-dimensional embeddings.
5
The work reports unexpected design choices and pitfalls that should guide the construction of future dimensional-reduction methods.

dimension reduction methods and their resulting lower-dimensional data embeddings, including t-SNE, UMAP, TriMAP, and PaCMAP

the empirical mechanisms and design choices governing the preservation of local and global structure in dimension-reduction embeddings

Publication Details
Publication Date
2020-12-08
Journal
Publisher
ISSN
Cited by
252
Access Type
Author Information
Authors
Cynthia D Rudin
Haiyang Huang
Yaron Shaposhnik
Yingfan Wang
Explore further
Open the scid.ai AI chat with a ready-made request: it will find papers on a similar topic and help build a literature review.
Find similar papers in the chat →
Make a presentation
100%