What are some non-toy applications of autoencoders?

Question

I haven't come across a real world application of autoencoders before. Usually, for dimensionality reduction I've used PCA or random projections instead.

Most examples I've come across of using autoencoders for dimensionality reduction are usually toy problems. For example, training an autoencoder on MNIST to use logistic regression as the final classifier. I wouldn't call this a practical application since usually there are more relevant vision models that you could apply instead of logistic regression (especially for datasets more complex than MNIST).

What are some non-toy examples of applications of autoencoders (over other dimensionality reduction techniques)? I'm particularly interested in applications on tabular datasets or datasets with sparse features. References to papers, blog posts or anecdotes would all be helpful.

Chris_Rands · Answer

One increasingly popular biological area of application for autoenconders is single cell transcriptomics, which typically generates large sparse data matrixes. Here autoencoders have been applied for both de-noising purposes and rapid dimensionality reduction.

Answered by Chris_Rands on November 2, 2021

Thomas Lumley · Answer

One statistical application of denoising autoencoders is multiple imputation: the autoencoder tries to compress the data to a low-dimensional signal (that isn't missing) plus noise (that's sometimes missing). Compared to either Bayesian data augmentation or the popular 'mice' algorithms, the autoencoders seem to scale better to large numbers of variables, and may potentially handle nonlinearity and interaction better. (This is still a research area, but it's a serious application.)
Andrew Gelman writes about an early attempt here, and the current version of that specific project is here

Sandip M · Answer

One of the application of auto-encoder that i am exploring is for building content-based image search engine.

Training auto-encoder network on product catalogue data ( images )
Extract encoder layer from the trained model and encode images [ based on latent dimensions ]
Index the encoded features of images
During query, encode query image features and search in indexes for finding "similar" images.

OmG · Answer

From the Autoencoder Wikipedia article:

One milestone paper on the subject was that of Geoffrey Hinton with his publication in Science Magazine in 2006 [Reducing the Dimensionality of Data with Neural Networks by     G. E. Hinton and et al.]: in that study, he pretrained a multi-layer autoencoder with a stack of RBMs and then used their weights to initialize a deep autoencoder with gradually smaller hidden layers until a bottleneck of 30 neurons. The resulting 30 dimensions of the code yielded a smaller reconstruction error compared to the first 30 principal components of a PCA, and learned a representation that was qualitatively easier to interpret, clearly separating clusters in the original data.

What are some non-toy applications of autoencoders?

4 Answers

Add your own answers!

Ask a Question