How to set the Reconstruction error threshold for anomaly detection using autoencoders?

Question

Hi I am doing anomaly detection using auto encoders.I have trained the model using 'Non Anomalous' values.Now when I give anomalous points as test data.

What should be the Reconstruction error threshold I should give to classify it as whether it is anomalous or not?

I have currently set it to 0.1 , but it doesnt find any anomalies(actually test data have lot of anomalies)
What are the factors that I should consider before setting the value?

lebedov · Answer

If one considers prediction of anomalous status as binary classification (i.e., if reconstruction error < threshold, classify as normal, else classify as anomalous), one can find the threshold that maximizes some appropriate metric of classification performance (e.g., F-beta) by optimizing the metric over a suitable validation set containing normal and anomalous data. See Malhotra et al., 2016 for an example of how to do this for time series.

Answered by lebedov on September 28, 2021

How to set the Reconstruction error threshold for anomaly detection using autoencoders?

One Answer

Add your own answers!

Ask a Question