Can Shapley/Lime values be used for unsupervised learning?

Question

One thing that is really useful when trying to understand what a machine learning model does, is seeing why some instances got predicted. For that Shapley Values and Lime are really usefull. But can they be used with unsupervised learning?
Let's say we are doing anomaly detection with tabular data and we run some algorithm like Isolation Forest (or any other).
Is it conceptually right to use Shapley or Lime to try to give a local explanation of the results when using unsupervised learning?

Sean Owen · Answer

I don't think so, not directly. SHAP is trying to explain each feature's effect on the prediction, but you have no label here. It might be better to ask therefore, what are you trying to explain?
In the case of an isolation forest, you can find the short path through the trees to any anomaly. That path tells you why the trees separated it, based on what features. It may not be super interpretable, but can be read off directly.

Can Shapley/Lime values be used for unsupervised learning?

One Answer

Add your own answers!

Ask a Question