Explain FastText model using SHAP values

I have trained fastText model and some fully connected network build on its embeddings. I figured out how to use Lime on it: complete example can be found in Natural Language Processing Is Fun Part 3: Explaining Model Predictions

The idea is clear – put 1 sentence into Lime, it drop words and generate some new sentences from my and check how score changes.

My next idea – use SHAP values for this.
SHAP values can be used for any deep model, using DeepExplainer. Here is a usage example: Keras LSTM for IMDB Sentiment Classification

But I can’t use it for my ensemble, because DeepExplainer needs tensors as input, but I want to fed sentences. I don’t want to use BoW or TF-IDF for that – I loose fastText power in that situation.
What I want to achieve – get some shap-plots built on words of my sentence.
Is it possible?