NLP SBert (Bert) for answer comparison STS

I tried GPT-2 with your prompt but I was not extremely successful:

I have used Siamese Bert and I can say it does a pretty good job. However, the issue is that the data that it has been fine-tuned atop of Bert may not necessarily, entirely represent the same semantic distance as with the answers between the true and the student's one. For instance, if there is a question about engineering, where a small change of word may mean a totally different thing; SBert would still find them quite similar cause they are related to the topic. Unless it's fine-tuned.

Moreover, you will not be able to interpret the similarity. Should a student ask you why my peer's answer is better you won't be able to explain.

My opinion: I believe you could use this tool as a way to reduce totally incoherent answers, but at some point, human evaluation will be needed. And maybe use interpretable metrics such as ROUGE or BLEU. I aware as well, that this topic is quite trendy in NLP, I won't be surprised if there is or will be good off the shelf tool for that, but I am not aware of one currently.

@b_the_builder Nice finds! The first seems to me like an advancement of the Word Mower's distance by using the similarities between each word. I believe still may lack the domain adaptation. Whereas the second link you provided does the pre-training for that specific reason. All in all, whatever method you use I believe you will need to pick some representative hard match sentences and see how they perform on them, after pre-training on your corpora. If you want to be sure. For inspiration, you can look into here on semantic similarity tasks between sentences.