Sentiment Extractor - The transformer

The problem here is to extract group of words or phrase from a given sentence which strongly supports the corresponding given sentiment. So unlike traditional text analysis task, here group of words are to be predicted. Inspired from the idea of how we train a object detector, I have done the same here. I have trained a transformer to identify one hots of ‘boundaries’. For a traditional image detection, we will predict four parameters x,y,w,h. In the similar manner I tried to predict the one hots of starting and ending words of phrase to be extracted. Focal Loss is used to train the model. A variety of transformers such as XLNet, RoBERTa, alBERT are trained and an ensemble of these models are considered to boost the performance.

want to know more? check out my github repo.