Evolution of Sentiment Analysis in Tweets and Aspect-Based Sentiment Analysis
The evolution of sentiment analysis on tweets from SemEval competitions in 2013 to 2017 is discussed, showcasing advancements in technology and the shift from SVM and sentiment lexicons to CNN with word embeddings. Aspect-Based Sentiment Analysis, as explored in SemEval2014, involves determining aspect terms, categories, and sentiments from customer reviews, with specific subtasks focusing on different aspects like term extraction and category detection.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. Download presentation by click this link. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
E N D
Presentation Transcript
Introduction SemEval Shared Task Competition 2013, Task 2 2014, Task 9 2015 2016 2017 Evolution of technology: Top system in 2013: SVM with sentiment lexicons and many lexical features Top system in 2016: CNN with word embeddings In 2017: most systems used CNN or variants
SemEval2013, Task 2 TNRC-Canada: Building the State-of-the-Art in Sentiment Analysis of Tweets, Saif M. Mohammad, Svetlana Kiritchenko, and Xiaodan Zhu, In Proceedings of the seventh international workshop on Semantic Evaluation Exercises (SemEval-2013), June 2013, Atlanta, USA.
Aspect Terms The lasagna was great, but we had to wait 20 minutes just to be seated. Aspect term: lasagna (positive sentiment)
Aspect Categories The lasagna was great, but we had to wait 20 minutes just to be seated. Aspect categories: food (positive sentiment), service (negative sentiment)
SemEval2014 SemEval-2014 Task 4 (Aspect Based Sentiment Analysis) given a customer review, automatic systems are to determine aspect terms, aspect categories, and sentiment towards these aspect terms and categories Training data Two domain-specific datasets for laptops and restaurants, consisting of over 6K sentences with fine- grained aspect-level human annotations.
Subtasks 1. Aspect Term Extraction 2. Aspect Term Polarity 3. Aspect Category Detection customer reviews provided for two domains: restaurants and laptops five aspect categories is defined for the restaurant domain: food, service, price, ambiance, and anecdotes. no aspect category for the laptop reviews Aspect Category Polarity Automatic systems are to determine if any of those aspect categories are described in a review
NRC-Canada NRC-Canada-2014: Detecting Aspects and Sentiment in Customer Reviews, Svetlana Kiritchenko, Xiaodan Zhu, Colin Cherry, and Saif M. Mohammad. In Proceedings of the eighth international workshop on Semantic Evaluation Exercises (SemEval-2014), August 2014, Dublin, Ireland. builds on the NRC-Canada sentiment analysis system which determines the overall sentiment of a message (top results in SemEval-2013 Task 2 and SemEval-2014 Task 9 on Sentiment Analysis of Tweets)
Resources In-domain corpora 180,000 Yelp restaurant reviews (Phoenix Academic dataset) 125,000 Amazon laptop reviews (McAuley & Leskovec, 2013) Sentiment lexicons - terms and degree of their association with positive or negative sentiment Word-Aspect Association lexicon - terms and degree of their association with the aspect categories
Corpora The Yelp Phoenix Academic Dataset contains customer reviews posted on the Yelp website. The businesses for which the reviews are posted are classified into over 500 categories Amazon laptop reviews corpus: McAuley and Leskovec (2013) collected reviews posted on Amazon.com. A subset of 124,712 reviews that mention either laptop or notebook 5 star ratings (1-2 negative, 4-5 positive)
Generation process 1. star ratings in reviews are used as weak labels 2. score(w) = PMI(w, positive) - PMI(w, negative) if score(w) > 0, then word w is positive if score(w) < 0, then word w is negative 3. affirmative and negated contexts are treated separately The desserts are very overpriced and not very tasty Sentiment of review Affirmative context Negated context Negated Context Corpus Affirmative context Corpus
Pointwise Mutual Information freq(?,???)? freq(?)freq(???) ??? ?,??? = log freq(w, pos) is the number of times a term w occurs in positive reviews freq(w) is the total frequency of term w in the corpus freq(pos) is the total number of tokens in positive reviews N is the total number of tokens in the corpus PMI(w, neg) calculated in a similar way ignored terms that occurred less than five times in each (positive and negative) groups of reviews
Negated Contexts Negated contexts (defined as text spans between a negation word and a punctuation mark) and affirmative (non-negated) contexts sentiment scores were then calculated separately for the two types of contexts the term goodin affirmative contexts has a sentiment score of 1.2 whereas the same term in negated contexts has a score of -1.4
Out Domain Sentiment Lexica large-coverage automatic tweet sentiment lexicons, Hashtag Sentiment lexicons and Sentiment140 lexicons (Kiritchenko et al., 2014) three manually created sentiment lexicons NRC Emotion Lexicon (Mohammad and Turney, 2010) Bing Liu s Lexicon (Hu and Liu, 2004) MPQA Subjectivity Lexicon (Wilson et al., 2005)
Yelp Restaurant WordAspect Association Lexicon Each sentence of the corpus was labeled with zero, one, or more of the five aspect categories by our aspect category classification system for each term w and each category c an association score was calculated: score(w, c) = PMI(w, c) PMI(w, c)
Subtask 1: Aspect Term Extraction Task: to detect aspect terms in a sentence Approach: Semi-Markov discriminative tagger, trained with MIRA (perceptron) tags phrases, not tokens, can use phrase-level features Features: emission features: token identity (cased, lowercased) in a 2-word window, prefixes and suffixes up to 3 chars, phrase identity (cased, lowercased) transition features: tag ngrams Results: Domain Restaurants Laptops P 84.41 78.77 R 78.37 60.70 F1 80.19 (3rd) 6.57 (3rd)
Tagger Features token feature templates for wi token-identity within a window (wi 2 ...wi+2) lower-cased token-identity within a window (lc(wi 2)...lc(wi+2)) prefixes and suffixes of wi (up to 3 characters in length) phrase-level emission feature templates: the cased and uncased identity of the entire phrase covered by a tag, which allow the system to memorize complete terms such as, getting a table or fish and chips. Transition features templates are short n-grams of tag identities: yj; yj,yj 1; and yj,yj 1,yj 2.
Subtask 2: Aspect Term Polarity Task: to detect sentiment towards a given aspect term Approach: SVM with linear kernel Features: surface features: ngrams, context-target bigrams sentiment lexicon features: counts, sum, max syntactic features: ngrams and context-target bigrams on parse trees, parse label features Results: Domain Restaurants Laptops Accuracy 80.16 (2nd) 70.49 (1st)
Features Surface features: unigrams (single words) and bigrams (2-word sequences) extracted from a term and its surface context context-target bigrams (i.e., bigrams formed by a word from the surface context and a word from the term itself) Lexicon features: the number of positive/negative tokens the sum of the tokens sentiment scores The maximal sentiments core Parse features: word- and POS-ngram context-target bigrams, i.e., bigrams composed of a word from the parse context and a word from the term all paths that start or end with the root of the target terms
Subtask 3: Aspect Category Detection Task: to detect aspect categories discussed in a sentence Approach: SVM with linear kernel five binary classifiers (one-vs-all) assign cmax= argmaxcP(c|d) if P(cmax|d > 0.4) Features: word and character ngrams stemmed ngrams (Porter stemmer) word cluster ngrams (Brown clustering algorithm) Yelp Restaurant Word-Aspect Association lexicon Results: Domain P Restaurants 91.04 R 86.24 F1 88.58 (1st)
Subtask 4: Aspect Category Polarity Task: to detect sentiment towards a given aspect category Approach: one 4-class SVM classifier with 2 copies of each feature: generic and category-specific add features for terms associated with aspect category Features: word and character ngrams, POS tags word cluster ngrams sentiment lexicon features Results: Domain Restaurants Accuracy 82.93 (1st)
Conclusions top results on subtasks 2, 3, and 4 statistical approaches with surface-form and lexicon features most useful features: derived from automatically generated in-domain lexical resources resources to download: www.purl.com/net/sentimentoftweets
