Skip to content

Home

Jump to bottom
Farhaan Ahmed edited this page May 9, 2018 · 10 revisions

SENTIMENTAL ANALYSIS ON TWITTER

ABSTRACT

In today’s technological world, Social networking websites like Twitter, Instagram, Facebook, Tumblr, etc. play a very significant role. Emotion AI is about dealing, recognizing and analyzing sentiments or opinions conveyed in a person’s text. In particular, Emotion is most frequently called Sentiment analysis. It helps us to understand the people’s point of view. A vast amount of sentiment rich data is produced by Social networking websites in the form of posts, tweets, statuses, blogs etc. Some users post reviews of certain products in social media which influences customers to buy the product. Companies can use such review data analyze it and improve the product. Sentiment analysis of Twitter is troublesome correlated to other social networking websites because of the existence of a lot of short words, misspellings and slang words applying emotion analysis to such data is more challenging. We have classified the sentiment into 5 categories. Machine learning strategies are preferred mostly for analyzing emotion AI. We have used neural network model word2vec with TF-IDF approach to predict the sentiment of the tweet.

INTRODUCTION

Technology is developing day-by-day because of the Internet. The Internet is a worldwide communication network with this, usage of social websites like Twitter, Facebook, Instagram, Tumblr etc. have increased. In fact, social websites have become a famous place where everyone expresses their opinion about anything i.e. either it may be a product or article or society issues. Everyone has right to express their feeling and many of them think a social website is the best place to express their feelings in the form of the social networking blogs, Tweets, posts, statuses etc.

The Sentiment analysis focuses on the utilization of regular dialect handling, and furthermore message investigation of tremendous amounts of data. This sentimental analysis sometimes alluded as emotion Artificial Intelligence. It is the study of affective states and emotions present in the given information. The sentimental analysis is applied to various aspects of the web such as client audits and reviews. Sentiment Analysis or Emotion AI is a mechanism for ‘computationally’ resolving or recognizing if certain text is neutral, negative or positive. In general terms, sentimental analysis aims to find the attitude of a customer expressed in a text. Sentiment analysis is again called as Opinion mining that is computing the sentiment or opinion of the person. Sentiment analysis is used to establish the person's opinion towards a product or problem with the help of variables such as emotion, tone and context, etc. Business corporations value this approach to compute and investigate the public opinion of their products and company to improve the customer satisfaction towards them. Business corporations further use this opinion mining approach to collect critical opinion about a new product and identifying the problems in it. Sentiment analysis is used by the major multinational companies to obtain the impression of their products and with the help of that, they can design their business strategies. Twitter is one of the platforms where people post the opinion on major real-world subjects, current affairs, etc. The social media websites like Facebook, Twitter and Instagram produce trillions of megabytes of data 24/7. This vast amount of data is interpreted to figure out the sentiment of people on various problems. Although a lot of research is done in sentiment analysis our developed model is one of its first kind as we have taken multi-classes in emotions of sentiment analysis and predicted the emotion with the help of Machine learning models. Machine learning is simply training the machine with the Previous set of data and predicting the future data. Machine learning is the most booming technology these days and is applied in various fields. There are various machine learning techniques that can be applied to Sentiment analysis. Through our study of multi-class sentiment analysis, we found that distance calculation between words and by reconstructing the linguistic context of words we can predict the emotion more effectively.

PURPOSE OF CHOOSING TWITTER:

  • Twitter is a micro-blogging platform where anyone can read or write short messages which are called as Tweets.
  • The amount of data accumulated on twitter is very huge. This data is unstructured and written in natural language.
  • Twitter Sentimental Analysis is the process of accessing tweets for a particular topic and predicts the sentiment of these tweets as positive, negative or neutral with the help of machine learning.

proposed techniques

1. Euclidean Distance In order to calculate the distance between two points, Euclidean is used. Euclidean distance will play a significant role to calculate the distance between words. Euclidean distance is used in many of classification algorithm such as K-Nearest Neighbour, Minimum Distance Classifier, TF-IDF etc. let p1 at (x1, y1) and p2 at (x2, y2), in a plane then the Euclidean can be calculated with the help of Pythagoras theorem by:

2. TF-IDF Tf-idf is an acronym for term frequency-inverse document frequency, it’s a way to grade the significant terms (or "words") in a document based on how often these terms appear across the whole document and across multiple documents and the tf-idf weight usually used in data retrieval and text data mining. Computing sum of tf-idf for each term is one of the simplest approaches.

2.1 Term Frequency (TF) It computes how often a word or term appears in a record. Since every record is divergent in length, it is possible that the term appears more times in some documents than others. Term frequency can be calculated by how frequent a term t occurs in a record, normalized by dividing by the total number of terms in that record.

2.2 Inverse Document Frequency (IDF) It computes how common a word is among all records, at the same time measuring Term frequency, every word is scrutinized as equally essential. Despite, there are certain common words like "the", "for", "is", "of" and "that" which appear in many documents very frequently but have little significant. Therefore we compute IDF by:

3. WORD2VEC model Word2Vec is a cluster of models which helps derive associations between a term and its contextual terms or words. The cluster models are hollow, two-layer neural networks that are schooled to reestablish linguistic contexts of words. The two important model architectures inside are Skip-grams and CBOW. Word2vec model can be built with the help of Tensor flow, it is an open source machine learning framework developed by Google for research purpose.

3.1 Skip-gram In Skip-gram model, a center word and a window of context (neighbor) words are chosen and context words are predicted out to some window size for each center word. So, the model goes to outline a probability distribution i.e. probability of a word occurring in the context given a center word and vector representations is chosen to maximize the probability.

3.2 Continuous Bag of Words (CBOW) In Actual terms, this is a mirror of skip-gram. In CBOW, we predict center word by summing vectors of surrounding (Neighbour) words. Essentially, we begin with small random initialization of word vectors. In Word2vec, we have a large matrix with each row for the “words” and columns for the “context”. CBOW is not sequential and does not have to be probabilistic.

USES OF SENTIMENTAL ANALYSIS:

  • It is useful in industries like film industry etc.
  • Opinion of the mass is important.
  • Political party may want to know whether people support their program or not.
  • Before investing into a company, one can leverage the sentiment of the people for the company to find out where it stands.
  • A company might want find out there views of its products.

Proposed techniques PRE-REQUISITIES:

  1. pip3 install tweepy
  2. pip3 install textblob
  3. pip3 install nltk
  4. In terminal python3
  • import nltk
  • nltk.download(“stopwords”)
  • nltk.download(“punkt”)

Search API of Twitter

The Twitter Search API is part of Twitter’s REST API. It allows queries against the indices of recent or popular Tweets and behaves similarly to, but not exactly like the Search feature available in Twitter mobile or web clients, such as Twitter.com search. The Twitter Search API searches against a sampling of recent Tweets published in the past 7 days. Before getting involved, it’s important to know that the Search API is focused on relevance and not completeness. This means that some Tweets and users may be missing from search results. If you want to match for completeness you should consider using a Streaming API instead.

LIBRARIES USED:

TEXT BLOB:

TextBlob is a Python (2 and 3) library for processing textual data. It provides a simple API for diving into common natural language processing (NLP) tasks such as part-of-speech tagging, noun phrase extraction, sentiment analysis, classification, translation, and more. The sentiment property returns a named tuple of the form Sentiment (polarity, subjectivity). The polarity score is a float within the range [-1.0, 1.0]. The subjectivity is a float within the range [0.0, 1.0] where 0.0 is very objective and 1.0 is very subjective. TextBlob will attempt to detect the language. You can specify the source language explicitly, like so. Raises Translator Error if the TextBlob cannot be translated into the requested language or Not Translated if the translated result is the same as the input string. Text blob contains analysed data sets which has positive and negativity based on polarity range which helps for the users to find polarity and subjective when sentence is given.

TWEEPY:

With tweepy, it's possible to get any object and use any method that the official Twitter API offers.

NLTK (Natural Language Toolkit):

Natural Language Processing with Python provides a practical introduction to programming for language processing. Written by the creators of NLTK, it guides the reader through the fundamentals of writing Python programs, working with corpora, categorizing text, analyzing linguistic structure, and more.

nltk.corpus:

nltk.corpus package automatically creates a set of corpus reader instances that can be used to access the corpora in the NLTK data package. Section Corpus Reader Objects ("Corpus Reader Objects") describes the corpus reader instances that can be used to read the corpora in the NLTK data package.

Word Tokenizer:

Tokenizers are used to divide strings into lists of substrings. For example, Sentence tokenizer can be used to find the list of sentences and Word tokenizer can be used to find the list of words in strings.

Stop words:

Stopwords are words that are generally considered useless. Most search engines ignore these words because they are so common that including them would greatly increase the size of the index without improving precision or recall. NLTK comes with a stopwords corpus that includes a list of 128 english stopwords.

PROCESS:

  • Taking sentiment analysis into consideration tweets will analyzed that users post on Specific Topic.
  • Using Tweet API we will be Searching on Particular topic which is tweeted by each and every individual.
  • Run a search for users similar to find people button on Twitter.com, the same results return by people search on twitter.com by using API.
  • Tweets will be verified word to word so that it will give result about the tweet whether it is positive or negative or neutral.
  • Polarity will be looking after this and if there increases more than 1.0 then it will be treated as positive else negative.
  • Next is finding the emotions

. word list is to be collected which consists of emotions like happy ,sad, angry, surprise ,love.

. Data cleaning is to be done for the words which we got from tweets. Eg: I am tired.

After cleaning ( tired ) only that word is considered.

. under which emotion does it comes will be verified and gives the output which kind of emotion it is.

Clone this wiki locally