COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 1

Comparing Prices of S&P 500 Consumer Discretionary Stocks with Twitter Sentiment

Advanced Scientific Research Paper

Submitted to the Center for Advanced Studies, Wheeler High School

by

Desai, Sarang

The Center for Advanced Studies

Wheeler High School Marietta, GA
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 2

Abstract

This study focuses on Twitter sentiment, or the attitudes and opinions expressed on Twitter by

users, and the stock price of five companies: Nike, Amazon, Disney, Home Depot, and Comcast.

The study seeks to answer the overarching question: “how is the value of consumer discretionary

companies in the S&P 500 impacted by the sentiment of tweets?” This study uses the

quantitative scientific method. The independent variable is the change the positive sentiment

ratio of tweets for a company, and the dependent variable is the change in the closing stock price

of a company. The researcher collected tweets and stock prices using the programming language

R for 25 days. Next, the researcher conducted a linear regression test to determine if a linear

relationship exists between the two variables; however, due to high p-values of 0.3287, 0.9625,

0.7301, 0.5952, and 0.2803 respectively for the companies listed above, the researcher

determined that a statistically significant relationship does not exist between the variables.

Several limitations of the study, such as external variables or differing market sentiment, may

have influenced the interpretation of the results. Professionals in the fields of business and

economics can consider the results of this study, and future researchers in this field can consider

prolonging the data collection period and considering other social media platforms to capture

more market sentiment.

Key Words: Sentiment Analysis, Consumer Discretionary, Twitter, S&P 500, Web Scraping
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 3

Table of Contents

Chapter 1: Introduction……………………………………………………………………………7

Statement of the Problem………………………………………………………………….7

Purpose of the Study………………………………………………………………………7
Research Questions………………………………………………………………………..8
Hypothesis Statement……………………………………………………………………...8
Significance of the Study………………………………………………………………….8
Definition of Key Terms…………………………………………………………………..9
Summary…………………………………………………………………………………10

Chapter 2: Literature Review…………………………………………………………………….11

Measuring Twitter Sentiment Using Programming……………………………………...11

Choosing Between R and Python for Sentiment Measurement………………………….14
Summary…………………………………………………………………………………16

Chapter 3: Research Method……………………………………………………………………..17

Research Methods and Design(s)………………………………....……………………...17

Population………………………………………………………………………………..19
Sample……………………………………………………………………………………19
Materials/Instruments……………………………………………………………………20
Data Collection, Processing, and Analysis………………………………………………21
Assumptions……………………………………………………………………………...22
Limitations……………………………………………………………………………….23
Delimitations……………………………………………………………………………..23
Ethical Assurances……………………………………………………………………….24
Summary…………………………………………………………………………………24

Chapter 4: Findings………………………………………………………………………………26

Results……………………………………………………………………………………26
Evaluation of Findings…………………………………………………………………...27
Summary…………………………………………………………………………………28

Chapter 5: Implications, Real World Connections, Recommendations, and Conclusions………29

Implications………………………………………………………………………………29
Real World Connections…………………………………………………………………31
Recommendations………………………………………………………………………..31
Conclusions………………………………………………………………………………32

References………………………………………………………………………………………..33
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 4

Appendix A: R Code……………………………………………………………………………..36

Appendix B: Raw Data…………………………………………………………………………..39

Appendix C: Percent Change Data………………………………………………………………40

Appendix D: Graphs of Collected Data………………………………………………………….41

COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 5

List of Tables

Table 1: Testing Keywords in the Search Algorithm……………………………………………26

Table 2: Results of Linear Regression…………………………………………………………...27

COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 6

List of Figures

Figure 1: Raw Data………………………………………………………………………………39

Figure 2: Percent Change Data…………………………………………………………………..40

Figure 3: Nike Graph…………………………………………………………………………….41

Figure 4: Amazon Graph………………………………………………………………………...42

Figure 5: Home Depot Graph……………………………………………………………………43

Figure 6: Comcast Graph………………………………………………………………………...44

Figure 7: Disney Graph…………………………………………………………………………..45

COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 7

Chapter 1: Introduction

In today’s world, financial experts and shareholders are seeking new and innovative ways

to predict the stock market. The stock market is a dynamic entity, and according to Hayford and

Malliaris (2004), it could be influenced by several factors such as government regulations,

economic strength, and inflation (p. 400). As society evolves, more factors will have their own

impacts on the market. One incredible invention to come out of this societal evolution is social

media, and researchers have sought to find a connection between the stock market and social

media, particularly using one platform: Twitter. Twitter is a significant part of today’s world.

According to Zhang (2013), Twitter’s active users in the United States generate upwards of 400

million tweets per day (p. 2), showing how pertinent Twitter is to the modern world. To predict

the stock market, Zhang focused on Twitter sentiment, or the attitude of different Twitter

messages, or tweets. Researchers hypothesized that the sentiment of tweets could forecast the

stock market, and this study will focus on Twitter sentiment and the consumer discretionary

sector of the S&P 500.

Statement of the Problem

The objective of this study is to determine if the ratio of positive tweets to total tweets

will impact the share prices of S&P 500 consumer discretionary companies.

Purpose of the Study

The purpose of conducting this study is to determine if a possible relationship exists

between the change in sentiment, or attitude, of Twitter messages discussing certain consumer

discretionary companies and the change in stock prices of those companies. This study will

track the stocks of Walt Disney, Amazon, Home Depot, Nike, and Comcast, as previous studies

have used these stocks to conduct similar research (Puri, 2017, p. 1; Ranco, Aleksovski,

Caldarelli, Grcar, & Mozetic, 2015, p. 4).

COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 8

Research Questions

The two research questions for this study are the following

1. How will the correct keywords be selected to gather the most relevant tweets related

to consumer discretionary companies?

2. Using R, how does the sentiment of tweets relate to the value of consumer

discretionary stocks?

Hypothesis Statement

In this study, the independent variable is the positive sentiment ratio of tweets. This

variable is obtained by dividing the total number of positive tweets collected by the total number

of tweets. The dependent variable is the closing price of the stock for each company. The null

hypothesis for this study is that no linear relationship exists between the change in positive

sentiment ratio of tweets and the change in closing stock price of the company. The alternative

hypothesis for this study is that a linear relationship exists between the change in positive

sentiment ratio and the change in closing price of the stock.

Significance of the Study

This study is important because it discusses how social media, a device that the general

public uses casually, can affect something as large at the stock market. The results of this study

can help consumer discretionary companies in potentially predicting how their respective

company’s share value will change based on the attitude of the shareholders of that company.

This can ultimately change decision making outcomes at the higher level to assure executives

that their company will be ready for a drop or rise in stock valuation. Without having an ability

to estimate future outcomes, corporations can suffer greatly due to their lack of preparedness
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 9

from a significant drop or rise in sentiment and share value, which can harm the strength of the

company all the way to bankruptcy.

Definitions of Key Terms

Consumer Discretionary. This term refers to one of the ten sectors of the S&P 500,

describing non-essential goods (Baker & Dumont, 2014, p. 110).

Exchange Traded Fund. This term refers to the securities that consumers trade on the major

stock exchanges. This instrument aims to replicate the performance of an index (Ben-David,

2016, p.1).

Machine Learning. This term refers to a method of data analysis that automates analytical

model building (SAS, n.d.)

Market Sentiment. This term refers to the emotional factor of market participants as

expressed through blogs, news, and social network services (Pyo, Lee, Cha, & Jang, 2017, p. 1)

Python. This term refers to a programming language that uses a hierarchical system of

classification which determines whether the text is neutral first and then the level of its polarity

only in the case of a non-neutral text (Maldonado & Sandhu, 2015, p. 290)

R. This term refers to a popular open-source statistical programming software package (Hill

& Scott, 2017, p. 1).

R Packages. This term refers to collections of functions, data, and compiled code in a well-

defined format (Rice & Thorton, 2013, p. 3).

Sentiment Analysis. This term refers to the computational study of opinions, sentiments and

emotions expressed in text (Liu, 2009, p. 3).

Support Vector Machine. This term refers to a well-known sentiment classifier used in

classifying sentiment in literature (Liu, Luo, Liu, Zhong, Wei, & Sun, 2015, p. 6).
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 10

Syntax. This term refers to a mental concept that binds together a finite number of words

into well-formed sentences that convey meaning (Lieberman, 2001, p. 36).

Web scraping. This term refers to a method of collecting data from websites (Krotov &

Tennyson, 2018, p.170).

Summary

The purpose of this study is to determine if the sentiment of messages shared on Twitter,

a globally popular social networking platform, will have an impact on the share price of S&P

500 consumer discretionary companies. Conducting the study will require determining accurate

keywords to use in gathering tweets through programming and determining the strength and

significance of the relationship. The null hypothesis is that no linear relationship exists, and the

alternative relationship is that there is a statistically significant linear relationship between the

two variables. This study is significant because it could have an impact on the business world by

preparing executives to make changes in their company to cope with changing market sentiment.
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 11

Chapter 2: Literature Review

To find out how to properly answer the overarching question, the researcher answered

two important questions to understand the logistics of the research. First, the researcher reviewed

several past studies to understand how different programs measure the sentiment of tweets. This

is important for the researcher to understand because of the significant role sentiment analysis

plays in the research. Next, the researcher sought to find out which programming language to use

to use to scrape Twitter and perform sentiment analysis on the gathered tweets. The researcher

reviewed previous literature to choose between R and Python for research. The researcher

decided between R and Python because the time needed to learn the languages for the research

study was minimal. Finally, the researcher analyzed the literature findings and related the

findings to the research questions. This helped the researcher in properly conducting the

necessary steps to answer the research questions.

Measuring Twitter Sentiment Using Programming

According to Zhang (2013), the volume of literature addressing social media sentiment

analysis continues to grow, and researchers continue to use sentiment analysis to discover public

mood and create relationships between certain trends, such as trends in the stock market (p. 2).

However, the collection and sorting of tweets into positive, neutral, and negative categories

requires certain permissions from Twitter and machine learning techniques. To begin, Kabir,

Karim, Newaz, & Hossain (2018) summarized that pulling in and sorting tweets according to

their sentiment requires programming languages that have the ability to do such an action (p. 30).

According to Yang, Mo, and Liu (2015), “sentiment analysis is applied to determine the

sentiment level of tweet messages. A major challenge is to find the optimal algorithm that

measures sentiment with highest accuracy” (p. 1644). Additionally, according to Giachanou and
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 12

Crestani (2016), a major challenge with performing sentiment analysis on tweets is removing

“stop words” (words such as “the”, “is”, or “a” that do not contribute to the meaning or

sentiment of the tweet), determining topic relevance, and accounting for incorrect English (p. 6-

7), because tweets must be edited by programs to retain only significant words or phrases for

further analysis. If stops words are not removed from the tweets, the program will interpret the

tweets incorrectly, resulting in an incorrect analysis. Furthermore, the irrelevant tweets will

return incorrect results for the data. For example, if the web scraping program gathers negative

tweets about Nike while scraping for Tweets about Amazon, the Amazon positive sentiment

tweet ratio will decrease, even though the tweets are irrelevant. For this reason, to gather the

correct tweets, Puri (2017) stressed that the keywords logged into the search algorithm must be

relevant to the company; otherwise, the function will not return tweets relevant to the company,

and this could result in the use of incorrect tweets for analysis, primarily when used for

applications in finance or marketing (p. 4).

Finally, Reed (2016) supported the argument that “sentiment can be measured via

analysis of social networks” and even goes as far as to say “such sentiment significantly affects

stock prices” (p. 349), signifying that the attitude of tweets can be measured by analyzing certain

keywords in the tweet. The analysis of the keywords in these tweets will provide the researcher

with the most accurate sentiment of the tweets, because keywords that carry high levels of

emotion are relevant to the overall emotion of the tweet. This sentiment of the tweets, which

Reed says can measure the sentiment of the social networks, can be used to express the market

sentiment of a certain company.

These sources pose the question of how the development of software, especially through

machine learning, can successfully and accurately sort tweets into positive, neutral, and negative
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 13

classifications. One such machine learning technique is specified by Zhang (2013) as “very

effective for text categorization”: The Support Vector Machine (p. 14). Additionally, Zhang

(2013) explained that Support Vector Machines work on optimizing classifications which, in

turn, makes the classifications more accurate with each analysis (p. 14-15), which is a prime

example of machine learning. A programmer will administer an initial test to a machine to allow

the machine to begin learning the sentiment of words, and repeated analysis will allow the

machine to learn which words are positive, negative, or neutral. Yang et al. (2015) also made

use of Support Vector Machines to determine the sentiment of a tweet in their study. However,

sentiment analysis does not come without problems. According to Kabir et al. (2018), the

perception of the tweet can change the actual sentiment of the tweet, as a positive word might be

used in a negative context (p. 36). In this case, the Support Vector Machine could read the tweet

as positive instead of negative. Giachanou and Crestani (2016) bring up a similar point, noting

that “data sparsity” occurs due to informal text use on Twitter (p. 34) and that this may result in

incorrect sentiment analysis as well. Antonakaki, Spiliotopoulos, Samaras, Pratikakis, Ioannidis,

and Fragopoulou (2017) noted that researches can improve the tools used for sentiment analysis

by using sarcasm detection methods to provide a more accurate analysis (p. 7). Without a method

to detect sarcasm, the analysis for some tweets could be incorrect. Regardless, as machine

learning and sentiment analysis techniques develop, researchers will solve these problems to

provide more accurate tools.

Overall, the literature search for this question provided several answers to how sentiment

is measured by programming. In general, programs measure sentiment by classifying tweets into

positive, negative, and neutral categories through machine learning. It is important for the

program to remove stop words to provide a more accurate sentiment analysis. To do this, the
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 14

researcher will include a section in the code during sentiment analysis that removes stop words

or find a certain sentiment analysis package that takes stop words into account. It is also

important for the researcher to use the correct keywords in the search function while gathering

tweets to assure that tweets being gathered are relevant to the company to capture the most

accurate market sentiment. Next, several sources explained how Support Vector Machines are

the best choice for sentiment analysis due to their high accuracy of sentiment detection and

classification; however, the person who tweets a message may have a different interpretation

from what the program concludes due to sarcasm or choice of vocabulary. These sources help set

the overall basis for the research by allowing the researcher to understand how the code will run

and analyze tweets and how to optimize the code to focus on the correct aspects of the tweets.

Choosing Between R and Python for Sentiment Measurement

Support Vector Machines are available for use in different programming languages. As

mentioned by Kabir et al. (2018), gathering and analyzing the sentiment of tweets requires

languages that have the ability to perform such an action, whether by using different packages or

in-built functions in the program itself (p. 4). Two programming languages that the research

chose to focus on due to their ease of use and time required to reach proficiency in sentiment

analysis with are R and Python. Past researchers used Support Vector Machines in the R

programming language as well as in Python to perform sentiment analysis. First, Ranco et al.

(2015) utilized Support Vector Machines to sort collected tweets into positive, neutral, and

negative sections (p. 5). According to Meyer (2018), a researcher can utilize Support Vector

Machines in R with great accuracy (p. 4). Additionally, Hill and Scott (2017) state that the

versatility of the R programming language allows it to pull previously archived tweets as well as

the most recent tweets (p. 44). These sources document that the researcher can use R to perform
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 15

several unique tasks. Then, according to Hill and Scott (2017), certain functions and R packages

remove stop words and other unnecessary words from tweets, and the packages compare the

remaining words to a database of positive, neutral, and negative words along with the syntax of

tweets to determine the sentiment of the tweet with good accuracy (p. 45). Next, both Asghar,

Khan, Ahmad, Qasim, and Khan (2017) and Antonakaki et al. (2017) used Support Vector

Machines in their studies, but, unlike the previously mentioned studies, used Python. Both

studies received good results, but Asghar et al. (2017) noted that a major limitation in Support

Vector Machines in Python was the needed improvement of domain specific vocabulary that

helps determine the sentiment of a tweet (p. 4). Antonakaki et al. (2017) noted that discrepancies

occurred with using Python because of the unusual use of multilingualism in the tweets his study

collected (p. 3). Finally, Maldonado and Sandhu (2015) noted the accuracy of Support Vector

Machines in Python when compared to analyses carried out by UMIGON, a less common

sentiment analysis software (p. 299). When Maldonado and Sandhu’s study used the two

techniques to categorize the same tweets, they discovered that the two techniques yielded

different results (p. 313). The researchers determined that UMIGON is more accurate concluded

that the Python Support Vector Machine is not very accurate because of the general assumption

that UMIGON is accurate (p. 313). Although Support Vector Machines are the most accurate

analyzer of Twitter sentiment and the easiest of the more complex sentiment classifiers to

implement, time constraints on the study will not allow for the use of this machine in the

sentiment analysis of tweets.

Overall, these studies showed that due to the extensive sentiment analysis packages and

level of accuracy, R is the better programming language to use for sentiment analysis. The

language is flexible due to the different analysis packages and the ability of the program to pull
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 16

recent and archived tweets. The program also showed great success with Support Vector

Machines, especially when classifying tweets into categories, detailing that R is accurate with

sentiment analysis and classification; however, due to the complexity of Support Vector

Machines and the time constraints on the study, the researcher will not use Support Vector

Machines in R and focus on using R for sentiment classification. Understanding that R is better

for sentiment analysis will allow the researcher to use it for data collection and analysis for the

research questions.

Summary

There were several important points discussed in this chapter. First, the researcher figured

out how programs perform sentiment analysis. According to Zhang (2013), programs called

Support Vector Machines provide the most effective sentiment analysis (p. 14). The machine

learning algorithm used with Support Vector Machines allow the machine to distinguish key

words in tweets and assign them to positive, negative, or neutral categories to give the tweet an

overall sentiment classification. Even though Support Vector Machines are accurate sentiment

analyzers, Giachanou and Crestani (2016) explain that informal use of language can give an

incorrect analysis of the tweet (p. 34), but such instances are due to the limitations of the

machine. Next, the researcher determined that R is the best programming language to use for

sentiment analysis. A key reason for is this is that according to Meyer (2018), Support Vector

Machines in R are very accurate (p. 4), and according to Ranco et al. (2015), researchers can

utilize them to categorize tweets according to their sentiment (p. 5). These sources indicate that

the researcher can ensure that tweets analyzed with R will be accurate.
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 17

Chapter 3: Research Method

The purpose of conducting this study is to determine if a possible relationship exists between

the sentiment, or attitude, of Twitter messages discussing certain consumer discretionary

companies and the stock prices of those companies. This study will track the stocks of Walt

Disney, Amazon, Home Depot, Nike, and Comcast, as previous studies have used these stocks

to conduct similar research (Puri, 2017, p. 1; Ranco et al., 2015, p. 4).

The two research questions for this study are the following

1. How will the correct keywords be selected to gather the most relevant tweets related

to consumer discretionary companies?

2. Using R, how does the sentiment of tweets relate to the value of consumer

discretionary stocks?

In this study, the independent variable is the positive sentiment ratio of tweets. This

variable is obtained by dividing the total number of positive tweets collected by the total number

of tweets. The dependent variable is the closing price of the stock for each company. The null

hypothesis for this study is that no linear relationship exists between the change in positive

sentiment ratio of tweets and the change in closing stock price of the company. The alternative

hypothesis for this study is that a linear relationship exists between the change in positive

sentiment ratio and the change in closing price of the stock.

Research Methodology and Design(s)

The research method used for this study is the quantitative scientific method. The

variables used for the research (the ratio of positive tweets and the closing stock price) are

ratio data, because they are continuous numerical data that can have a value of zero.

Answering the overarching question for this study begins by formulating a hypothesis. In
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 18

this study, the null hypothesis is that there is no linear relationship between the change in

positive sentiment ratio of tweets and the change in closing stock price of the company,

and the alternative hypothesis is that a linear relationship exists between the change in

positive sentiment ratio of tweets and the change in closing stock price of the company.

Then, the researcher collects the data (tweets sentiment ratios and closing stock prices),

and after running an inferential statistics test, the researcher analyzes the findings to

determine if the results agree with the hypothesis or reject the hypothesis. These steps are

characteristic with the scientific method.

To begin the research study, the researcher and the mentor brainstormed keywords

to place into the search algorithm into R for each company. After selecting certain

brainstormed keywords, the researcher placed them into the algorithm and ran the program

to collect tweets. The researcher and mentor reviewed the tweets for face validity to

determine whether the tweets were relevant to the company. If the keywords are not

relevant, the researcher brainstormed more words to place into the algorithm and repeated

the process until the researcher brainstormed relevant keywords for each company. After

selecting the most relevant keywords, the researcher continued gathering tweets and

checking them for relevancy for five days to ensure the reliability of the keywords.

After choosing the keywords, the researcher used R to analyze the tweets. The researcher

collected tweets for six weeks on the days that the stock market was open. The researcher

uploaded the tweet data into R, and after stripping and cleaning irrelevant words in the tweets,

the researcher used the R package “syuzhet” to analyze the sentiment of tweets. The analysis

returned numerical values to the researcher, and the researcher grouped the numerical values

based on whether the number is positive, neutral, or negative. The researcher used R to create a
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 19

table of the number of positive, neutral, and negative tweets and exported this table to an Excel

file on the researcher’s personal computer for storage. In this Excel file, the researcher calculated

the ratio of positive to total tweets for each company and document it in the Excel file. The

researcher calculated the percent change of the ratio for each collection day.

Next, the researcher collected the closing value of the stock of the five selected

companies (Nike, Amazon, Disney, Home Depot, and Comcast) using Yahoo Finance. The

researcher stored the closing value of the companies in an Excel file on the researcher’s personal

computer and calculated the percent change of the price for each collection day.

The researcher accomplished the study goals by collecting the stock price and tweet ratio

every business day. The scientific method is the optimum choice for the research because of the

hypotheses involved in determining a relationship between the variables.

Population

The population of this study is the tweets that mention each of the five companies

(Nike, Amazon, Disney, Home Depot, and Comcast). According to Zhang (2013), upwards

of 400 million tweets are generated daily by users in the United States (p. 2). This population

is appropriate because the study focuses on the five previously mentioned companies, and the

researcher draws the tweets for the analysis from the pool of tweets about a specific

company.

Sample

The sampling method for this study is convenience sampling, because the selected

tweets are the most recently archived tweets from a certain day. The researcher chose this

sampling method because the R package “twitteR” allows users to collect previously

archived tweets or the most recently tweeted tweets, and recent tweets are simpler to collect
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 20

using the R package. For each company, the researcher analyzed 10,000 tweets in total,

because the primary source used for this study, Zhang (2013) used 10,000 tweets in analysis

(p. 11). Considering the number of collection days, the researcher collected 417 tweets for

each company every data collection day.

Materials/Instruments

The source of the data was Twitter, an online social networking platform where users

can share messages with other users using messages called tweets. Twitter archives every tweet

into their database for developers and researchers to use for personal or professional research,

and using the proper permissions, developers and researchers can access the archives. The

researcher collected tweets using previously written code. The researcher collected tweets for

sentiment analysis, which the researcher would use to find the sentiment ratios. First, the

researcher used R, specifically the “twitteR” package, to test different keywords and determine

which one(s) to use in the data collection process. To ensure interrater reliability and face

validity, the researcher and mentor reviewed the tweets to assure that the tweets were relevant

to the company, and the researcher ensured construct validity by reviewing the content of the

tweets. To ensure test-retest reliability, the researcher repeated the process for five days with

the selected keyword to ensure that the relevancy and content of the tweets were specific to the

company and to ensure that the previously written code worked appropriately. Once the

researcher selected the keywords, the researcher used the R package “syuzhet” to perform

sentiment analysis. This package ensures content validity, as it accounts for stop words,

assuring that the program accurately measured the sentiment of the tweet by taking all words

into account.
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 21

Next, the researcher used Yahoo Finance to collect the closing price of the stock for the

five companies. The researcher ensured test-retest reliability because researchers in previous

studies used this tool to gather stock prices (Puri, 2017, p. 9; Reed, 2016, p. 344).

Data Collection, Processing, and Analysis

First, the researcher used R for scraping Twitter. The researcher gathered the appropriate

permissions from Twitter and used the “searchTwitter” function in the R package “twitteR” to

gather 417 tweets for each company every day. The researcher stored the tweets as a comma-

separated values file (.csv) to the researcher’s local computer, and then, the researcher pulled the

tweets back into R. Then, the researcher converted the tweets to the appropriate format for

sentiment analysis using the R package “syuzhet”. The “syuzhet” package determines the

sentiment of a tweets by isolating keywords and determining their sentiment based on a database

of positive, negative, and neutral words that is automatically updated through machine learning.

The researcher used the “get_sentiment” function in the R package “syuzhet” to determine the

sentiment of each tweet, and the program returned the number of positive, negative, and neutral

tweets. The researcher recorded these values in an Excel spreadsheet and calculated the positive

sentiment ratio of the tweets by dividing the number of positive tweets by the total number of

tweets collected that day. Next, the researcher calculated the percent change of the ratio,

expressed as a decimal, every day.

Then, the researcher uploaded the closing price of the stock for each company into the

Excel spreadsheet and calculated the percent change of the prices. Finally, the researcher used R

to perform a linear regression test on the percent change in sentiment ratio and the percent

change of share price. The researcher used a linear regression test to test the hypothesis of a
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 22

possible linear relationship between the change in positive sentiment ratio and the change in

stock price.

Assumptions

The sample tweets collected daily are representative of the general public’s market

sentiment. It is reasonable to make this assumption because Twitter is a common channel used

by the general public to express their opinions, and these opinions make up the market sentiment

about a company.

The algorithms used to analyze the sentiment of tweets correctly sort tweets into the

proper categories. The researcher used an R package created for sentiment analysis, and it is safe

to assume that the creator of the package has the correct qualifications and knowledge to create

functions that accurately sort the tweets by sentiment.

The time that the tweets are collected are during a time that Twitter traffic is high.

Previously mentioned by Zhang (2013), Twitter users generate upwards of 400 million tweets

per day in the United States (p. 2) or more than 16 million tweets per hour. Based on this

number, it is safe to assume that Twitter traffic is high during the collection process.

The keywords used to compile tweets are representative of the companies analyzed. The

student and the mentor confirmed that the keywords gather relevant tweets in the search

function, so it is safe to assume that the keywords appropriately represent the company and pull

in the correct tweets.

The “syuzhet” package available within the R programming language is the best choice

to analyze tweets. The “syuzhet” package is one of the several sentiment analysis packages

available in R, and due to its ease of use and method of analysis, it is the best sentiment analysis

software choice for the researcher.

COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 23

Limitations

Other variables may influence stock price along with market sentiment; however, this

study does not consider those variables. The study does not consider other variables due to the

focus of the study on market sentiment. This may threaten internal consistency reliability, as

influences not related to market sentiment may significantly impact the stock price of a

company. This may harm the interpretation of the final results, as the stock price will be

impacted by influences other than market sentiment.

This study only takes the sentiment of Twitter messages into account to determine the

market sentiment. Other platforms, such as Facebook or personal blogging websites, might have

an impact on the market sentiment, but this study only focuses on Twitter. This could threaten

construct validity because Twitter may not demonstrate the actual market sentiment at the time

of data collection; however, the researcher addressed this issue by gathering an appropriate

number of tweets for analysis to capture the market sentiment through Twitter.

Delimitations

The study only considers tweets in English for analysis. The study only considers English

tweets to make the sentiment analysis easier for the machine and to decrease any errors in the

machine due to the use of words in another language. Analyzing tweets in a different language

could threaten internal validity, as the tweet may not be appropriate for analysis for a certain

company, the program could provide an incorrect sentiment analysis on the tweet.

The study only considers the closing price of the stock. The closing price of the stock is

the easiest to record, for due to time constrictions in the study, the researcher did not gain the

necessary skills required to take the daily fluctuation of the market into account.
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 24

This study only applies to companies in the consumer discretionary sector of the stock

market. The scope of this study only applies to the consumer discretionary sector of the S&P

500, so the researcher only gathered tweets on companies in the consumer discretionary sector of

the S&P 500.

Ethical Assurances

This study does not involve human data. Additionally, once a Twitter user tweets a

message, as per Twitter policy, the tweets are publicly available for research purposes and

app development with the proper permissions from Twitter. The researcher has the proper

permissions to use tweets for this study.

Summary

This study used the quantitative scientific method to test the alternative hypothesis, which

states that there is a statistically significant linear relationship between the change in the positive

sentiment ratio (the independent variable) and the change in the stock price of a company (the

dependent variable). The null hypothesis states that there is no relationship between the two

variables. After determining the hypotheses, the researcher began the study. First, the researcher

and mentor brainstormed keywords to place into the search algorithm. After determining the

keywords, the researcher began scraping Twitter using the R package “twitteR”. The researcher

gathered 10,000 tweets for each company, because the primary source, Zhang (2013), used

10,000 tweets for analysis (p. 11). The researcher performed sentiment analysis using the R

package “syuzhet” and calculated and stored the positive sentiment ratio on an Excel

spreadsheet. The researcher calculated the percent change of the positive sentiment ratio. Then,

the researcher used Yahoo Finance, a tool used in previous studies (Puri, 2017, p. 9; Reed, 2016,

p. 344), to gather the closing stock price of each company. The researcher stored the price on an
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 25

Excel spreadsheet and calculated the percent change in the stock price. Then, the researcher

performed a linear regression test to test the hypothesis.

COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 26

Chapter 4: Findings

The purpose of this study is to determine if a linear relationship exists between the

change in the positive sentiment ratio of tweets about a company and the change in stock

price of that company. The null hypothesis for this study is that a linear relationship does

not exist between variables, and the alternative hypothesis is that a linear relationship

exists. This chapter will detail the results of the study and the evaluation of the results.

Results

The following tables detail the results of the study.

Table 1

Testing Keywords in the Search Algorithm

Company Keywords Pass or Fail

Nike “shoes”, “clothes” Fail
Nike “Nike” Pass – used for search
Amazon “shipping”, “drones” Fail
Amazon “Amazon” Pass – used for search
Disney “Mickey”, “Disneyworld” Fail
Disney “Disney” Pass – used for search
Home Depot “Building”, “construction” Fail
Home Depot “Home Depot” Pass – used for search
Comcast “Xfinity”, “TV” “cable” Fail
Comcast “Comcast” Pass – used for search

Table 1 shows the keywords tested by the researcher and the mentor. The researcher and

mentor brainstormed several keywords, and the final results showed that the best keywords to

place into the search algorithm to gather relevant tweets are the names of the companies

themselves. The researcher used the name of the companies in the search algorithms to pull in

the tweets used for analysis.

COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 27

Table 2

Results of Linear Regression

Residual Degrees
Intercept Slope Std. T R- P-
Company Standard F-Statistic of
Coefficient Coefficient Error Value Squared value
Error Freedom
Nike 0.0043 -1.5095 1.5127 -0.9980 0.1189 0.0415 0.9958 23 0.3287

Amazon 0.0016 -0.0178 0.3747 -0.0470 0.06864 9.8e-5 0.0023 23 0.9625

Disney 0.0016 -0.4814 1.3783 -0.3490 0.1088 0.0053 0.1220 23 0.7301

Home 0.013 0.7794 1.4462 0.5390 0.1144 0.0125 0.2903 23 0.5952

Depot
Comcast 0.018 -2.1033 1.9023 -1.1060 0.1929 0.0505 1.2220 23 0.2803

Note: Decimals rounded to the ten-thousandths place for clarity.

Table 2 documents the results of the linear regression test. The assumptions associated

with linear regression are that the relationship between the variables is linear and the variables

follow a normal distribution. The standard error is high related to the coefficients, indicating that

there is much error in the model. The t-values are relatively close to zero, indicating that the

relationship is not statistically significant. The low r-squared values note that the regression does

not explain the variance in percent change in the stock price, indicating that the model is not a

good fit for the data. The values of the residual standard error for the companies notes that the

quality of the regression is low. The f-statistics are relatively close to 1, indicating that there is a

lower chance of a relationship between the percent change in sentiment and the percent change

in the stock price. The degrees of freedom are the number of observations (25) minus the number

of variables (2). Finally, the results for this study are not statistically significant at a confidence

level of 95% because all the p-values are above 0.05.

Evaluation of Findings

The results of this study show that the results are not statistically significant. The high p-

values, irregular residual standard errors, low f-statistics, and low r-squared value show that the
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 28

results are not significant and that there is much room for error. These conclude that the

researcher must fail to reject the null hypothesis and state that there is no statistically significant

relationship between sentiment and stock price. The researcher did not expect these results, as

previous studies concluded a statistically significant relationship between sentiment and stock

price (Zhang, 2013, p. 1; Ranco et al., 2015, p. 18). External factors other than market sentiment,

such as the overall strength of the economy or changing government regulations, that influence

the stock market may explain the discrepancy between the results.

The researcher can interpret these results by stating that market sentiment does not have a

significant impact on the stock market. Corporate leadership can use these findings in the context

of public relations, especially on social media, to focus on what their company shares on social

media. The statistically insignificant findings are revelatory, as previous researchers determined

that there is a mild relationship between the two variables.

Summary

The results of this study indicate that there is no statistically significant relationship

between the change in sentiment and the change in stock price. This is due to the high p-values

associated with the linear regression test. The researcher did not expect these results, as previous

studies indicated statistically significant relationships between the two variables (Zhang, 2013, p.

1; Ranco et al., 2015, p. 18), even though these relationships were mild.
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 29

Chapter 5: Implications, Real World Connections, Recommendations, and Conclusions

The objective of this study is to determine if the ratio of positive tweets to total tweets

will impact the share prices of S&P 500 consumer discretionary companies. The purpose of this

study is find if a statistically significant relationship exists between the change in the positive

sentiment ratio and the change in the stock price of a company. First, the researcher and the

mentor brainstormed several keywords for each company to place into the search algorithm for

gathering tweets. After selecting the keywords for the search algorithms, the researcher

gathered tweets, analyzed the sentiment of the tweets, and performed a linear regression test to

determine the significance and possible relationship of two variables. The results indicated

insignificant results due to the high p-values associated with each company.

One of the limitations of this study is that other variables, such as government regulations

and may impact the fluctuation of the stock market; however, this study does not consider those

variables due to the scope and the focus of the study. Next, the study does not consider social

media messages shared on any platform other than Twitter. Additionally, this study only

considers tweets in English, the closing price of the stock, and only consider consumer

discretionary companies due to the scope of the study.

This chapter will discuss the implications of the research, real world applications of the

research, and recommendations for future research.

Implications

The researcher can draw pertinent implications from the second research question. The

second research question is: how does the sentiment of tweets relate to the value of consumer

discretionary stocks? According to the results depicted in Table 2, the p-values for each company

are above 0.05, signifying that the results of this research question are not statistically
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 30

significant. Therefore, the researcher must fail to reject the null hypothesis, which states that

there is no relationship between sentiment and stock price, and reject the alternative hypothesis,

which states that there is a relationship. The researcher must fail to reject the null because the

researcher cannot draw conclusions from the study due to the p-values indicating statistical

insignificance.

The first limitation, stating that factors other than market sentiment impact stock price,

may have impacted the results of the study. Factors such as the strength of the overall economy,

international trade regulations, tariffs, and inflation could have impacted the closing stock price

of the companies. This could overpower any influence that market potentially had, resulting in

high p-values and high residual errors and impacting the relationship of the two variables. This

would result in a poor interpretation, as the two variables would show little or no relationship.

The next limitation, discussing that other social media platforms may play a role in the

market sentiment, may have affected the interpretation of the results. Other places on the internet

where consumers can voice their sentiment, such as Facebook or personal blogging sites, may

play a stronger role in determining the market sentiment of consumer for a certain company. If

the sentiment on these websites is a more accurate indicator of market sentiment than Twitter

and different from sentiment expressed on Twitter, then the results of the study can be

misinterpreted, causing an interpretation of no relationship and resulting in high p-values such as

those demonstrated in Table 2.

The results in this study were not similar to the results of previous studies. The study of

Zhang (2013) indicated a statistically significant relationship between the Twitter sentiment of

the technology sector of the stock market and the price of that sector (p. 1). His study showed a

mild but significant relationship. The study of Ranco et al. (2015) also indicated a statistically
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 31

significant relationship. This study did not receive those results, partly due to the limitations

discussed above.

The results of this study contribute to the field of study by adding to the literature

discussing variables that may influence the stock market. The results of this study shed doubt

over market sentiment having a significant influence on the stock market.

Real World Connections

Researchers and professionals cannot safely make conclusions due to the high p-values

associated with the study. Professionals in the field of economics can consider the results of this

study when making executive level decisions based on the fluctuation of the share price of their

companies. Due to the statistically insignificant results indicated by this study and the limitations

brought on to the study due to time constraints, future researchers can consider the limitations

and modify the methods of the study to produce more accurate, relevant, and significant results.

Recommendations

For future studies, future researchers can take several recommendations into account.

First, future researchers can conduct the study for a company’s entire fiscal year. This will help

eliminate the impact of fluctuating supply and demand based on the time of year for a company.

Next, to capture a larger share of market sentiment, future researchers can consider several

platforms, such as Twitter, Facebook, Instagram, and other opinion sharing platforms. This will

allow the researcher to measure a larger portion of the consumer base and capture more market

sentiment, which could result in statistically significant results.

COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 32

Conclusions

The results of this study can be applied to the fields of economic and business, but due to

the high p-values associated with the results, professionals and researchers cannot safely draw

conclusions from the results. The limitations on this study impacted the interpretation of the

results, especially the limitation stating that there are other factors besides from market sentiment

that impact the fluctuation of stocks. These other variables may impact the stock market, causing

a misinterpretation of the results gathered from measuring market sentiment. For future research,

researchers could consider extending the period of study and considering several different social

media platforms.
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 33

References

Antonakaki, D., Spiliotopoulos, D., V. Samaras, C., Pratikakis, P., Ioannidis, S., & Fragopoulou,

P. (2017). Social media analysis during political turbulence. PLoS ONE, 12(10), 1-23.

http://dx.doi.org/10.1371/journal.pone.0186836

Asghar, M. Z., Khan, A., Ahmad, S., Qasim, M., & Khan, I. A. (2017). Lexicon-enhanced

sentiment analysis framework using rule-based classification scheme. PLoS ONE, 12(2),

1-22. http://dx.doi.org/10.1371/journal.pone.0171649

Baker, W., & Dumont, G. (2014). Equity analyst recommendations: a case for affirmative

disclosure? Journal of Consumer Affairs, 48(1), 96-123. Retrieved from the Student

Resources in Context database.

Ben-David, I. (2016, November 23). Exchange traded funds (ETFs). Retrieved from

https://corpgov.law.harvard.edu/2016/11/23/exchange-traded-funds-etfs/

Giachanou, A., & Crestani, F. (2016). Like it or not: A survey of Twitter sentiment analysis

methods. ACM Computing Surveys, 49(2), 1-41. https://doi.org/10.1145/2938640

Hayford, M. D., & Malliaris, A. G. (2004). Monetary policy and the U.S. stock market.

Economic Inquiry, 42(3), 387-401. Retrieved from the Gale Biography in Context

database.

Hill, S., & Scott, R. (2017). Developing an approach to harvesting, cleaning, and analyzing data

from Twitter Using R. Information Systems Education Journal, 15(3), 42–54. Retrieved

from Explora database.

Kabir, A. I., Karim, R., Newaz, S., & Hossain, M. I. (2018). The power of social media

analytics: Text analytics based on sentiment analysis and word clouds on R. Informatica

Economica, 22(1), 25-38. doi:10.12948/issn14531305/22.1.2018.0

COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 34

Krotov, V., & Tennyson, M. (2018). Research Note: Scraping Financial Data from the Web

Using the R Language. Journal of Emerging Technologies in Accounting, 15(1), 169–

181. https://doi.org/10.2308/jeta-52063

Lieberman, P. (2001). Human language and our reptilian brain: The subcortical bases of speech,

syntax, and thought. Perspectives in Biology and Medicine, 44(1), 32-51. Retrieved from

the Student Resources in Context database.

Liu, B. (2009). Sentiment Analysis and Subjectivity. Retrieved from

https://www.cs.uic.edu/~liub/FBS/NLP-handbook-sentiment-analysis.pdf

Liu, L., Luo, D., Liu, M., Zhong, J., Wei, Y., & Sun, L. (2015). A self-adaptive hidden Markov

model for emotion classification in Chinese microblogs. Mathematical Problems in

Engineering. 2015, 1-8. Retrieved from the Science in Context database.

Maldonado García, M. I., & Sandhu, A. H. (2015). The text of the memorandum “Sikhs and the

New Constitution for India” (1930). Political importance and linguistic sentiment

analysis. South Asian Studies (1026-678X), 30(1), 289-303. Retrieved from the Explora

database.

Meyer, D. (2018). Support Vector Machines. Retrieved from https://cran.r-

project.org/web/packages/e1071/vignettes/Support Vector Machinedoc.pdf

Puri, D. (2017). Can Tweets predict intraday stock price movements? Retrieved from

https://digitalcommons.bryant.edu/cgi/viewcontent.cgi?referer=https://www.google.com/

&httpsredir=1&article=1034&context=honors_finance

Pyo, S., Lee, J., Cha, M., & Jang, H. (2017). Predictability of machine learning techniques to

forecast the trends of market index prices: Hypothesis testing for the Korean stock
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 35

markets. PLoS ONE, 12(11), 1-17. Retrieved from the Opposing Viewpoints in Context

database.

Ranco, G., Aleksovski, D., Caldarelli, G., Grcar, M., & Mozetic, I. (2015). The effects of Twitter

sentiment on stock price returns. PLoS ONE, 10(9), 1-21.

http://dx.doi.org/10.1371/journal.pone.0138441

Reed, M. (2016). A study of social network effects on the stock market. Journal of Behavioral

Finance, 17(4), 342-351. https://doi.org/10.1080/15427560.2016.1238371

Rice, K., & Thornton, T. (2013, July). 8. Introduction to R Packages. Retrieved September 17,

2018, from http://faculty.washington.edu/kenrice/rintro/sess08.pdf

SAS. (n.d.). Machine learning. Retrieved September 17, 2018, from SAS website:

https://www.sas.com/en_us/insights/analytics/machine-learning.html

Yang, S. Y., Mo, S. Y. K., & Liu, A. (2015). Twitter financial community sentiment and its

predictive relationship to stock market movement. Quantitative Finance, 15(10), 1637–

1656. https://doi.org/10.1080/14697688.2015.1071078

Zhang, L. (2013). Sentiment analysis on Twitter with stock price and significant keyword

correlation. Retrieved from University of Texas Libraries.

COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 36

Appendix A: R Code

The content of this appendix consists of the R code created to gather and analyze the

tweets and perform the linear regression test. The researcher has kept the consumer key and

access tokens private due to Twitter restrictions and obtained permissions.

install.packages("twitteR")

install.packages(“syuzhet”)

library(twitteR)

library(syuzhet)

#Here we are establishing our keys and tokens from the Twitter API

consumer_key = '****'

consumer_secret = '****'

access_token = ‘****'

access_secret= '****'

#Here we are creating the "handshake" with Twitter

setup_twitter_oauth(consumer_key= consumer_key, consumer_secret= consumer_secret,

access_token= access_token, access_secret= access_secret)

tw<-searchTwitter("Nike -filter:retweets", n=417, lang='en', resultType = "recent")

class(tw)

#Converting the list to a dataframe and selecting specific variables

df1<-twListToDF(tw)

df<-rbind(df1)

df <- subset(df, select=c(text, created))

COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 37

#Storing tweets

write.csv(df,file="tweets",row.names=FALSE)

#Reading the tweets

twitter<-read.csv("tweets")

ls(twitter)

str(twitter)

#Converting the text to the proper format to perform sentiment analysis

twitter$text<-as.character(twitter$text)

#Performing analysis

twitter$sentiment<-get_sentiment(twitter$text, method="syuzhet")

twitter$labels[twitter$sentiment==0]<-"neutral"

twitter$labels[twitter$sentiment>0]<-"positive"

twitter$labels[twitter$sentiment<0]<-"negative"

table(twitter$labels)

write.csv(table(twitter$labels),"table")

#Linear regression

Install.packages(“readxl”)

library(readxl)

#Uploading the percent changes, which the researcher stored into a separate document called

“linreg”

analysis<-read_excel("linreg.xlsx")

ls(analysis)
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 38

head(analysis)

#Making the scatterplot

scatter.smooth(x=analysis$THDR, y=analysis$THDS, xlab="Change in Sentiment Ratio",

ylab="Change in Stock Price",main="Home Depot")

#Performing linear regression

linearmod<-lm(ComcastR~ComcastS, data=analysis)

#Gives coefficients

print(linearmod)

#Gives entire summary of linear regression results

summary(linearmod)
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 39

Appendix B: Raw Data

This appendix consists of a figure with the raw data collected from Twitter and Yahoo

Finance and compiled into the Excel spreadsheet.

Figure 1: Raw Data. This figure shows the raw data compiled by the researcher for each
company. The cost represents the closing price of the stock, and the ratio represents the
positive sentiment ratio.
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 40

Appendix C: Percent Change Data

This appendix consists of the percent changes calculated by the researcher in Excel using

the gathered data.

Figure 2: Percent Change Data. This figure shows the percent change data. The left table
shows the day by day percent changes in the positive sentiment ratio for each company, and
the right table show the day by day percent changes in the stock price for each company.
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 41

Appendix D: Graphs of Collected Data

The content in Appendix D is the graphs associated with the collected data. A single

point on the graphs represents a data collection day.

Figure 3: Nike Graph. This graph depicts the results of the collected data for Nike. Each
point represents a data collection day. For example, the right most data point represents a day
where the sentiment increased by about 27% and the stock price decreased by about 0.5%.
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 42

Figure 4: Amazon Graph. This graph depicts the results of the collected data for Amazon.
Each point represents a data collection day.
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 43

Figure 5: Home Depot Graph. This graph depicts the results of the collected data for Home
Depot. Each point represents a data collection day.
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 44

Figure 6: Comcast Graph. This graph depicts the results of the collected data for Comcast.
Each point represents a data collection day.
COMPARING PRICES OF S&P 500 CONSUMER DISCRETIONARY 45

Figure 7: Disney Graph. This graph depicts the results of the collected data for Disney. Each
point represents a data collection day.