3-2. Assessment of the Sentimental Analysis Tools

There are several possible causes for the accuracy differences between the tools. For instance, “domain-transfer problem” would be one of them. Sentimental classifiers like “Corenlp” have an ability that improve themselves. Supervises such as human programmers can teach them the ways to deal with specific domains. By doing so, those software is able to show high performance on the supervised field. On the other hand, if the supervised sentiment classifiers are used to analyze the different domains (unsupervised domains), their performance becomes very low. This phenomenon is called "domain-transfer problem" (Tan, Cheng, Wang, & Xu, 2009). Probably, this explains the cause of the lowest accuracy of “Corenlp” among the three applications. Unlike other two which were Web-based software, “Corenlp” was Java-based program and must had been trained for the domain of “News headline” in order to evaluate it properly. The other reason for the “Negative-oriented” analysis of Sentimental classifiers would be the latent nature of “News Headlines”. As some people suggested,“News Headlines” tend to be negative because readers are more attracted by negative headlines than positive ones (Headlines: When, 2013). What is more, unfortunately, there is a high possibility that the immatureness of the research in terms of the survey process caused the inaccurate outcome.

(All the results for this investigation are appended to the end of this thesis as appendix.)

Contents for my thesis

              This report shows the topics which I would handle with in my graduation thesis. Currently, there are a lot of subjects which I hope to look into, but it would be necessary to narrow down the options in order to make my thesis more plain and understandable for readers as well as for myself. As previously conducted for evaluating methods, this time, I also assessed some of the topics and tried to find out what I should treat for my final report.

              First of all, my most focused theme for my final report must be “Sentimental Analysis”. This is because the method seems relatively new compared to other linguistic analysis approaches and, in Japan, many people still do not know even what it is. As a result, there is no Japanese page on Wikipedia for this jargon yet. If I analyze various phenomenon using this technique, my thesis can be unique one which other Japanese scholars have never written. In addition, thanks to the recent the rapid development of computer technology, there are a lot of software which conduct diverse linguistic analysis including “Sentimental Analysis”. The assessment for those would be also interesting and make my thesis instructive.

              In order to make the most use of the latest mean, I would need to find the proper subjects to analyze with it. Currently, there are some categories for that. First one is the media trend about “Syrian refugee crisis”. This topic seems to have notable changes over the periods, so we would be able to find interesting results from them. Second is the news reports for 2016’s US election. As you already know, the political race this year have been one of the most unique ones. Therefore, we could expect a lot of various sentimental elements in the news for it. Thirdly, Japanese sensational movement “Kanryu (Korean wave)” might be one of the candidates for my research. This is because the Korean movement in Japan had had strange trend changes and it had seemed that media coverages had had great influence for it. 

             In addition to “Sentimental Analysis”, I also want to look into its analysis tool itself. As I stated above, so many new evaluation tools for “Sentimental Analysis” are there these days. So scrutinizing those would lead to very interesting outcomes for my thesis.

              Finally, by writing this report, I could find that deciding the topics for my thesis is much more difficult and critical thing than looking for some methods or data. It would be necessary for me to trim away the needless parts and focus on the most important ones so as to make my thesis simple and understandable.   

“Introduction (Temporary)”

“Introduction (Temporary)”

In the modern times, the media coverages have played an important role to pervade various events around the world. In addition, they have a crucial power to influence on the public images or opinions on certain topics. These days, the media trends created by news agencies have been analyzed in many ways. One of the unique methods is to analyzing the news headlines instead of the news contents.

In 1973, Charles Grivel, professor of Modern French literature at the University of Mannheim, focus on the title’s potential abilities and defined its functions as “Identify the work”, “Designate the work's subject matter” and “Play up the work”. (*1) Those title evaluation is now applied for the analysis of news headlines as well. Also, a number of sentence analysis methods have been proposed. “Sentimental Analysis” is an text analysis which categorize sentences into specified emotional groups like “sad”, “fun”, “negative” and so on. This shows what kind of sentimental elements those sentences have and we can assume what influence they have. Recently, thanks to the rapid technological development, a lot of computer software which analyze text implications have been created by major companies or researchers.

Likewise, the research on the influence of the media have been a popular subject.

For example, “Bandwagon effect” and “Underdog effect” would be well known theories which illustrate the media coverages’ leverage with people’s decisions on the diverse issues.

“Bandwagon effect” is an assumed movement that people tend to support for a person or a thing which seems to gain majority backing so that their actions can be effective. On the contrary, “Underdog effect” is a phenomenon that leads people to support the “Underdog” no matter their aids may be useless. At the same time, is also means that the backers of the popular ones occasionally think that their help might not be necessary and they quit assisting them. (*2)

In this report, we analyze the media trend over the Syrian refugee crisis using various methods. The main reason for choosing this subject is that many people these days begin to think the media reports are not reliable enough. (*3) Similarly, Syrian refugee crisis is a perfect example to exemplify the transitory traits of the public reactions. Moreover, the historical tragedy includes many aspects which the world today is facing with. For example, multiculturalism, Muslimazation, anti-immigrant sentiment, and things like that. The aim of this study is mainly to show how the public reactions towards the event have changed over a period and what influence the media coverages have attributed to them. Besides, we also try to evaluate the reliabilities of the new tools which are being developed to analyze the various social tendency including the media trend.

One of the research questions for this thesis is “How have the public reactions changed as the trends of the media coverages have shifted?” The hypothesis for this query is that the increase of the news reports which contains sympathetic elements towards the refugees will contributed the improvement of their public image and vice versa.   

This paper consists of several experiments regarding to the various reactions over the Syrian refugee crisis. First, we seek for the media trend about the event and its influence on the public opinions by evaluating headlines and news articles. Secondly, the results of analysis tools are shown and careful reviews for them are also conducted. Finally, the summary of the media influence based on those results of the experiments is illustrated.

Brief Reference

*1

https://books.google.co.jp/books?id=AmWhQzemk2EC&pg=PA76&lpg=PA76&dq=Charles+Grivel+title&source=bl&ots=4augxFfLRO&sig=6vKFlcGcwYYny1HUqUvwJxuwOk0&hl=ja&sa=X&ved=0ahUKEwjY4cm85MLPAhWBv5QKHdzXDIQQ6AEIPzAE#v=onepage&q=Charles%20Grivel%20title&f=false

*2

https://books.google.co.jp/books?id=Q6u5ADXTpR4C&pg=PA95&lpg=PA95&dq=media+trend+poll+results&source=bl&ots=WkC1zcI5WU&sig=p2zNW7y_nRWK5x0SFLzTg8cBXZs&hl=ja&sa=X&ved=0ahUKEwim1tyJw83PAhVCtJQKHRSoDQwQ6AEIPjAE#v=onepage&q=media%20trend%20poll%20results&f=false

*3

http://www.journalism.org/2016/07/07/trust-and-accuracy/

News Headlines

Abstract

              For this week’s assignment, I conducted research on sentimental analysis for news headlines. Especially, this time I collected and analyzed several news articles from three periods (2000, 2007 and 2015).

Result review

              Thanks to this experiment, I could find a points that is needed to be dealt with. The point is that all of the result for news headlines focused on the keyword “immigrants” tend to “Negative” or “Neutral”. “Positive” articles were quite rare and hardly found them.

Maybe I need to find another analysis method for it to improve my analysis reliabili

*Result (P: Positive / N: Negative / L: Neutral)

2000（BBC）

News headlines	My Reaction	Corenlp	Text Analysis	Result
Immigrant suspects found in lorries.	N	L	L	-
Immigration or extinction.	N	L	L	-
NY police 'deliberately' killed immigrant.	N	N	L	N
'Immigrant' deaths shock papers.	N	N	L	N
Should immigration policy be relaxed?	L	P	L	-
Spain struggles with race riots.	N	L	L	-
German CDU says immigrants must 'conform'.	L	N	L	L
US immigration policy branded 'racist'.	N	N	N	N
Mutual fears behind Spain's race riots.	N	L	L	-

2007（BBC）

News headlines	My Reaction	Corenlp	Text Analysis	Result
Joining the immigrant underclass.	N	L	L	-
The BIG Immigration Debate.	L	L	L	L
Health | Immigrant pregnancies stretch NHS.	L	N	L	L
England | London | Violent immigrants fuelling crime.	N	N	L	N
Hispanic immigrants in the USA.	L	L	L	L
Migrants 'deceived and exploited'.	N	N	L	N
Immigrants re-energise Irish Church.	P	L	L	-
Panorama | Immigration: How we lost count.	N	N	N	N
Immigration to the USA.	L	L	L

 

News headlines	My Reaction	Corenlp	Text Analysis	Result
Immigrant spouses must speak English, court rules.	N	N	L	N
'Blatant' illegal immigrant job offers in Sussex and Kent.	N	N	N	N
Immigration Bill: Landlords 'must evict' illegal immigrants.	N	N	N	N
UK net migration hits record high.	N	P	L	X
The battle over the words used to describe migrants.	N	N	N	N
Illegal immigrants smuggled out of UK and back again.	N	N	L	N
We are beautiful too, say women from immigrant backgrounds.	P	P	L	P
Under the radar with the UK's illegal migrants.	N	L	L	X
EU immigration rules - in 90 seconds.	L	L	L	L

Headline analyses

*Abstract

             

              For this week’s experiment, I did continuously research on the methods of “Sentimental Analysis” for news headlines. Previously, I found two software which can analyze sentences. They are “Corenlp” and “Text analysis (Google sheet’s add-on)”. Therefore, this time, I tried to find the way that increases the credibility of the results of Sentimental analysis using them.

*Method

              As you instructed in the last meeting, I compared the three results of “my reaction”, “Corenlp” and “Text Analysis” to calculate the credible sentimental trend. Then, I also searched for the news headlines which all three criteria regarded as the same category (Positive, Negative and Neutral).

*Result (P: Positive / N: Negative / L: Neutral)

News headlines	My Reaction	Corenlp	Text Analysis	Mean
UK net migration hits record high.	L	P	L	L
Who will be banned under Trump's immigration plan?	L	L	L	L
Eastern European workers in UK pass one million.	N	L	L	L
Australia's immigration minister accuses asylum seekers of lying.	N	N	N	N
Germany's AfD leader wants failed asylum seekers housed on islands.	N	N	N	N
Changing attitudes of a German city.	L	L	N	L
Anti-Islam group storms Anglican church in Australia.	N	L	L	L

*Conclusion

All in all, the result of the experiment was useful in that I could find some possibility that there might be some ways to evaluate the sentimental analysis for news headline using those software. However, I found that there were also many drawbacks to fix in order to get credible information. For example, the reliability of the assessment is still not so strong. It would be necessary for me to learn from the similar themes to make my research method more academically suitable.

Useful Sites for analysis

*Abstract

             

              This time, I scrutinized two research sites which enable me to analyze the sentimental trend among people. One is “Social Searcher” which shows us a lot of useful information about particular keywords. The other is "Google Sheets" that has many remarkable evaluation methods in it. I actually used them, and tried to find out whether it is convenient or inconvenient for my experiment.

---“Social Searcher”---

[Review]

This site can evaluate the detailed sentimental analysis for various SNS community sites including “YouTube”, “Instagram”, “Twitter” and things like that. Also it can evaluate linguistic features like “types of users” and “Keywords density”. However, unfortunately, there are some drawbacks as well. For example, “Social Searcher” has no option to search the result data by the certain date. This means if you want to get information about public reactions one or two years ago, it would be quite difficult because of the deficiency. Given its unstable trait, collecting past data from SNS like “Twitter” might be so complicated. 

---“Google Spread Sheets (Using its add-on “Text Analysis”)”---

[Review]

              Nowadays, there are many computer software which can calculate sentimental analysis. Those divide sentences into three categories “Positive”, “Neutral” and “Negative”. For this week’s report, I happened to find another analysis application called “Google Spread Sheets”. “Google Spread Sheets” is a software which is provided by Google. It is quite similar to “Microsoft Excel” and many add-ons are available for it. “Google Spread Sheets” does not support sentimental analysis. However, its add-on “Text Analysis” enable us to evaluate sentences’ trend. The below is the example of the add-on function.

*For comparison, I attach the result of the Corenlp’s analysis.