Please use this identifier to cite or link to this item:
Title: Analysing the simultaneous effects of personalities, demographics, competition and collaboration : a case study with Indian politics
Authors: Lai, Alvin Weijian
Keywords: DRNTU::Engineering::Computer science and engineering::Information systems::Information storage and retrieval
Issue Date: 2014
Abstract: Twitter is widely regarded as a fast growing social networking medium. The massive amounts of data available on twitter encourage researchers to crawl and analyse data from it. The purpose of this project is to create a complete system which contains a twitter crawler, a multidimensional database, a data digester and an analytics engine. The crawler is used for the crawling of tweets, the multidimensional database for storing the crawled data and a data digester for formatting the data crawled before storing it into the database while the analytics engine used to perform analysis on the data. Due to the magnitude of this project, multiple students are allocated to it. The author of this report was involved in the schema design of the twitter database and came up with a list of the types of content of the tweets data to be crawled by the twitter crawler. The data digester and part-of-speech tagging for the analytics engine were developed by the author of this report as well. The testing of the system is spread over a period of 4 days, from 14th to 17th March 2014. Within these 4 days, a total of 8,021,933 tweets were collected. The objective of this project has been met as the data digester was found to have a large amount of data successfully. The collected tweets were also successfully formatted by the data digester and inserted into the database. The testing of the analytical engine shows that the Stanford POS tagger was able to identify the name of politician, political party or a state in India with a recall value of 1 and a precision value of 0.47. In order to crawl more relevant tweets relating to Indian politics, improvement should be made to how the twitter crawler crawls for tweets. Instead of keywords, the crawler should start crawling tweets from a list of twitter users which are very active in Indian politics. Currently the visualizer, operational database and twitter crawler reside on the same server. To improve performance of the overall system, two servers should be used; one for hosting the operational database and one for data warehousing.
Schools: School of Computer Engineering 
Research Centres: Parallel and Distributed Computing Centre 
Rights: Nanyang Technological University
Fulltext Permission: restricted
Fulltext Availability: With Fulltext
Appears in Collections:SCSE Student Reports (FYP/IA/PA/PI)

Files in This Item:
File Description SizeFormat 
  Restricted Access
FYP Report Amended796.05 kBAdobe PDFView/Open

Page view(s)

Updated on Jun 21, 2024


Updated on Jun 21, 2024

Google ScholarTM


Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.