Please use this identifier to cite or link to this item:
https://hdl.handle.net/10356/55019
Title: | MapReduce for data analytics | Authors: | Roy Ananya. | Keywords: | DRNTU::Engineering::Computer science and engineering | Issue Date: | 2013 | Abstract: | The author’s final year project is a part of the Green Campus project which aims to conserve energy by using smart technologies. For developing smart technologies that conserve energy, historical data about energy resource usage has to be analysed by building mathematical models to uncover patterns and correlations and be able to predict abnormal energy usage. The historical data to be analysed can be huge in size if accurate mathematical models need to be built. Processing this huge data set using sequential programming is not possible if the time complexity of the algorithm is not very efficient. Open source frameworks like Hadoop and the MapReduce programming paradigm have made it possible to process huge data sets in parallel on a cluster of machines. As part of this project the author has designed and implemented a RapidMiner customized operator using MapReduce framework for a Hidden Markov Model based outlier detection of power consumption data. The MapReduce version of the algorithm has then been analysed for accuracy as well as a timing analysis of a dynamic programming implementation of the algorithm vis-à-vis the MapReduce implementation has been done. The time complexity of the MapReduce version of the model developed by the author, when run on a cluster of 8 machines is linear whereas the time complexity of the dynamic programming implementation of the same model is exponential. The accuracy of the model built by the author is between 80% to 100%. | URI: | http://hdl.handle.net/10356/55019 | Schools: | School of Computer Engineering | Rights: | Nanyang Technological University | Fulltext Permission: | restricted | Fulltext Availability: | With Fulltext |
Appears in Collections: | SCSE Student Reports (FYP/IA/PA/PI) |
Files in This Item:
File | Description | Size | Format | |
---|---|---|---|---|
ReportFinal5.pdf Restricted Access | FYP Report | 1.83 MB | Adobe PDF | View/Open |
Page view(s) 50
482
Updated on Mar 29, 2024
Download(s)
10
Updated on Mar 29, 2024
Google ScholarTM
Check
Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.