Comments summarization

Abstract

Comments are generally short pieces of text that one uses to share their opinions online. With the increasing popularity of social media, the act of commenting to share the thoughts and opinions of a person online has been becoming more widespread. This often causes small discussions to occur in the comments section of web page and in turn causes a large number of comments to be posted over a short timeframe.

Some of these social media platforms like Facebook and Google+ attempts to control the way these large amounts of comments are displayed by employing various ranking mechanism to display only comments that has a high number of interaction or comments that are most recent. This brings about a disorganization of the comments thus making it hard to follow the comments or discussions posted by other readers.

In this project, we aim to alleviate such issues by implementing a method to produce a summary of comments that are related to news articles obtained from news website Yahoo! News. Through the use of Natural Language Processing(NLP) and Information Retrieval(IR) techniques, the program implemented in this project is able to generate a summary that contains comments that are of a certain level of relevance to the topic that is covered by a news article.