Big data analytics for smart transportation

Abstract

Data are being generated every day as technologies continue to strive. Data are also being generated by vehicles every day as well. We can analyse these data from vehicles to understand how people navigate around a place. The objective of this project is to understand the traffic patterns in Singapore by mining various traffic data (e.g. smart card data of bus and MRT, taxi data), develop some advanced algorithms to assist in urban transport. This project aims to develop a system to process, visualize and analyse traffic data of Singapore. The previous FYP students that worked on this project had process and visualize traffic data of Singapore. Data in the database are also sliced and clustered together in an attempt to speed up the visualization process. However, there are some improvement that can still be made. First, initial load on the traffic visualization is fine (due to data slicing), but as we zoom further into the map and as more data are being visualize, the system gets unresponsive. Second, the data on the map does not show directions. Meaning, we will not know if the road is a unidirectional or bidirectional road. Thirdly, it is slow when we visualize another day/time from the current day/time. The main objective of the current phase of the project is to further analyse the traffic in Singapore with new data and with improved speed in visualization. To speed up the visualization, data integration and database tables have been redesigned to achieve optimal performance. Visualization was also speed up by using newer technologies such as WebGL API on the front-end of the system. Python was migrated from version 2.7 to version 3.6 as well (with Django to ver2.0). Addition of Bus and pathway data were added to visualization as well. In conclusion, we have made improvement to the system as listed above and successful speed up the overall visualization time.