Chart-to-text : a large-scale benchmark for chart summarisation

Abstract

Charts are popularly used for communicating insights and exploring data. However, interpreting charts can be a challenging task. Automatically generated natural language summaries can help readers more easily understand charts by identifying the key insights.

In this report, we present Chart-to-Text, a large-scale benchmark with two datasets containing a total of 44,096 charts covering a diverse range of topics and chart types. We present the dataset construction process and an analysis of the datasets. We also formally define the Chart-to-Text task with two variations: one which assumes the availability of the underlying data table and another which does not. To tackle this problem, we introduce several state-of-the-art neural models, that utilise computer vision and data-to-text generation techniques, as baselines. Through a combination of automatic and human evaluation, we show that while our best models usually generate fluent summaries and yield reasonable BLEU scores, they unfortunately suffer from hallucinations and factual errors as well as difficulties in accurately describing complex patterns and trends in charts.