Augmenting image data using noise, rotation and shifting

Abstract

Image augmentation is widely used in preprocessing modules in the field of machine learning. It can not only solve the under-fitting problem caused by the scarcity of data sets, but also increase the robustness of the model, thereby improving the model's generalization ability and reducing over-fitting. In the field of medical images, due to ethical issues in obtaining data sets, the number of samples for some disease pictures is insufficient, especially for some difficult and complex diseases. Therefore, image augmentation for scarce sample images is very necessary. Many studies have shown that through methods such as rotation, translation, and flipping, the model can learn features at different angles and scales, thereby reducing the dependence on certain features and increasing the robustness of the model. Some studies have also found that artificially injecting noise into images, such as Gaussian noise, Poisson noise, etc., is regarded as a kind of regularization to reduce the problem of model overfitting. In this article, we will use the real chest X-ray (CXR-8) data set to explore the impact of different image augmentation methods on model performance through metrics such as area under the curve (AUC) and standard deviation (SDV), and provide confusion Matrix to help analyze error samples and propose some additional pre-training methods to further optimize the model. We conclude that methods such as rotation and translation can more effectively improve the accuracy and stability of the model. The best performing method is the rotation augmentation method. Compared with the original data set, the AUC score is increased by about 6.9% in average. Combined with the histogram equalization technology, the AUC score can be further improved to 10.2% compared with the original data set in case of 90,000 samples. Although noise injection slightly improves the AUC of the model, considering the SDV factor, it cannot be judged that it is necessarily better than the model trained on the original data set.