Restoring blood vessel patterns and tattoos from JPEG compressed skin images for forensic analysis

Abstract

Using digital evidence images for criminal and victim identification in some legal cases, such as child sexual abuse and masked gunman, can be very challenging, because the faces of criminals or victims are not visible. Blood vessel patterns under human skin and tattoos have been proposed as biometrics to overcome this challenge. However, these images are always taken by consumer cameras and compressed by the JPEG compression method. As a lossy compression method, the JPEG compression method seriously degrades the clarity of the blood vessel patterns and tattoos. To finally utilize blood vessels and tattoos for identifying criminals and victims or searching suspects, overcoming this challenge from the JPEG compression method is essential. Existing methods are not suitable for restoring the blood vessel patterns and tattoos because most of them are designed for generic images and their main targets are to remove the blocking artifacts and improve the visual quality. In this thesis, a skin image analysis and a compression test are performed to study the characteristics of compressed skin images. Then, based on the skin image analysis and the compression test, two skin image restoration algorithms are specially designed to restore blood vessel patterns from compressed images. Finally, an algorithm based on deep learning is proposed to recover tattoo images from their degraded versions. Skin image databases and tattoo image database are constructed for algorithm development. To recover the blood vessel patterns from compressed skin images, a skin image analysis and a compression test are first conducted to find out the critical factors influencing blood vessel pattern quality. The skin image analysis aims to figure out how blood vessel information distributes in the luminance channel (the Y channel) and the chrominance channels (the U and V channels). The compression test is further performed to identify how the lossy operations in the JPEG compression method degrade the blood vessel pattern quality. The down-sampling operation and the quantization operation are examined separately in the test. The importance of the Discrete Cosine Transform (DCT) coefficients is also tested step by step. Based on the analysis, two restoration algorithms are designed to recover blood vessel patterns by estimating the original critical DCT coefficients. Firstly, a global algorithm is developed to recover the critical DCT coefficients. JPEG compressed images and original images are used to construct image patch pairs. The DCT coefficients in the JPEG compressed images are extracted as features and the critical DCT coefficients from the original images are extracted as targets. Regression models are learned to predict the original DCT coefficients. The models are applied to the JPEG compressed image uniformly to restore the images. Experimental results show that the proposed algorithm restores blood vessels more effectively than the state-of-the-art deblocking methods in terms of matching accuracy. Secondly, a multi-model restoration algorithm (MMRA) is presented to remove blocking artifacts in JPEG compressed skin images and restore the lost information. Two mathematical properties in the JPEG compression process are identified and used to design MMRA. MMRA is based on a tailor-made clustering scheme to group training data and learns a model, which predicts original DCT coefficients, from each grouped dataset. Under a minor assumption, MMRA is mathematically guaranteed that it always offers lower training error comparing with the corresponding global method without using the clustering scheme. Experimental results demonstrate that MMRA outperforms the existing deblocking methods for different scales and different compression levels and effectively restores blood vessel information. A public skin image database and an Internet image database are constructed for blood vessel restoration algorithm development. The public skin image database has two datasets, a high resolution image dataset and its downscaled version. Images in this database are used to perform the skin image analysis, compression tests, model training and algorithm evaluation. The Internet image database also contains two datasets. Both the two datasets are used to evaluate the proposed algorithms, while images from the first dataset are used for the visual comparison and objective evaluation based on quality metrics and images from the second dataset are used for the evaluation based on blood vessel matching. For restoring tattoo images, a tree-structured deep framework is developed to recover the original DCT coefficients in the frequency domain. All the coefficients are arranged in the order of zigzag and divided into three groups according to their frequency, the low frequency group, the middle frequency group and the high frequency group. The proposed framework takes the image patches in the RGB domain as inputs. The DCT coefficient residuals, i.e., the differences between the original DCT coefficients and the compressed DCT coefficients are employed as the target. In the training phase, three parallel branches are trained simultaneously to recover the three groups of DCT coefficients. A constraint layer is specially designed to guarantee that the outputs of the deep framework are in proper ranges. Since the DCT coefficients are not equally important to the image quality, the weighted mean squared error function is utilized as the loss function. Tattoo images collected from the Internet are used to construct a tattoo image database for algorithm design and evaluation. Extensive experiments show that the proposed algorithm can suppress the blocking artifacts and recover the missing information effectively. The proposed algorithm outperforms the state-of-the-art deblocking methods in terms of image quality metrics.