Improved compression, demosaicking and denoising for digital still cameras

Abstract

Digital Still Cameras (DSC) processing consists of a sequence of several basic operations. Since most DSCs acquire color imagery with a color filter array (CFA), capturing only one color sample for each pixel, the other two colors need to be interpolated. This interpolation process is referred to as demosaicking. The reconstructed full-color image then undergoes a set of correction operations that may include color and gamma correction, white balancing, and denoising. Finally, the image is compressed and stored in the memory. Potential improvements of SC processing chains are generally restricted by camera's limited computational resources. Also, once built, the camera's processing algorithms are difficult to upgrade. Hence, a recent trend is to move most DSC operations (demosaicking, denoising) out of the camera to a computer (alternative processing chain). At this point it is not clear whether this trend will continue to develop as several questions remain unanswered. For example, since the demosaicking is to be performed outside the camera, the compression will have to be done directly on the CFA data. However, traditional JPEG compression standard and a new wavelet-based standard JPEG2000 were developed for full color images, and it is necessary to know whether there will be any performance drawbacks if these algorithms are applied to CFA data. In this thesis we develop analytical models to analyze the performance of conventional and alternative processing chains, and show that the alternative chain can outperform the conventional one at a large range of compression ratios. We also show that compression error of JPEG2000 compression has inhomogeneous structure, and suggest a way to reduce this inhomogeneity. For demosaicking and denoising, computer-based processing opens up possibilities to explore advanced and computationally intensive algorithms capable of achieving best-possible quality. Generally, demosaicking and denoising can be considered as low pass filtering processes, and hence might lead to image oversmoothing due to loss of high frequency content. In this thesis we show that performance of demosaicking and denoising can be ubstantially improved by incorporating high frequency content correlations, which exist between color channels, video frames, and even scales of wavelet transformation. For demosaicking, we propose an adaptive filtering approach based on the strong inter-color correlations. For denoising, we propose three wavelet-based approaches that show how incorporation of inter- and intra-scale, inter-color, and inter-frame correlations can improve image quality.