GAN-Based Image Compression Using Mutual Information for Optimizing Subjective Image Similarity

Shinobu KUDO  Shota ORIHASHI  Ryuichi TANIDA  Seishi TAKAMURA  Hideaki KIMATA  

IEICE TRANSACTIONS on Information and Systems   Vol.E104-D   No.3   pp.450-460
Publication Date: 2021/03/01
Publicized: 2020/12/02
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2020EDP7080
Type of Manuscript: PAPER
Category: Image Processing and Video Processing
image compression,  CNN,  GAN,  mutual information,  

Full Text: PDF>>
Buy this Article

Recently, image compression systems based on convolutional neural networks that use flexible nonlinear analysis and synthesis transformations have been developed to improve the restoration accuracy of decoded images. Although these methods that use objective metric such as peak signal-to-noise ratio and multi-scale structural similarity for optimization attain high objective results, such metric may not reflect human visual characteristics and thus degrade subjective image quality. A method using a framework called a generative adversarial network (GAN) has been reported as one of the methods aiming to improve the subjective image quality. It optimizes the distribution of restored images to be close to that of natural images; thus it suppresses visual artifacts such as blurring, ringing, and blocking. However, since methods of this type are optimized to focus on whether the restored image is subjectively natural or not, components that are not correlated with the original image are mixed into the restored image during the decoding process. Thus, even though the appearance looks natural, subjective similarity may be degraded. In this paper, we investigated why the conventional GAN-based compression techniques degrade subjective similarity, then tackled this problem by rethinking how to handle image generation in the GAN framework between image sources with different probability distributions. The paper describes a method to maximize mutual information between the coding features and the restored images. Experimental results show that the proposed mutual information amount is clearly correlated with subjective similarity and the method makes it possible to develop image compression systems with high subjective similarity.