A No-Reference Adaptive Blockiness Measure for JPEG Compressed Images

Digital images have been extensively used in education, research, and entertainment. Many of these images, taken by consumer cameras, are compressed by the JPEG algorithm for effective storage and transmission. Blocking artifact is a well-known problem caused by this algorithm. Effective measurement of blocking artifacts plays an important role in the design, optimization, and evaluation of image compression algorithms. In this paper, we propose a no-reference objective blockiness measure, which is adaptive to high frequency component in an image. Difference of entropies across blocks and variation of block boundary pixel values in edge images are adopted to calculate the blockiness level in areas with low and high frequency component, respectively. Extensive experimental results prove that the proposed measure is effective and stable across a wide variety of images. It is robust to image noise and can be used for real-world image quality monitoring and control. Index Terms—JPEG, no-reference, blockiness measure


Introduction
Although digital images provide great convenience to our daily life, storing and transmitting these images are problematic. Consumer cameras can produce images with size more than 12 mega pixels. Some of them can even produce images with over 20 mega pixels. The standard JPEG algorithm installed in consumer cameras are commonly used to compress these images. JPEG is a discrete cosine transform (DCT) -based compression scheme. The input image is divided into 8×8 blocks, each block being transformed independently to DCT coefficients. The DCT coefficients are then quantized using a scalar quantization matrix. At low bitrates, the DCT-coded images generally suffer from visually annoying blocking artifacts as a result of coarse quantization. It can dramatically degrade the image quality.
Over the past decades, research has been carried out to alleviate blocking effects in JPEGcompressed images. Effective measurement of blocking artifacts plays an important role in the development and evaluation of these algorithms. The most widely used point-wise comparison measures, namely mean square error (MSE) and peak signal to noise ratio (PSNR), are not very well matched to perceived visual quality [1]. A lot of effort has been put in the development of effective quality measures.
In this paper, we propose a no-reference adaptive blockiness measure for JPEG compressed images. For areas with low frequency component, we use the difference of entropies across blocks as a measure. For areas with high frequency component, we use the variation of block boundary pixel values in edge images as a measure. The final index is the weighted average of the two measures. The rest of this paper is organized as follows. Section 2 gives a short review of the related work. Section 3 introduces our blockiness measure. Section 4 reports experimental results. Section 5 offers some concluding remarks.

Literature Review
Image quality assessment techniques can be divided into two classes: subjective assessment and objective assessment. In subjective assessment, a group of testers are required to rate a batch of images compressed under different ratios. Objective methods try to estimate the compression amount using mathematical techniques. They can be further divided into two types: full-reference and no-reference methods.
Full-reference methods give an assessment through a comparison between reference and compressed images. They mainly utilize the sensitivity of human visual system and a combination of multiple features [2]. Wang and Bovik [3] proposed a universal image quality index which models image distortion as a combination of three factors: loss of correlation, luminance distortion, and contrast distortion. In [4], the measure was generalized to the Structural SIMilarity index (SSIM) to characterize the saturation effects of the visual system at low luminance and contrast regions and to assure numerical stability.
No-reference methods directly evaluate the compressed images. They are more useful because the reference image is usually unavailable. Moorthy and Bovik [5] presented two strategies-visual fixation-based weighting, and quality-based weighting to weight image quality measurements by visual importance, and demonstrated improvements on the SSIM index in both its multi-scale and single-scale versions. Wang and Bovik [6] proposed an algorithm which evaluates blockiness by computing the FFT along the rows and columns. Pan et al. [7] proposed a method based on the fact that for images with more severe blocking artifacts, more edge pixels are arrayed horizontally and vertically. Lee et al. [8] classified blocks by analyzing block boundaries and converted boundary strength into blockiness score. Li et al. [9] presented a blockiness metric which computes the regularities of pseudo structures. In [10], Li et al. proposed a quality measure for JPEG-compressed images based on Tchebichef kernels. Golestaneh and Chandler [11] presented a measure that gets the number of zero-valued discrete cosine transform (DCT) coefficients within each block. Then it uses a quality map to represent these numbers. Freitas et. al [12] proposed a machine-learning-based method that uses the histograms of local ternary pattern (LTP) as features for the training procedure. Li et al. [13] proposed a deep-neural-network-basedalgorithm that extracts features using shearlet transform and evolves the features using stacked auto-encoders. Then the differences of evolved features are identified by a softmax classifier. Ren et al. [14] developed a method which uses shearlet transform to localize the distributed discontinuities induced by image degradation, and exhibits the alteration of image quality by the nature scene statistics of shearlet coefficients.
A common problem with the current blockiness measures is that none of them have adaptability to image characteristics. Different areas in an image usually have different amount of high frequency component. The compression process has different influences on these areas. This property should be considered in the blockiness measure.

Proposed Algorithm
In this paper, we propose a no-reference blockiness measure which is adaptive to different areas in an image. Low frequency component in images corresponds to pixel values that change slowly over space, while high frequency component means pixel values that change rapidly.
Edges in a digital image are the points at which the image brightness changes sharply or has discontinuities. Therefore sharp transients like edges represent high frequency component, while long unchanging spaces correspond to low frequency component. For the areas with low frequency component, the compression process reduces intensity variability inside a block, sometimes even generates blocks of constant gray scale, especially under high compression ratio. The block boundaries produce abrupt change of intensities across blocks. Fig 1 illustrates the influence of JPEG compression on different image areas. Fig 1(A) is an original image of a building, where the square shows an area with low frequency component (see Fig 1(B)). After compression, blocks of constant gray scale are produced (see Fig 1(C)). Comparing Fig 1(B) and 1(C), we find that before compression, the entropies of pixel values inside a block and across blocks are almost the same. However, they are quite different after compression, because of the generated block boundaries. Considering this property, we use the difference between the entropies of pixel values inside a block and across blocks to represent the compression level.
For the areas with high frequency component, the entropy difference is not influenced by the compression process, since the entropy of pixel values inside a block is already high before compression (see Fig 1D-1F). However, the generated blocks do change the edge information. Discontinuities appear around block boundaries in the edge image, and the higher the compression ratio is, the larger the discontinuities are. Therefore, in this case, we use Sobel operator to get horizontal and vertical edge images first. Then we capture the difference of pixel values around block boundaries in the edge images to represent the compression level. Let I be a compressed image, and M x and M y be the Sobel masks defined as follows: The convolution results between I and M x , M y are denoted as: where Ã is the convolution operator, and C x and C y are the horizontal and vertical edge images, respectively. Then C x and C y are decomposed into 8×8 non-overlapping blocks, since DCT transform is performed by 8×8 blocks in JPEG compression. For the k th 8×8 block in the image, if it has edges (with high frequency component), four regions in C x and C y , R ih , R iv , R oh , and R ov , are considered. R ih and R iv denote the pixels just inside the horizontal and vertical boundaries, respectively. R oh and R ov denote the pixels just outside the horizontal and vertical boundaries, respectively. Fig 2A-2D illustrate these regions. We define measure s 1 for the areas with high frequency component. It is calculated according to the following equations: where N i and N o are the number of elements in the summation (here N i = 32, N o = 40). N k is the number of blocks in this area. The measure s 1 is obtained from the luminance variance of pixels inside and outside block boundaries. For the t th 8×8 block in the areas without edges (with low frequency component), the regions I b and I bo are considered. I b denotes an 8×8 block, while I bo represents a 10×10 pixel area with the 8×8 block in the center. Fig 2E and 2F illustrate these regions. We define measure s 2 for the areas with low frequency component. It is calculated according to the following equations: 8 > < > : where En(Á) is the function to compute entropy. N t is the number of blocks in this area. The measure s 2 is obtained from the entropy difference of pixels inside and across blocks. The final blockiness measure is defined as the weighted average of the two measures: where w i (i = 1, 2) is the weight of the two measures which is determined as the total numbers of the two different kinds of blocks in an image. Take Fig 1(A) as an example. After compression, N k , N t , s 1 and s 2 are 637, 1721, 0.220 and 0.812, respectively. In the areas with low frequency component, the average values of En(I b ) and En(I bo ) are 0.366 and 0.734, respectively. It can be seen that the entropy increases after compression, because of the generated block boundaries in I bo . The final blockiness measure s is 0.652.

Experimental Results and Discussion
Extensive experiments were conducted to evaluate the performance of the proposed blockiness measure. In the first experiment, the images "sailing2" from LIVE database [15] with different levels of blocking artifacts were adopted. The bitrates of the three compressed images are 0.3929, 0.1710 and 0.1638, respectively. The subjective evaluating results of the images are represented using difference mean opinion score (DMOS). In general, the higher DMOS an image has, the lower the image quality is. Therefore a reasonable blockiness measure should be proportional to the DMOS values. The blockiness measures given by Wang (8) for the three images and put them in the last column of Table 1. It can be seen that s 2 increases monotonically with the compression ratio, which means that entropy is able to describe compression level.
In the second experiment, we used six benchmark images with different themes to do evaluation. The image sizes vary from 256×256, 320×240, 512×512, to 480×720. These images were compressed under quality factors of 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, and 95, using the baseline JPEG standard in MATLAB. The corresponding bitrates can be obtained accordingly. We compared our results with a classical no-reference measure, Perra's [2], and two full-reference measures, Qi index [3] and SSIM index [4]. Because Qi index and SSIM index give higher results for images with better quality, we use their reciprocals as blockiness indices, i.e., higher index corresponds to more serious blocking artifacts. To have a better observation, we normalize their variations to the range of 0 and 1. Fig 3 reports the experimental results. All of them have areas of different amount of high frequency component. The sensibility to the JPEG blockiness distortion was evaluated using the images compressed at different bitrates. The results show that Perra's [2] does not change monotonically with the increase of bitrates for any of the images. In other words, its value does not represent the blockiness level in an image. Qi index has the property of monotonicity only for the image of 'lena' (see Fig 3  (A)). For the images of 'couple' (Fig 3(B)), 'bird' (Fig 3(D)), and 'statue' (Fig 3(F)), the index varies drastically up and down for bitrates between 0.5 and 2.5. For the image of 'landscape' (Fig 3(C)) and 'lighthouse' (Fig 3(E)), it even increases for bitrates higher than 1. SSIM index decreases monotonically with the increase of bitrates for all the images. However, it is a full-reference measure, so it requires the original image as a reference. The proposed measure has a good property of monotonicity for all the images.
In the third experiment, we evaluated the influence of noise on the proposed blockiness measure. The six benchmark images were adopted again. In this experiment, we added Gaussian white noise of mean 0 and variance 0.01 into the images, and then compressed them using the baseline JPEG standard in MATLAB. The following procedure was the same as in the second experiment. Fig 4 reports the experimental results. It can be seen that even SSIM does not have correct result for the image of 'couple' (Fig 4(B)). The proposed measure still has better performance than other no-reference measures. To evaluate whether the entropy component   space limit, we show the results of one image ("rushmore" from CSIQ database [16]) in Table 2 and Table 3. The bitrates, the blockiness measure and the corresponding s 2 are shown for each variance and each compression level. It can be seen that for any case of the noises, both of the measures increase monotonically with the decrease of bitrates, which means that neither of them is interfered by noise. Therefore, the block differentiation mechanism in the proposed measure which depends on high frequency component will not be influenced by noise. In the fourth experiment, the JPEG images from four image-quality databases, including MICT [17], IVC [18], TID2008 [19], and TID2013 [20], were used to evaluate the overall performance of the proposed measure. The JPEG subsets in MICT, IVC, TID2008, and TID2013 contain 98, 60, 125, and 150 images, respectively. In these databases, 14 of 98, 10 of 60, 25 of 125, and 25 of 150 are reference images. Pearson linear correlation coefficient (PLCC) and root mean square error (RMSE) were used to evaluate the prediction accuracy, and Spearman rank-order correlation coefficient (SROCC) was used to evaluate the prediction monotonicity. These criterions were computed between the predicted scores and the subjective scores, where nonlinear fitting was first conducted to bring them on the same scale [10] [21]. For comparison, the results of some popular no-reference blocking artifact metrics were also provided,  [28], and Liu's [29]. A good quality measure should achieve high  Tables 4 and 5 provide the results, where the best result is marked in boldface. As shown in Table 4, the proposed measure yields the best results of both PLCC and SROCC for MICT, IVC, and TID2013. For TID2008, its SRCC is also the best, and its PLCC is very close to the best result. It can be seen that the proposed measure has a very high correlation with the subjective quality ratings. As for RMSE, the proposed measure also provides the best result for TID2008.
The experimental results show that the proposed blockiness measure is effective and stable across a wide variety of images. It has a good estimation of compression level over different image areas. Moreover, it is robust to image noise and can be used for real-world image quality monitoring and control.

Conclusions
In this paper, we proposed a no-reference JPEG blockiness measure which is adaptive to the high frequency component in different image areas. It uses difference of entropies across blocks and variation of block boundary pixel values in edge images to estimate the blockiness level in an image. Extensive experiments were conducted to evaluate the performance of the proposed measure. The results indicated that compared with other blockiness indices, the proposed measure has satisfying and stable performance across a wide variety of images. It is robust to image noise and has positive perspectives in applications such as image quality monitoring and control.