Perceptual-Based Quality Metrics for Image and Video Services

Other types of images can also be treated as layered images via the use of bit plane separation. This assigns one bit from the binary representation of the pixel value into 20 each bi-level layer, thus lossless separating any gray scale image into eight layers. In the case where there is a correlation between the bit layers, it is possible to utilize this to gain better compression efficiency. For example, in context modeling, involving neighboring pixels from already processed binary images can improve the probability estimation and, therefore, the compression. Among existing implementations we can mention the EIDAC (Mallat, 1999) lossless compression algorithm, which uses a binary multi-layer context model that operates on bit-layers of the image using both the actual bit values and their differential characteristics as context information. Two-layer context modeling with optimization of the order of layer processing was considered in(Balasubramanian, Allebach, 1995). In P2, we study how well the bit-plane-based approach can work on natural and palette images. We consider four different bit plane separation schemes: straightforward bit plane separation, Gray-coded bit plane separation, bit plane separation of prediction errors and separation of Gray coded prediction errors. We use the highly optimized MCT context modeling method for lossless compression and, furthermore, extend the two-layer MCT model to a multi-layer context model for better utilization of cross-layer dependencies. In general, any previously compressed layer can be used to provide the contextual information for the next layer being compressed. An example of a multi-layer neighborhood used in P2 is presented in Figure 6. We extensively evaluate the proposed combinations of the different bit plane separation and context modeling schemes, by applying them to natural and palette images. The efficiency of the bit-plane-based compression is compared to the existing compressors. Moreover, the dependency of the compression on the image content is studied by modeling the transition between natural and palette image classes. Here, the integral parts of a wireless link are shown including source encoder, channel encoder, modulator, and the wireless channel. In this scenario, the received image or video may from artifacts, and consequently quality degradations, from both the source encoder and the error-prone wire-less channel. The impact of the source coding artifacts is somewhat easier to predict since for different codes certain artifacts can be expected. On the other hand, the time variant nature of the fading channel makes the range of artifacts in the received signal much more unpredictable. With JPEG source encoding and a Rayleigh °at fading wireless channel with additive white Gaussian noise (AWGN). The symbiosis of this setup resulted in a wide range of different artifacts, hence, substantially complicating the assessment of the artifacts and the related visual quality. The additional shaded boxes in comprise of the necessary components to facilitate perceptual quality assessment, as we propose it in this study. The blocks surrounded by dashed lines indicate optional parts of the quality assessment, which are applied when reference features are extracted from the

This latter goal is followed by incorporating several characteristics of the HVS into the metric design to allow for superior quality prediction performance as compared to metrics that purely measure similarity between images (Clarke, 1985). In order to further support the design of the objective metrics we have conducted subjective image quality experiments at the Western Australian Telecommunications Research Institute (WATRI) in Perth, Australia, and at the Blekinge Institute of Technology (BTH) in Ronneby, Sweden. The mean opinion scores (MOS) obtained from these experiments allowed us to relate the different measures incorporated in the objective metrics to subjectively perceived visual quality. The MOS further enabled evaluation of the quality prediction performance of the metrics on both a set of training images that were used for the metric design and a set of validation images that were unknown during metric training. Unlike previously proposed HVS based quality metrics (Gonzalez, Woods, 2002. Pitas, 2000. Dougherty, Astola, 1997) that incorporate a large number of HVS properties, we focus on a few simple approximations of HVS characteristics that have been shown to be essential for the visual perception of quality. Specifically, the basis for the metric designs is motivated by the phenomenon that the HVS is adapted to extraction of structural information (Gersho, Gray, 1992. Pitas, Venetsanopoulos, 1990). Thus, a number of structural features are extracted that accurately quantify the artifacts observed in wireless image and video communication. An additional weighting then controls the impact of each feature on the overall metric. The weights are derived in relation to the MOS from the experiments and thus account for the perceptual relevance of each of the artifacts. Additional HVS characteristics, such as, multiple-scale processing and regional attention will be shown to further enhance the metrics quality prediction performance. The latter characteristic has been supported by an additional subjective experiment that we conducted at BTH to identify regions-of-interest in the set of reference images and thus allow for implementation of region- a last step subject to an exponential mapping. The mapping translates the metric values into so-called predicted MOS which aim to measure the quality as it would be rated by a human observer.

The metric is based on the work conducted earlier in (Kenney, et. al., 2001. Ageenko, Fränti, 2000. Plataniotis, et. al., 1997).

The structural feature algorithms included in the objective metric are implemented according to algorithms as outlined in different publications (Gonzalez, Woods, 2002. Pitas, 2000. Dougherty, Astola, 1997. Pitas, Venetsanopoulos, 1990. Kuwahara, et. al., 1976). Consequently, the ranges of the different features were strongly varying. In this work, we therefore introduce extreme value

An alternative feature pooling based on a perceptual relevance weighted LP-norm is proposed. The resulting metric provides similar quality prediction performance as NHIQM while at the same time allowing tracking the structural degradations independently for each of the features. Thus, insight into the artifacts induced during transmission may be gained using this feature pooling.

An in-depth statistical analysis is provided for the subjective experiments that we conducted in two independent laboratories. The analysis reveals the relevance of the subjective scores obtained in the experiments. In addition, a detailed analysis of the objective feature scores from the experiment test images is discussed revealing insight into the artifacts that were objectively quantized by the feature metrics.

The concept of metric training and validation has further been introduced to the work to verify that the metric design does not result in over writing to the training data but rather allows for good generalization to unknown images.

Using the training and validation approach we further motivate the use of an exponential prediction function to account for the non-linear processing in the HVS. Other prediction functions can be excluded due to inferior goodness of measures, visual inspection, and over fitting on the training set of images.

State of the art visual quality metrics (Gersho, Gray, 1992. Ding, et. al., 2007. Gonzalez, Woods, 2002) are considered in this work for comparison of quality prediction accuracy, prediction monotonicity, and prediction consistency (Mallat, 1999) on both the training and the validation set of images. The evaluation reveals the superior quality prediction performance of NHIQM with respect to all three criteria.

A. Gersho, R.M. Gray (1992). Vector quantization and signal compression. Norwell, MA: Kluwer. Ageenko E., Fränti P. (2000). "Context-based filtering of document images", Pattern Recognition Letters, 21 (6-7), pp. 483-491, Elsevier Science. C. Kenney, Y. Deng, B.S. Manjunath, G. Hewer (2001). “Peer group image enhancement”, IEEE Trans. on Image Processing, vol. 10 (2), pp. 326-334. C. Ueffing (2001). “Wavelet based ECW image compression”, Photogrammetric Week 01, Wichmann Verlag, Heidelberg, pp. 299–306. Dougherty E.R., Astola J. (eds) (1997). Nonlinear Filters for Image Processing, SPIE Optical Engineering Press. Gonzalez, R. C., Woods, R. E. (2002). Digital Image Processing, 2nd ed., Prentice Hall, Upper Saddle River, NJ. I. Pitas (2000). Digital Image Processing Algorithms and Applications, New York: Wiley. K. N. Plataniotis, D. Androutsos, and A. N. Venetsanopoulos (1997). “Color image filters: The vector directional approach,” Optical Engineering, vol. 36, no. 9, pp. 2375–2383. L. Bottou, P. Haffner, P. G. Howard, P. Simard, Y. Bengio, Y. Le Cun (1998). “High Quality Document Image Compression with DjVu”, LizardTech, “MrSID Technology Primer”, http://www.lizardtech.com/files/geo/techinfo/MrSID_Tech_Primer.pdf M. Kuwahara, K. Hachimura, S. Eiho, and K. Kinoshita (1976). “Digital Processing of Biomedical Images”, Plenum Press, pp. 187-203, New York, NY. Pitas, I., Venetsanopoulos A.N. (1990). Nonlinear digital filters: principles and applications, Boston, Mass.: Kluwer. R. Balasubramanian, J.P. Allebach (1995). “Sequantial scalar quantization of vectors: an analysis”, IEEE Trans. on Image Processing, vol. 4 (9). R. J. Clarke (1985). Transform Coding of Images, Academic Press. S.-C. Pei, C.-M. Cheng (1995). “Dependent scalar quantization of color images”, IEEE Trans. on Circuits and Systems for Video Technology, vol. 5(2), pp. 124-139. W. Ding, Y. Lu, F. Wu, S. Li (2007). “Rate-distortion optimized color quantization for compound image compression,” Proc. SPIE Visual Communications and Image Processing (VCIP 2007) (6508-98), San Jose, CA, USA. Y.-Kheong Chee. “Survey of progressive image transmission methods”, International Journal of Imaging Systems and Technology, Special Issue: Image and Video Compression, vol. 10(1), pp. 3-19.