{"id":292,"date":"2015-06-29T18:25:52","date_gmt":"2015-06-29T17:25:52","guid":{"rendered":"http:\/\/www.compucara.ie\/articles\/?page_id=292"},"modified":"2015-06-29T18:25:52","modified_gmt":"2015-06-29T17:25:52","slug":"video-quality-metrics-psnr-v-ssim","status":"publish","type":"page","link":"https:\/\/www.compucara.ie\/articles\/?page_id=292","title":{"rendered":"Video Quality Metrics PSNR v SSIM"},"content":{"rendered":"

\u00a0Video Quality Metrics<\/strong><\/p>\n

Digital images are subject to a wide variety of distortions during acquisition, compression storage, transmission and reproduction.\u00a0 Any of these may result in degradation of the image quality.\u00a0 To ensure an acceptable level of quality control, video metrics are applied to judge the quality of the image.\u00a0 Ideally the image quality assessment process would supply image quality metrics that would predict the image quality automatically.\u00a0 In reality subjective tests are carried out which require human experts to view the video and assess it for quality applying the Mean Opinion Score(MOS) weighting.\u00a0 This is time consuming and expensive.\u00a0 Objective tests can be carried out automatically, these objective quality metric systems predict visual quality by using information known about the human visual system to compare signals.<\/p>\n

This essay will compare the performance of PSNR and SSIM – two objective video quality metrics.\u00a0 For this purpose two video clips will be employed, the video clips will be compressed using the MPEG-4 codec to 128Kbps output rate and to 1Mbps output rate, which are suitable for low speed and high speed internet connections respectively.<\/p>\n

Ffmpeg a computer programme which can record, convert and stream audio visual content in many formats was used to convert the files to mp4.\u00a0 The command line is as follows:<\/p>\n

ffmpeg \u2013i coastguard_cif.avi \u2013vcodec MPEG4\u00a0 – b 1Mb \u2013 bf 1 \u2013an \u2013psnr \u2013vstats coastbig.mp4<\/p>\n

-i filename, -vcodec: the codec to be used, -b: the output bit rate, -bf: the number of B frames to be included between the P frames, -an: specifies no audio, -vstats the production of statistics,\u00a0 -psnr include the psnr details.<\/p>\n

MSU video quality measurement tool version 2.0 was the programme used to produce SSIM information.
\n<\/strong>The following tables detail information on the two video clips, information is included for the original clips and the output from the two MPEG-4 compressions performed for each.<\/p>\n

Coastguard.avi<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n\n\n
\u00a0<\/strong><\/td>\nOriginal file<\/strong><\/p>\n

Coastguard_cif.avi<\/strong><\/td>\n

Compressed with Mpeg-4 for Internet slow\u00a0 connection <\/strong><\/td>\nCompressed with MPEG-4 for Internet fast connection<\/strong><\/td>\n<\/tr>\n
File Size<\/strong><\/td>\n45.6 Mbytes<\/td>\n351Kbps<\/td>\n1.506MBps<\/td>\n<\/tr>\n
Frame size<\/strong><\/td>\n352 x 288<\/td>\n352 x 288<\/td>\n352 x 288<\/td>\n<\/tr>\n
Bit rate <\/strong><\/td>\n30421Kbps<\/td>\n145Kbps<\/td>\n914Kbps<\/td>\n<\/tr>\n
Frame rate<\/strong><\/td>\n25 fps<\/td>\n25 fps<\/td>\n25 fps<\/td>\n<\/tr>\n
Length<\/strong><\/td>\n12 secs<\/td>\n12 secs<\/td>\n12 secs<\/td>\n<\/tr>\n
Compression factor <\/strong><\/td>\n<\/td>\n125<\/td>\n29<\/td>\n<\/tr>\n
Frame size<\/strong><\/td>\n152105 bytes<\/td>\n72 \u2013 27018bytes<\/td>\n113 -27018 bytes<\/td>\n<\/tr>\n
Recognition<\/strong><\/td>\n<\/td>\nSignificant blocking, particularly when there is activity in the scene<\/td>\nVery little variance from original<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

News.avi file<\/strong><\/p>\n\n\n\n\n\n\n\n\n\n\n\n
\u00a0<\/strong><\/td>\nOriginal file<\/strong><\/p>\n

news_cif.avi<\/strong><\/td>\n

Compressed with Mpeg-4 for Internet slow\u00a0 connection <\/strong><\/td>\nCompressed with MPEG-4 for Internet fast connection<\/strong><\/td>\n<\/tr>\n
File Size<\/strong><\/td>\n45.6 MB<\/td>\n339KB<\/td>\n1.532MB<\/td>\n<\/tr>\n
Frame size<\/strong><\/td>\n352 x 288<\/td>\n352 x 288<\/td>\n352 x 288<\/td>\n<\/tr>\n
Bit rate <\/strong><\/td>\n30421Kbps<\/td>\n197Kbps<\/td>\n1007Kbps<\/td>\n<\/tr>\n
Frame rate<\/strong><\/td>\n25 fps<\/td>\n25 fps<\/td>\n25 fps<\/td>\n<\/tr>\n
Length<\/strong><\/td>\n12 secs<\/td>\n12 secs<\/td>\n12 secs<\/td>\n<\/tr>\n
Compression factor <\/strong><\/td>\n1<\/td>\n130<\/td>\n29<\/td>\n<\/tr>\n
Frame size<\/strong><\/td>\n152105bytes<\/td>\n43 -12970 bytes<\/td>\n166-24865bytes<\/td>\n<\/tr>\n
Recognition<\/strong><\/td>\n<\/td>\nSignificant blocking, particularly when there is activity dancers<\/td>\nVery little variance from original<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

 <\/p>\n

 <\/p>\n

Peak signal to noise ratio (PSNR is frequently used as a measure of the distortion introduced by MPEG compression.\u00a0 For colour video, it is typically acceptable to compute PSNR on the luminance component only.\u00a0 However, PSNR is not based on signal difference as processed by the human visual system. \u00a0Typical PSNR values for lossy compression are between 30 and 50db, the higher the value the better the quality.<\/p>\n

The actual metric that is in use in this part of the assignment is the peak signal-to-reconstructed image measure which is called PSNR.\u00a0 The MOS refers to the Mean Opinion Score to describe the subjective viewing comparison.\u00a0 The following is a brief comment on each video clip regarding the subjective and objective quality measures.<\/p>\n

Coastguard output at quality of 128Kbps<\/strong><\/p>\n

The maximum value of PSNR in this sequence is 41.41 and the minimum is 25.19 with an average of 27.35.\u00a0 On subjective analysis of the clip, significant blocking takes place (MOS = 1) between frames 25 and 80. In this section of the film clip there is a scene in which two boats are in motion and this, in turn, causes \u00a0movement in the water. However, this is not reflected in the PSNR(figure 1)\u00a0 whose value in these frames does not\u00a0 fall below 32, which is well above average for the clip.<\/p>\n

 <\/p>\n

The SSIM (Figure 2) \u00a0values for this clip range from a minimum of .21164 to a maximum of .96969 \u00a0with an average value of .649144 over the 300 frames. The SSIM\u00a0 has its lowest values in the frames 50 to 80 which coincides with the part of the clip where the two boats are moving on the water\u00a0 the SSIM graph output clearly indicates that there is a significant issue with quality at this point in the video with some of the frames in this range labelled as bad frames.<\/p>\n

 <\/p>\n

\u00a0<\/strong><\/p>\n

Figure 4: SSIM for coastguard.avi compressed using MPEG-4 to output rate of 1Mbps<\/strong><\/p>\n

Coastguard output at 1Mbps<\/strong><\/p>\n

The maximum value of PSNR in this sequence is 41.43 and the minimum is 31.5 with an average of 34.43.\u00a0 This higher PSNR average demonstrates the superior quality of the output at these compression settings.\u00a0 Whereas the highly compressed video mentioned above has a greater range between the maximum and the minimum PSNR, the difference is not so great in this, with the minimum well above the average in the previous clip. In this case, the PSNR results correlate with \u00a0the subjective measure with MOS not lower than 4 at any point.<\/p>\n

Figure 6: SSIM for news.avi compressed using MPEG-4 to output rate of 128Kbps<\/strong><\/p>\n

Figure 8: SSIM for news.avi compressed using MPEG-4 to output rate of 1Mbps<\/strong><\/p>\n

News output at 1Mbps<\/strong><\/p>\n

The maximum value of PSNR in this sequence is 45.25 and the minimum is 39.87 with an average of 43.89.\u00a0 This small range of values reflects the high quality result of this compression with little or no interruption to the viewer with a MOS of 5.\u00a0\u00a0 The SSIM values for this clip range from a minimum of .83389 to a maximum of .98662 with an average value of .956755 over the 299 frames.\u00a0 The SSIM values display a dip in three areas as mentioned previously (frames 89,148,239), as SSIM incorporates structural information, luminance and contrast information the changing of a large part of the background from dark to light will register as a significant difference to the SSIM model, whereas subjectively this would not register as a change in the background is perceived to be a normal event.<\/p>\n

Group of Pictures<\/strong><\/p>\n

Each sequence of is subdivided into groups of pictures and the groups of pictures are subdivided into slices or frames, each slice will be of type I, P, B or D.\u00a0 I frames are Intra frames and as are compressed soley in a spatial manner they will have a higher PSNR value and SSIM, P frames are predicted and will have a lower value for PSNR and SSIM and B frames are bi-directionally predicted they will have the lowest value for PSNR and SSIM.\u00a0 Each GOP starts with an I frame , the periodic peaks on the graph represent the I-frames, counting the frames between the peaks gives a GOP of between 10 and 15 frames.\u00a0 The GOP length is constant for the file, and in this instance the GOP length is the approximately the same for every clip.\u00a0 This is contrary to good practice as for the best performance, at low bit rate streaming, the visual quality of the output can be considerably improved by reducing the frame rate and the GOP size.<\/p>\n

The relationship between increased compression levels and reduced PSNR is demonstrated in figure 10, the frames of the output produced by compressing to a rate of 128Kbps in the main has all frames compressed above 95:1, with all frames compressed above 90:1and the majority of teh PSNR values are in the range 25 to 35 with equivalent MOS values of poor to good, whereas the 1Mbps output has a more diverse range of compression ratio form 80:1 to 98:1 and has the PSNR values in the range 32 to 43 with MOS values of good to excellent.<\/p>\n

Figure 12: PSNR vs compression ratio for news.avi compressed using MPEG-4 <\/strong><\/p>\n

The PSNR values for news.avi compressed to 1Mbps are contained within a small range of 42 to 45, where as the compression ratio is spread over 82: to 99.9:1, the type of video lent itself well to compression, with the majority of the contents of the frame staying constant from frame to frame.\u00a0 The compression rate for the majority of frames in the clip compressed at 128Kbps is in the region of 99:1 with only the I frames compressed at a lower rate(90:1-96:1) as evidenced by Figure 9.<\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

\u00a0Video Quality Metrics Digital images are subject to a wide variety of distortions during acquisition, compression storage, transmission and reproduction.\u00a0 Any of these may result in degradation of the image quality.\u00a0 To ensure an acceptable level of quality control, video metrics are applied to judge the quality of the image.\u00a0 Ideally the image quality assessment…<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"open","ping_status":"open","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/www.compucara.ie\/articles\/index.php?rest_route=\/wp\/v2\/pages\/292"}],"collection":[{"href":"https:\/\/www.compucara.ie\/articles\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.compucara.ie\/articles\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.compucara.ie\/articles\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.compucara.ie\/articles\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=292"}],"version-history":[{"count":1,"href":"https:\/\/www.compucara.ie\/articles\/index.php?rest_route=\/wp\/v2\/pages\/292\/revisions"}],"predecessor-version":[{"id":293,"href":"https:\/\/www.compucara.ie\/articles\/index.php?rest_route=\/wp\/v2\/pages\/292\/revisions\/293"}],"wp:attachment":[{"href":"https:\/\/www.compucara.ie\/articles\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=292"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}