We first demonstrate, via representative results, that S₃ is able to accurately perform across-image and within-image sharpness prediction. Figure 1 depicts S₃ maps and indices for a variety of images containing commonplace subject matter.

(1) Image ball, S3 = 0.384 (2) Image airplane, S3 = 0.509 (3) Image petal, S3 = 0.610 (4) Image zebra, S3 = 0.662 (5) Image pelicans, S3 = 0.760 (6) Image branches, S3 = 0.849

Fig. 1. S3 maps and indices for a variety of images containing commonplace subject matter. The ordering of these images was determined based on subjective rankings of sharpness: (1) denotes the image ranked to be least sharp; (6) denotes the image ranked to be the most sharp.

The images in Figure 1 have been ordered according to increasing overall perceived sharpness determined by asking eight naive subjects to rank-order the images in terms of sharpness. The interface of the experiment is shown in Figure 2.

Fig. 2. The interface of the rank-ordering experiment.

In this experiment, the subjects were asked to rank the six images in terms of overall sharpness. The screen used was a LaCie 324 24-inch LCD monitor (1920x1200 at 60 Hz). The display yielded minimum and maximum luminances of 0.80 and 259 cd/m², respectively, with a luminance gamma of γ = 2.2. Stimuli were viewed binocularly through natural pupils in a darkened room at a distance of approximately 60 cm. The six images yielded 15 pairs, and each subject took approximately five minutes to finish comparing all the pairs. The order of images shown in the Figure 1 was obtained by averaging the ranks from all subjects.

In terms of across-image prediction, comparing the S₃ indices to the rank ordering, S₃ is generally able to perform across-image sharpness prediction accurately. In terms of within-image sharpness prediction, all of the S₃ maps quite accurately capture the sharp regions in each image.

1. In the image ball, ball is blurred, which is correctly shown as dark in the map. The map is also able to point out that, in the background, the right vertical line is sharper than the left line.

2. The sharpest regions in the image airplane are the part of the mountain at the bottom of the image and the plane’s body. This fact is correctly captured in the corresponding S₃ map where these regions appear brightest. On the other hand, the snow region, which is apparently blurry, correctly receives the smallest sharpness value.

3. In the image petal, the flower’s stamens are the sharpest regions, which is accurately noted in its corresponding S₃ map. The second sharpest region in this image occurs along the borders of the flower’s petals, particularly, the lower petals. This fact is also captured in the map: The borders of the upper petals receive a low sharpness value, the lower petals receive a higher sharpness value, and the stamens receive the greatest sharpness value.

4. The perceived sharpness of the image zebra is due to the high contrast stripes of the two zebras. This fact is accurately shown in the sharpness map. Although most of the background is blurred, which correctly appears dark in the map, the map is also able to capture the sharp bushes between the two zebras.

5. In the image pelicans, the sharpest region includes the pelicans and bushes in the middle of the image. The regions in the background, which include the building and the sky, are very blurred, and the water is increasingly blurred toward the bottom of the image. Accordingly, the S₃ map indicates the greatest sharpness toward the center of the image (corresponding to the pelicans and bushes) and decreasing sharpness away from the center.

6. The image branches includes two regions: the very sharp branches and very smooth sky. Again, as can be seen from the S₃ map, the branches receive a very high sharpness value while the sky receives a very low sharpness value.

Go to top

Fig. 3. The interface of the experiment to generate ground-truth sharpness map.

The interface of the experiment to generate ground-truth sharpness map (as described in Section IV-D in the manuscript) is shown the Figure 3. Two versions of the same image were displayed against a mid-gray background. The left image was divided into blocks of size 16x16 (shown by the grid) and the same image (without the grid) was displayed on the right-hand side for reference. Subjects rated the sharpness value for each block by using an integer scale from 1 to 3, where 1 denoted that the block was very sharp, 3 denoted that the block was not sharp, and 2 was anything in-between. The testing interface contained options to assign a sharpness level to multiple blocks and undo an assignment, which facilitated the testing.

Image dragon Input image Ground truth sharpness map Inter-subject std map CPBD sharpness map ST sharpness map MMZ sharpness map MDWE sharpness map S3 sharpness map BLINDS-II sharpness map JNB sharpness map           Image flower Input image Ground truth sharpness map Inter-subject std map CPBD sharpness map ST sharpness map MMZ sharpness map MDWE sharpness map S3 sharpness map BLINDS-II sharpness map JNB sharpness map   Image monkey Input image Ground truth sharpness map Inter-subject std map CPBD sharpness map ST sharpness map MMZ sharpness map MDWE sharpness map S3 sharpness map BLINDS-II sharpness map JNB sharpness map           Image orchid Input image Ground truth sharpness map Inter-subject std map CPBD sharpness map ST sharpness map MMZ sharpness map MDWE sharpness map S3 sharpness map BLINDS-II sharpness map JNB sharpness map   Image peak Input image Ground truth sharpness map Inter-subject std map CPBD sharpness map ST sharpness map MMZ sharpness map MDWE sharpness map S3 sharpness map BLINDS-II sharpness map JNB sharpness map         Image squirrel Input image Ground truth sharpness map Inter-subject std map CPBD sharpness map ST sharpness map MMZ sharpness map MDWE sharpness map S3 sharpness map BLINDS-II sharpness map JNB sharpness map

Fig. 4. Six original images, their corresponding average subjective sharpness maps, the inter-subject standard deviation maps, and the maps generated by different algorithms. (Black blocks in the MDWE maps are blocks on which MDWE failed to run).

Performances: In addition to Figure 12 in the manuscript, Figure 4 shows maps of all algorithms in comparison for all six images. We also include in this figure maps from the BLIINDS-II algorithm, which has been run on each 32x32 block with 24 pixels of overlap. In general, the S₃ algorithm creates the best estimation of the ground truth maps.

Table I below shows the performance of all seven algorithms using three criteria: CC, SROCC and KLD (Kullback-Leibler Divergence). Again, this table shows that S₃ holds the best performance in general. Note that this is not quite a fair comparison since the other algorithms were not designed to generate a sharpness map. There may certainly be more optimal techniques of applying these estimators locally rather than via the block-based fashion employed here.

Go to top

We discuss in this section the performance of S₃ on no-reference image quality assessment for JPEG2000-compressed images. One might argue that a sharpness/blurriness estimator can also be used for no-reference quality assessment of JPEG2000 images since JPEG2000 compression can induce blurring. To analyze the performance of S₃ on this task, we used the JPEG2000 image subsets from LIVE, CSIQ, and TID2008, which contain 169, 150, and 96 JPEG2000 distorted images, respectively. For comparison, the same sets of images were analyzed by using the following sharpness algorithms: (1) JNB; (2) CPBD; (3) MMZ; (4) ST; (5) MDWE; and (6) BLIINDS-II. The results of this analysis are shown in Table II.

In comparison to JNB, ST, MMZ, and MDWE, except for images from the LIVE database, the performance of S₃ is quite competitive. However, the CPBD and BLIINDS-II algorithm clearly outperform others. We believe this limitation of S₃ is due to the fact that JPEG2000 compression does not generate blurring uniformly across an image. A highly compressed JPEG2000 image might still contain relatively sharp regions which yield a high S₃ index because of our computation of the index in Equation (11) in the manuscript. On the other hand, the artifacts from JPEG2000 compression might ruin edges in the image, thus other sharpness/blur measures which rely on the appearance of edges in the image might give more veridical results for those images. Figure 5 demonstrates this argument. Even though most regions of the image have been blurred by JPEG2000 compression, some regions still appear quite sharp (such as the buildings in the middle of the image). Notice that the corresponding S₃ map both correctly detects the blurred regions and is able to capture the sharp regions. The S₃ index of this image, computed as the average of the maximum 1% pixel values in the S₃ map is therefore fairly high.

S3 index is 0.7607

Fig. 5. A highly compressed JPEG2000 image might still contain relatively sharp regions (e.g., the buildings in the middle of the image). Notice that the corresponding S3 map both correctly detects the blurred regions and is able to capture the sharp regions.

Go to top

In Eq. (6) in the manuscript, the two parameters τ₁, τ₂ control the shape of the sigmoid transducer function (see Fig. 5 in the manuscript). Here we analyze the performance of S₃ with different selections of τ₁ and τ₂. The corresponding sigmoid shapes are shown below in Figure 6.

τ1 = -5; τ2 = 1 τ1 = -5; τ2 = 2 τ1 = -5; τ2 = 3 τ1 = -4; τ2 = 2 τ1 = -4; τ2 = 3 τ1 = -4; τ2 = 4 τ1 = -3.5; τ2 = 1.5 τ1 = -3.5; τ2 = 2.5 τ1 = -2.5; τ2 = 2.5 τ1 = -3; τ2 = 2 τ1 = -3; τ2 = 3 τ1 = -2; τ2 = 3

Fig. 6. The sigmoid transducer function with different selection of τ1 and τ2.

Table III shows the performance of S3 on blurred images of TID2008, LIVE, and CSIQ databases with different selections of the pair (τ₁, τ₂) [see Equation (6) in the paper]. The best performance for each criterion on each database is highlighted. The selection of [τ₁, τ₂] = [-3, 2] was not chosen based on this analysis; rather it was determined empirically by fitting a sigmoid to subjective ratings of sharpness (using the authors as subjects) for the noise images shown in Figure 5 in the paper. Indeed, this pair gives the best CC only on the TID2008 and the best SROCC only on the LIVE database.

With the chosen constants τ₁ and τ₂, [-3 2], different values of the parameter η (see Equation (10) in the paper) were also tested to see the influence of this parameter. Table IV shows the performance of S₃ when η = 1/4, 1/3, 1/2, 2/3, and 3/4. When η = 1/2, S₃ gives the best CCs on TID2008 and LIVE, and SROCC on LIVE. Note that η = 1/2 was not chosen based on this analysis; rather, because S1 and S2 were designed to compensate for each other’s failures, the selection of η = 1/2 is a reasonable choice.

Go to top

 

(a) (b) (c)

Fig. 7. The three images (a), (b), (c) all have the same spectral slope. However, image (b) and (c) appear much less sharp than image (a) due to the difference in contrast.

While the slope of the magnitude spectrum can be a potential measure of sharpness, it does not take into account contrast, which is known to affect perceived sharpness. Two images can have the same spectral slope but appear to be of different sharpness due to a difference in contrast. Figure 7 demonstrates this assertion. Image (b) was generated from the image (a) by taking 40% of the value of every pixel in (a). This image (b) is then added a constant to make image (c) which has the same mean luminance as image (a). The magnitude spectrums of image (b) and (c) (ignoring the DC component) are therefore the same, and proportional to that of image (a). Thus the three images have the same slope factor. However, the image in (a) appears much sharper than (b) and (c) due to the low contrast of the latter two.

In this section, we computationally show the validity of this demonstration using Weber contrast, Michelson luminance contrast, and RMS luminance contrast.

Weber contrast is defined as: C = (L_max – L_min)/ L_min

Michelson contrast (Peak-to-Peak contrast) is defined as: C = (L_max – L_min)/ (L_max + L_min)

RMS contrast is defined as: C = σ_L/μ_L in which the conversion to luminance L from 8-bit pixel value image I is given by: L = (a + bI)^γ

L_max, L_min, σ_L, and μ_L are the maximum, minimum, standard deviation, and the mean of L, respectively.

Assuming the Adobe RGB color space, a = 0.7656; b = 0.0364 and γ = 2.2, the luminance contrast of each image in the demonstration above is given as:
Weber contrast: (a) 287.2, (b) 47.71, (c) 5.877
Michelson contrast: (a) 0.993, (b) 0.959, (c) 0.746
RMS contrast: (a) 1.453, (b) 1.125, (c) 0.534

These results support the validity of the demonstration. However, notice from these results that image (c), which is currently used in Figure 2 in the manuscript, better demonstrates the limitation of the slope of magnitude spectrum than image (b).

Go to top

Fig. 8. Relationship between the standard deviation of the Gaussian blurring filter’s impulse response and the sharpness/blurriness index of different algorithms.

Figure 8 repeats Figure 10 in the manuscript and adds the curve from BLIINDS-II. As can be seen from this Figure, BLIINDS-II shows several failure cases in this test. However, note that this behavior is expected given that BLIINDS-II was designed for quality assessment rather than sharpness estimation.

σ = 0.4 σ = 0.8 σ = 1.2 σ = 1.6 σ = 2 σ = 2.4 σ = 2.8 σ = 0.4 σ = 0.8 σ = 1.2 σ = 1.6 σ = 2 σ = 2.4 σ = 2.8

Fig. 9. Examples of images that have been blurred by a Gaussian filter of size 15 x 15 pixels and standard deviation of σ = {0.4, 0.8, 1.2, 1.6, 2.0, 2.4, 2.8}.

As discussed in the last paragraph of Section IV-C, notice from the curves of S₃ that for σ ≥ 1.6, the S₃ indices are close to zero.  This fact agrees with a visual examination of these images; visually, the overall sharpnesses of images blurred with σ ≥ 1.6 are all very low and significantly lower than images blurred with σ ≤ 1.2. Examples of these images are shown in Figure 9 (click on the thumbnails for full-sized versions).

Figure 10 depicts graphs of the subjective ratings of DMOS (on the vertical axis; MOS for TID) plotted against S₃ transformed outputs (on the horizontal axis). These graphs demonstrate that S₃ performs quite well on very blurred images (high DMOS values for CSIQ and LIVE database, and low MOS values for TID database).

Fig. 10. Subjective ratings of perceived distortion plotted agains S3 transformed outputs for CSIQ, LIVE and TID2008 database. The subjective score is DMOS for CSIQ and LIVE and MOS for TID

Go to top

Sharpness maps database
S₃ algorithm Matlab code

Go to top