Single Image Super-Resolution via AdaptiveTransform-Based Nonlocal Self-Similarity Modeling and Learning-Based Gradient Regularization

Single Image Super-Resolution via AdaptiveTransform-Based Nonlocal Self-Similarity Modelingand Learning-Based Gradient Regularization


Single image super-resolution (SISR) is a challenging work, which aims to recover the missing information in an observed low-resolution (LR) image and generate the corresponding high-resolution (HR) version. As SISR problem is severely ill-conditioned, effective prior knowledge of HR images is necessary to well pose the HR estimation. In this paper, an effective SISR method is proposed via the local structure-adaptive transform based nonlocal self-similarity modeling and learning-based gradient regularization (LSNSGR). The LSNSGR exploits both the natural and learned priors of HR images, thus integrating the merits of conventional reconstruction-based and learning based SISR algorithms. More specifically, on the one hand, we characterize nonlocal self-similarity prior (natural prior) in transform domain by using the designed local structure-adaptive transform; on the other hand, the gradient prior (learned prior) is learned via the jointly optimized regression (JOR) model. The former prior is effective in suppressing visual artifacts, while the latter performs well in recovering sharp edges and fine structures. By incorporating the two complementary priors into the MAP based reconstruction framework, we optimize a hybrid L1- and L2-regularized minimization problem to achieve an estimation of the desired HR image. Extensive experimental results suggest that the proposed LSNSGR produces better HR estimations than many state-of-the-art works in terms of both perceptual and quantitative evaluations.


  • In Existing paper, they presented a unified frame based on collaborative representation (CR) for single-image super-resolution (SR), which learns low-resolution (LR) and high-resolution (HR) dictionaries independently in the training stage and adopts a consistent coding scheme (CCS) to guarantee the prediction accuracy of HR coding coefficients during SR reconstruction.
  • The independent LR and HR dictionaries are learned based on CR with l2-norm regularization, which can well describe the corresponding LR and HR patch space, respectively.
  • Furthermore, a mapping function is learned to map LR coding coefficients onto the corresponding HR coding coefficients. Propagation filtering can achieve smoothing over an image while preserving image context like edges or textural regions.


  • In existing, artifacts were introduced and more fine details were degraded.


To suppress artifacts and restore more fine details in the super-resolved image, we propose a SISR method via local structure-adaptive transform-based NLSS modeling and learning-based gradient regularization (LSNSGR). On the one hand, by characterizing the NLSS prior with the proposed local structure-adaptive transform, the local structure and NLSS priors are combined together; on the other hand, aiming at integrating the advantages of reconstruction- and learning based SISR methods, the estimated gradient field is utilized to construct a regularization term. In sum, the main contributions of our work are four-folds:

  • We propose to characterize the NLSS property in transform domain by using the developed local structure adaptive transform. Further, a regularization term, which can be easily applied to solve various image restoration problems, is constructed based on the proposed local structure-adaptive NLSS model.
  • We present to use the jointly optimized regression (JOR) model to estimate the gradient field of the desired HR image, which serves as a gradient regularization term in SR process.
  • With the MAP-based SR framework, an effective SISR algorithm is developed by incorporating the local structure adaptive NLSS model and gradient constraint. In addition, the Split Bregman Iteration (SBI) is imitated to optimize the proposed minimization problem effectively.
  • Benefiting from the complementary properties of nonlocal self-similarity model and gradient constraint, the artifacts in super-resolved image can be removed, and the local structures and sharp edges can be well recovered. Extensive experiments validate the state-of-the-art performance of the developed SISR scheme.


  • By contrast, the proposed method LSNSGR is more effective in both suppressing artifacts as well as recovering sharp edges.
  • Our framework also performs better than all of the compared methods as a whole. It can be observed that the proposed LSNSGR achieves the highest average PSNR, SSIM, and IFC values. Therefore, we can believe that our method produces more reliable HR estimations.
  • Our results are more visually pleasant.




  • System : Pentium Dual Core.
  • Hard Disk : 120 GB.
  • Monitor : 15’’LED
  • Input Devices : Keyboard, Mouse
  • Ram :1GB


  • Operating system : Windows 7.
  • Coding Language : MATLAB
  • Tool : MATLAB R2013A


Honggang Chen, Student Member, IEEE, Xiaohai He, Member, IEEE,Linbo Qing, Member, IEEE, and Qizhi Teng, Member, IEEE, “Single Image Super-Resolution via AdaptiveTransform-Based Nonlocal Self-Similarity Modelingand Learning-Based Gradient Regularization”, IEEETransactions on Multimedia, 2017.


About the Author