SSDH: Semi-supervised Deep Hashing for Large Scale Image Retrieval

SSDH: Semi-supervised Deep Hashing for Large Scale Image Retrieval

ABSTRACT:

Hashing methods have been widely used for efficient similarity retrieval on large scale image database. Traditional hashing methods learn hash functions to generate binary codes from hand-crafted features, which achieve limited accuracy since the hand-crafted features cannot optimally represent the image content and preserve the semantic similarity. Recently, several deep hashing methods have shown better performance because the deep architectures generate more discriminative feature representations. However, these deep hashing methods are mainly designed for supervised scenarios, which only exploit the semantic similarity information, but ignore the underlying data structures. In this paper, we propose the semi-supervised deep hashing (SSDH) approach, to perform more effective hash function learning by simultaneously preserving semantic similarity and underlying data structures. The main contributions are as follows: (1) We propose a semi-supervised loss to jointly minimize the empirical error on labeled data, as well as the embedding error on both labeled and unlabeled data, which can preserve the semantic similarity and capture the meaningful neighbors on the underlying data structures for effective hashing. (2) A semi-supervised deep hashing network is designed to extensively exploit both labeled and unlabeled data, in which we propose an online graph construction method to benefit from the evolving deep features during training to better capture semantic neighbors. To the best of our knowledge, the proposed deep network is the first deep hashing method that can perform hash code learning and feature learning simultaneously in a semi-supervised fashion. Experimental results on 5 widely-used datasets show that our proposed approach outperforms the state-of-the-art hashing methods.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS: 

  • System : Pentium Dual Core.
  • Hard Disk : 120 GB.
  • Monitor : 15’’ LED
  • Input Devices : Keyboard, Mouse
  • Ram :1 GB

SOFTWARE REQUIREMENTS: 

  • Operating system : Windows 7.
  • Coding Language :
  • Tool : MATLAB R2013A /2018

REFERENCE:

Jian Zhang and Yuxin Peng, “SSDH: Semi-supervised Deep Hashing for Large Scale Image Retrieval”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2019.

Single Image Depth Estimation with Normal Guided Scale Invariant Deep Convolutional Fields

Single Image Depth Estimation with Normal Guided Scale Invariant Deep Convolutional Fields

ABSTRACT:

Estimating scene depth from a single image can be widely applied to understand 3D environments due to the easy access of the images captured by consumer-level cameras. Previous works exploit Conditional Random Fields (CRF) to estimate image depth, where neighboring pixels (superpixels) with similar appearance are constrained to share the same depth. However, the depth may vary significantly in the slanted surface, thus leading to severe estimation errors. In order to eliminate those errors, we propose a superpixel based normal guided scale invariant deep convolutional field by encouraging the neighboring superpixels with similar appearance to lie on the same 3D plane of the scene. In doing so, a depth-normal multitask CNN is introduced to produce the superpixel-wise depth and surface normal predictions simultaneously. To correct the errors of the roughly estimated superpiexl-wise depth, we develop a normal guided scale invariant CRF (NGSI-CRF). NGSI-CRF consists of a scale invariant unary potential which is able to measure the relative depth between superpixels as well as the absolute depth of superpixels, and a normal guided pairwise potential which constrains spatial relationships between superpixels in accordance with the 3D layout of the scene. In other words, the normal guided pairwise potential is designed to smooth the depth prediction without deteriorating the 3D structure of the depth prediction. The superpixel-wise depth maps estimated by NGSI-CRF will be fed into a pixel-wise refinement module to produce a smooth fine-grained depth prediction. Furthermore, we derive a closed form solution for the maximum a posteriori (MAP) inference of NGSI-CRF. Thus, our proposed network can be efficiently trained in an end-to-end manner. We conduct our experiments on various datasets, such as NYU-D2, KITTI and Make3D. As demonstrated in the experimental results, our method achieves superior performance in both indoor and outdoor scenes.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS: 

  • System : Pentium Dual Core.
  • Hard Disk : 120 GB.
  • Monitor : 15’’ LED
  • Input Devices : Keyboard, Mouse
  • Ram :1 GB

SOFTWARE REQUIREMENTS: 

  • Operating system : Windows 7.
  • Coding Language :
  • Tool : MATLAB R2013A /2018

REFERENCE:

Han Yan, Xin Yu, Yu Zhang, Shunli Zhang, Xiaolin Zhao, and Li Zhang, “Single Image Depth Estimation with Normal Guided Scale Invariant Deep Convolutional Fields”, IEEE 2019.

Scale-Free Single Image Deraining Via Visibility-Enhanced Recurrent Wavelet Learning

Scale-Free Single Image Deraining Via Visibility-Enhanced Recurrent Wavelet Learning

ABSTRACT:

In this paper, we address a rain removal problem from a single image, even in the presence of large rain streaks and rain streak accumulation (where individual streaks cannot be seen, and thus visually similar to mist or fog). For rain streak removal, the mismatch problem between different streak sizes in training and testing phases leads to a poor performance, especially when there are large streaks. To mitigate this problem, we embed a hierarchical representation of wavelet transform into a recurrent rain removal process: 1) rain removal on the low-frequency component; 2) recurrent detail recovery on high frequency components under the guidance of the recovered low frequency component. Benefiting from the recurrent multi-scale modeling of wavelet transform-like design, the proposed network trained on streaks with one size can adapt to those with larger sizes, which significantly favors real rain streak removal. The dilated residual dense network is used as the basic model of the recurrent recovery process. The network includes multiple paths with different receptive fields, thus can make full use of multi-scale redundancy and utilize context information in large regions. Furthermore, to handle heavy rain cases where rain streak accumulation is presented, we construct a detail appearing rain accumulation removal to not only improve the visibility but also enhance the details in dark regions. The evaluation on both synthetic and real images, particularly on those containing large rain streaks and heavy accumulation, shows the effectiveness of our novel models, which significantly outperforms the state-of-the-art methods.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS: 

  • System : Pentium Dual Core.
  • Hard Disk : 120 GB.
  • Monitor : 15’’ LED
  • Input Devices : Keyboard, Mouse
  • Ram :1 GB

SOFTWARE REQUIREMENTS: 

  • Operating system : Windows 7.
  • Coding Language :
  • Tool : MATLAB R2013A /2018

REFERENCE:

Wenhan Yang, Member, IEEE, Jiaying Liu, Senior Member, IEEE, Shuai Yang, Zongming Guo, Member, IEEE, “Scale-Free Single Image Deraining Via Visibility-Enhanced Recurrent Wavelet Learning”, IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019.

Remote Sensing Image Sharpening by Integrating Multispectral Image Super-Resolution and Convolutional Sparse Representation Fusion

Remote Sensing Image Sharpening by Integrating Multispectral Image Super-Resolution and Convolutional Sparse Representation Fusion

ABSTRACT:

In remote sensing, it is quite necessary to fuse spectral information of low-resolution multispectral (LRMS) images and spatial information of panchromatic (PAN) images for obtaining high-resolution multispectral (HRMS) images. In this paper, an effective fusion method integrating multispectral (MS) image super-resolution and convolutional sparse representation (CSR) fusion is proposed to make full use of the spatial information of remote sensing images. First, for enhancing the spatial information of LRMS images with suitable sizes, a fast iterative image super-resolution algorithm based on the learned iterative shrinkage and thresholding algorithm (LISTA) is exploited in the first stage. It employs a feed-forward neural network to simplify the solution of sparse coefficients in the process of super-resolution. In the fusion stage, we propose a CSR-based image fusion framework, in which each MS super-resolution image and PAN image is decomposed into a basic layer and a detail layer, then we fuse the basic layers and the detail layers of the images, respectively. This hierarchical fusion strategy guarantees great performance in detail preservation. The experimental results on QuickBird, WorldView-2, and Landsat ETMC datasets demonstrate that the proposed method outperforms other methods in terms of both objective evaluation and visual effect.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS: 

  • System : Pentium Dual Core.
  • Hard Disk : 120 GB.
  • Monitor : 15’’ LED
  • Input Devices : Keyboard, Mouse
  • Ram :1 GB

SOFTWARE REQUIREMENTS: 

  • Operating system : Windows 7.
  • Coding Language :
  • Tool : MATLAB R2013A /2018

REFERENCE:

HONGLIN WU , SHUZHEN ZHAO, JIANMING ZHANG , AND CHAOQUAN LU, “Remote Sensing Image Sharpening by Integrating Multispectral Image Super-Resolution and Convolutional Sparse Representation Fusion”, IEEE Access, 2019.

Recolored Image Detection via a Deep Discriminative Model

Recolored Image Detection via a Deep Discriminative Model

ABSTRACT:

Image recoloring is a technique that can transfer image color or theme and result in an imperceptible change in human eyes. Although image recoloring is one of the most important image manipulation techniques, there is no special method designed for detecting this kind of forgery. In this paper, we propose a trainable end-to-end system for distinguishing recolored images from natural images. The proposed network takes the original image and two derived inputs based on illumination consistency and inter-channel correlation of the original input into consideration and outputs the probability that it is recolored. Our algorithm adopts a CNN-based deep architecture, which consists of three feature extraction blocks and a feature fusion module. To train the deep neural network, we synthesize a dataset comprised of recolored images and corresponding ground truth using different recoloring methods. Extensive experimental results on the recolored images generated by various methods show that our proposed network is well generalized and much robust.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS: 

  • System : Pentium Dual Core.
  • Hard Disk : 120 GB.
  • Monitor : 15’’ LED
  • Input Devices : Keyboard, Mouse
  • Ram :1 GB

SOFTWARE REQUIREMENTS: 

  • Operating system : Windows 7.
  • Coding Language :
  • Tool : MATLAB R2013A /2018

REFERENCE:

Yanyang Yan, Student Member, IEEE, Wenqi Ren, Member, IEEE, and Xiaochun Cao_, Senior Member, IEEE, “Recolored Image Detection via a Deep Discriminative Model”, IEEE TRANSACTIONS ON INFORMATION FORENSICS AND SECURITY, 2019.

Rapid Color Grading for Fruit Quality Evaluation Using Direct Color Mapping

Rapid Color Grading for Fruit Quality Evaluation Using Direct Color Mapping

ABSTRACT:

Color grading is a crucial step in the processing of fruits and vegetables that directly affects profitability, because the quality of agricultural products is often associated with their color. Most existing automatic color grading systems determine color quality either by directly comparing product color against a predefined and fixed set of reference colors or by using a set of color separating parameters, often in three-dimensional color spaces. Using these methods, it is not convenient for the user to adjust color preferences or grading parameters. In this paper, we present an effective and user-friendly color mapping concept for automated color grading that is well suited for commercial production. User friendliness is often viewed by the industry as a very important factor to the acceptance and success of automation equipment. This color mapping method uses preselected colors of interest specific to a given application to calculate a unique set of coefficients for color space conversion. The three-dimensional RGB color space is converted into a small set of color indices unique to the application. In contrast with more complex color grading techniques, the proposed method makes it easy for a human operator to specify and adjust color-preference settings Tomato and date maturity evaluation and date surface defect detection are used to demonstrate the performance of this novel color mapping concept.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS: 

  • System : Pentium Dual Core.
  • Hard Disk : 120 GB.
  • Monitor : 15’’ LED
  • Input Devices : Keyboard, Mouse
  • Ram :1 GB

SOFTWARE REQUIREMENTS: 

  • Operating system : Windows 7.
  • Coding Language :
  • Tool : MATLAB R2013A /2018

REFERENCE:

Dah-Jye Lee, Senior Member, IEEE, James K. Archibald, Senior Member, IEEE, and Guangming Xiong, “Rapid Color Grading for Fruit Quality Evaluation Using Direct Color Mapping”, IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, 2019.

RAN: Resolution-Aware Network for Image Super-resolution

RAN: Resolution-Aware Network for Image Super-resolution

ABSTRACT:

In existing deep network based image super-resolution (SR) methods, each network is only trained for a fixed upscaling factor and can hardly generalize to unseen factors at test time, which is non-scalable in real applications. To mitigate this issue, this paper proposes a resolution-aware network (RAN) for simultaneous SR of multiple factors. The key insight is that SR of multiple factors are essentially different but also share common operations. To attain stronger generalization across factors, we design an upsampling network (U-Net) consisting of several sub-modules, where each sub-module implements an intermediate step of the overall image SR and can be shared by SR of different factors. A decision network (D-Net) is further adopted to identify the quality of the input low-resolution (LR) image and adaptively select suitable sub-modules to perform SR. U-Net and D-Net together constitute the proposed RAN model, and are jointly trained using a new hierarchical loss function on SR tasks of multiple factors. Experimental evaluations demonstrate that the proposed RAN compares favorably against state-of-the-art methods and its performance can well generalize across different upscaling factors.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS: 

  • System : Pentium Dual Core.
  • Hard Disk : 120 GB.
  • Monitor : 15’’ LED
  • Input Devices : Keyboard, Mouse
  • Ram :1 GB

SOFTWARE REQUIREMENTS: 

  • Operating system : Windows 7.
  • Coding Language :
  • Tool : MATLAB R2013A /2018

REFERENCE:

Yifan Wang, Lijun Wang, Hongyu Wang, Member, IEEE, Peihua Li, Member, IEEE, “RAN: Resolution-Aware Network for Image Super-resolution”, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2019.

Patch-Sparsity-Based Image Inpainting through Facet Deduced Directional Derivative

Patch-Sparsity-Based Image Inpainting through Facet Deduced Directional Derivative

ABSTRACT:

This paper presents a patch-sparsity-based image inpainting algorithm through facet deduced directional derivative. The algorithm could ensure the continuity of boundaries of the inpainted region and achieve a better performance on restoring missing structure of an image. In this paper, two improvements are proposed. First, facet model is introduced to get direction features of the image, which could efficiently reduce the effect of noises. The first-order directional derivatives, along with pixel values, are used to measure the difference between patches. Consequently, a more reliable and accurate matching result is promised. At the same time, the local patch consistency constraint of sparse representation of the target patch is also rewritten in the form of the first-order directional derivative. Therefore, a more precise sparse linear combination could be obtained under constraints for both color and derivative information. Second, the value of patch confidence in the traditional exemplar-based inpainting algorithms drops sharply in the late stage so that the data term or structure sparsity has little influence on priority function. Aiming at this problem, the algorithm makes a modification to the calculating of priority. Thus, the filling order decided by priority function appears more reasonable as result of a better balance between the values of modified confidence and structure sparsity. Experiments on different types of damages to images show the superiority of the algorithm.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS: 

  • System : Pentium Dual Core.
  • Hard Disk : 120 GB.
  • Monitor : 15’’ LED
  • Input Devices : Keyboard, Mouse
  • Ram :1 GB

SOFTWARE REQUIREMENTS: 

  • Operating system : Windows 7.
  • Coding Language :
  • Tool : MATLAB R2013A /2018

REFERENCE:

Darui Jin, Xiangzhi Bai, “Patch-Sparsity-Based Image Inpainting through Facet Deduced Directional Derivative”, IEEE 2019.

Multi-Domain & Multi-Task Learning for Human Action Recognition

Multi-Domain & Multi-Task Learning for Human Action Recognition

ABSTRACT:

Domain-invariant (view-invariant & modality invariant) feature representation is essential for human action recognition. Moreover, given a discriminative visual representation, it is critical to discover the latent correlations among multiple actions in order to facilitate action modeling. To address these problems, we propose a multi-domain & multi-task learning (MDMTL) method to (1) extract domain-invariant information for multi-view and multi-modal action representation and (2) explore the relatedness among multiple action categories. Specifically, we present a sparse transfer learning-based method to co-embed multi-domain (multi-view & multi-modality) data into a single common space for discriminative feature learning. Additionally, visual feature learning is incorporated into the multitask learning framework, with the Frobenius-norm regularization term and the sparse constraint term, for joint task modeling and task relatedness-induced feature learning. To the best of our knowledge, MDMTL is the first supervised framework to jointly realize domain-invariant feature learning and task modeling for multi-domain action recognition. Experiments conducted on the INRIA Xmas Motion Acquisition Sequences (IXMAS) dataset, the MSR Daily Activity 3D (DailyActivity3D) dataset, and the Multi-modal & Multi-view & Interactive (M2I) dataset, which is the most recent and largest multi-view and multi-model action recognition dataset, demonstrate the superiority of MDMTL over the state-of-the-art approaches.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS: 

  • System : Pentium Dual Core.
  • Hard Disk : 120 GB.
  • Monitor : 15’’ LED
  • Input Devices : Keyboard, Mouse
  • Ram :1 GB

SOFTWARE REQUIREMENTS: 

  • Operating system : Windows 7.
  • Coding Language :
  • Tool : MATLAB R2013A /2018

REFERENCE:

An-An Liu*, Ning Xu*, Wei-Zhi Nie*, Yu-Ting Su, and Yong-Dong Zhang, “Multi-Domain & Multi-Task Learning for Human Action Recognition”, IEEE TRANSACTIONS ON IMAGE PROCESSING, 2019.

Image De-raining Using a Conditional Generative Adversarial Network

Image De-raining Using a Conditional Generative Adversarial Network

ABSTRACT:

Severe weather conditions such as rain and snow adversely affect the visual quality of images captured under such conditions thus rendering them useless for further usage and sharing. In addition, such degraded images drastically affect performance of vision systems. Hence, it is important to address the problem of single image de-raining. However, the inherent ill-posed nature of the problem presents several challenges. We attempt to leverage powerful generative modeling capabilities of the recently introduced Conditional Generative Adversarial Networks (CGAN) by enforcing an additional constraint that the de-rained image must be indistinguishable from its corresponding ground truth clean image. The adversarial loss from GAN provides additional regularization and helps to achieve superior results. In addition to presenting a new approach to de-rain images, we introduce a new refined loss function and architectural novelties in the generator-discriminator pair for achieving improved results. The loss function is aimed at reducing artifacts introduced by GANs and ensure better visual quality. The generator sub-network is constructed using the recently introduced densely connected networks, whereas the discriminator is designed to leverage global and local information to decide if an image is real/fake. Based on this, we propose a novel single image de-raining method called Image De-raining Conditional Generative Adversarial Network (ID-CGAN), which considers quantitative, visual and also discriminative performance into the objective function. Experiments evaluated on synthetic and real images show that the proposed method outperforms many recent state-of-the-art single image de-raining methods in terms of quantitative and visual performance. Furthermore, experimental results evaluated on object detection datasets using Faster-RCNN also demonstrate the effectiveness of proposed method in improving the detection performance on images degraded by rain.

SYSTEM REQUIREMENTS:

HARDWARE REQUIREMENTS: 

  • System : Pentium Dual Core.
  • Hard Disk : 120 GB.
  • Monitor : 15’’ LED
  • Input Devices : Keyboard, Mouse
  • Ram :1 GB

SOFTWARE REQUIREMENTS: 

  • Operating system : Windows 7.
  • Coding Language :
  • Tool : MATLAB R2013A /2018

REFERENCE:

He Zhang, Member, IEEE, Vishwanath Sindagi, Student Member, IEEE Vishal M. Patel, Senior Member, IEEE, “Image De-raining Using a Conditional Generative Adversarial Network”, IEEE 2019.