Tri-Clustered Tensor Completion for Social-Aware Image Tag Refinement
Social image tag refinement, which aims to improve tag quality by automatically completing the missing tags and rectifying the noise-corrupted ones, is an essential component for social image search. Conventional approaches mainly focus on exploring the visual and tag information, without considering the user information, which often reveals important hints on the (in)correct tags of social images. Towards this end, we propose a novel tri-clustered tensor completion framework to collaboratively explore these three kinds of information to improve the performance of social image tag refinement. Specifically, the inter-relations among users, images and tags are modeled by a tensor, and the intra-relations between users, images and tags are explored by three regularizations respectively. To address the challenges of the super-sparse and large-scale tensor factorization that demands expensive computing and memory cost, we propose a novel tri-clustering method to divide the tensor into some sub-tensors by simultaneously clustering users, images and tags into a bunch of tri-clusters. And then we investigate two strategies to complete these sub-tensors by considering (in) dependence between the sub-tensors. Experimental results on a real-world social image database demonstrate the superiority of the proposed method compared with the state-of-the-art methods.
PROJECT OUTPUT VIDEO: (Click the below link to see the project output video):
- The prior works related to image tag refinement mainly focus on exploring semantic correlation among tags.
- Jin et al. identified and filtered out the weakly irrelevant annotated tags by exploring tag semantic correlation on WordNet.
- Xu et al. proposed a tag refinement scheme based on tag similarity and relevance by using LDA to mine latent topics.
- Recently, matrix completion based on low-rank approximation has been explored, which refers to a process of inferring missing entries from a small part of the observed entries in the original matrix between the dyad data (such as word-document in text mining, user-item in recommendation system, and image feature in image processing).
- Inspired by matrix completion, several approaches have been proposed, to leverage a small number of observed noisy tags to simultaneously recover the missing tags, remove the noisy tags, and even re-rank the complete tag list. These methods have achieved the impressive performance in tag refinement.
DISADVANTAGES OF EXISTING SYSTEM:
- All the existing aforementioned methods only explore the visual and tag information, without considering the user information (e.g., user interests and backgrounds) that usually reveals important hints on the (in)correct tags of social images. Therefore, these above methods lacking the consideration of user information cannot achieve satisfied performance when the visual content and label taxonomy (e.g. WordNet taxonomy) are inconsistent.
- It requires several selected ”negative” tags before ranking, which will bring in some incorrect correlations.
- There are several problems in the tensor completion for real-world applications. First, the dimension of the constructed tensor is usually extremely large. The process of tensor completion generates large temporal matrices and tensors, which requires expensive computing and memory cost.
- Existing works mainly explore parallel solutions to achieve low complexity and reduce memory cost.
- Second, the associated 3rd-order tensor is usually very sparse, because the number of observed elements only accounts for a small ratio compared to the size of the tensor.
- We explore the user information to assist social image tag refinement, especially for those images with context information, e.g., geo-related tags, event tags, etc.
- To address the above issues, we propose a novel tri-clustered tensor completion (TTC) framework for social image tag refinement.
- First, we utilize the clustering method to divide the original tensor into several sub-tensors to reduce the computing and memory cost. As to the clustering problem, existing approaches use the associated matrix to model the relationships between two types of data, and then cluster the rows and columns of this matrix simultaneously into co-clusters, which is known as the co-clustering.
- Motivated by this, we propose an efficient tri-clustering method to identify the block structures in the rows, columns and tubes. Specifically, the proposed tri-clustering divides the image-tag-user associated tensor into several subtensors based on the explicit associations and latent structure of the tensor. Second, to handle the supersparsity problem of tensor, we select the denser subtensors, and then complete the selected sub-tensors.
ADVANTAGES OF PROPOSED SYSTEM:
- The results of social image tag refinement on a real-world social image database demonstrate the superiority of the proposed method compared with the state-of-the-art methods.
- A novel tri-clustered tensor completion (TTC) framework for social image tag refinement is proposed by solving the low-rank approximation problem of the image-tag-user associated tensor.
- The tri-clustering method is proposed to divide the tensor into several sub-tensors, in order to overcome the challenges of large-scale tensor factorization.
- The sub-tensor completion method is proposed to complete the denser sub-tensors, in order to effectively solve the super-sparse tensor completion problem.
- Two variants of TTC are proposed respectively, by considering the two assumptions whether or not the sub-tensors are independent of each other.
- System : Pentium Dual Core.
- Hard Disk : 120 GB.
- Monitor : 15’’ LED
- Input Devices : Keyboard, Mouse
- Ram : 1 GB
- Operating system : Windows 7.
- Coding Language : JAVA/J2EE
- Tool : ECLIPSE
- Database : MYSQL
Jinhui Tang, Xiangbo Shu, Guo-Jun Qi, Zechao Li, Meng Wang, Shuicheng Yan, and Ramesh Jain, Life Fellow, IEEE, “Tri-Clustered Tensor Completion for Social-Aware Image Tag Refinement”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017.