|
|
Multiple hypergraph ranking for video concept detection |
Ya-hong Han, Jian Shao*, Fei Wu, Bao-gang Wei |
Department of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China |
|
|
Abstract This paper tackles the problem of video concept detection using the multi-modality fusion method. Motivated by multi-view learning algorithms, multi-modality features of videos can be represented by multiple graphs. And the graph-based semi-supervised learning methods can be extended to multiple graphs to predict the semantic labels for unlabeled video data. However, traditional graphs represent only homogeneous pairwise linking relations, and therefore the high-order correlations inherent in videos, such as high-order visual similarities, are ignored. In this paper we represent heterogeneous features by multiple hypergraphs and then the high-order correlated samples can be associated with hyperedges. Furthermore, the multi-hypergraph ranking (MHR) algorithm is proposed by defining Markov random walk on each hypergraph and then forming the mixture Markov chains so as to perform transductive learning in multiple hypergraphs. In experiments on the TRECVID dataset, a triple-hypergraph consisting of visual, textual features and multiple labeled tags is constructed to predict concept labels for unlabeled video shots by the MHR framework. Experimental results show that our approach is effective.
|
Received: 25 July 2009
Published: 06 July 2010
|
|
Fund: Project supported by the National Natural Science Foundation of China (Nos. 60603096 and 60673088), the National High-Tech Re-
search and Development Program (863) of China (No. 2006AA010107), and the Program for Changjiang Scholars and Innovative Re-search Team in University of China (No. IRT0652) |
Multiple hypergraph ranking for video concept detection
This paper tackles the problem of video concept detection using the multi-modality fusion method. Motivated by
multi-view learning algorithms, multi-modality features of videos can be represented by multiple graphs. And the graph-based
semi-supervised learning methods can be extended to multiple graphs to predict the semantic labels for unlabeled video data.
However, traditional graphs represent only homogeneous pairwise linking relations, and therefore the high-order correlations
inherent in videos, such as high-order visual similarities, are ignored. In this paper we represent heterogeneous features by multiple
hypergraphs and then the high-order correlated samples can be associated with hyperedges. Furthermore, the multi-hypergraph
ranking (MHR) algorithm is proposed by defining Markov random walk on each hypergraph and then forming the mixture Markov
chains so as to perform transductive learning in multiple hypergraphs. In experiments on the TRECVID dataset, a triple-
hypergraph consisting of visual, textual features and multiple labeled tags is constructed to predict concept labels for unlabeled
video shots by the MHR framework. Experimental results show that our approach is effective.
关键词:
Multiple hypergraph ranking,
Video concept detection,
Multi-view learning,
Multiple labeled tags,
Clustering
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|