|
|
Short text expansion and classification based on pseudo-relevance feedback |
WANG Meng, LIN Lan-fen, WANG Feng |
College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China |
|
|
Abstract A novel classification method based on pseudo-relevance feedback (PFR) was proposed in order to solve the sparseness problems in short text classification. The short texts were expanded using the web pages which are similar to them in semantic level. The feature vector generation algorithm was modified to extract both the local features and the global features. The method can alleviate the sparseness problem of the final feature matrix, which is common in short text classification because of the limited length of the texts. The experimental results on an open dataset show that the method can significantly improve the short text classification effect compared with state-of-the-art methods.
|
Published: 01 October 2014
|
|
基于伪相关反馈的短文本扩展与分类
针对短文本分类问题,提出基于伪相关反馈(PFR)的短文本扩展与分类方法.在保持语义不变的情况下,利用互联网中的相似语料对短文本的内容进行了扩展.对现有的仅使用局部特征的扩展语料特征抽取方法进行改进,引入全局特征抽取,将全局特征与局部特征相结合得到了更好的特征向量,有效地解决了分类过程中由短文本长度有限导致的特征矩阵高度稀疏的问题.通过在开放数据集上的测试和与其他文献的结果比对,验证了该方法在短文本分类的问题上可以取得较好的效果.
|
|
[1] SRIRAM B, FUHRY D, DEMIR E, et al. Short text classification in twitter to improve information filtering [C]∥ Proceedings of the 33rd international ACM SIGIR Conference on Research and Development in Information Retrieval. Geneva: ACM, 2010: 841-842.
[2] SUN A. Short text classification using very few words [C]∥ Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval. Portland: ACM, 2012: 1145-1146.
[3] YUAN Q, CONG G, THALMANN N M. Enhancing Naive Bayes with various smoothing methods for short text classification [C]∥ Proceedings of the 21st International Conference on World Wide Web. Seoul: ACM, 2012: 645-646.
[4] 李卫疆,赵铁军,王宪刚. 基于上下文的查询扩展[J].计算机研究与发展,2010,47(2): 300-304.
LI Wei-jiang,ZHAO Tie-jun, WANG Xian-gang. Context-sensitive query expansion [J]. Journal of Computer Research and Development, 2010, 47(2): 300-304.
[5] BANERJEE S, RAMANATHAN K, GUPTA A. Clustering short texts using Wikipedia [C]∥ Proceedings of the 30th International ACM SIGIR Conference on Research and Development in Information Retrieval. Amsterdam: ACM, 2007: 787-788.
[6] HU X, SUN N, ZHANG C, et al. Exploiting internal and external semantics for the clustering of short texts using world knowledge [C]∥ Proceedings of the 18th ACM Conference on Information and Knowledge Management. Hong Kong: ACM, 2009: 919-928.
[7] PHAN X H, NGUYEN L M, HORIGUCHI S. Learning to classify short and sparse text & web with hidden topics from large-scale data collections [C]∥ Proceedings of the 17th International Conference on World Wide Web. Beijing: ACM, 2008: 91-100.
[8] CHEN M, JIN X, SHEN D. Short text classification improved by learning multi-granularity topics [C]∥ Proceedings of the 22nd International Joint Conference on Artificial Intelligence. Barcelona: AAAI, 2011: 1776-1781.
[9] SAHAMI M, HEILMAN T D. A web-based kernel function for measuring the similarity of short text snippets [C]∥ Proceedings of the 15th International Conference on World Wide Web. Edinburgh: ACM, 2006: 377-386.
[10] YIH W T, CHRISTOPHER M. Improving similarity measures for short segments of text [C]∥ Proceedings of the 22nd Conference on Artificial Intelligence. Vancouver: AAAI, 2007: 1489-1494.
[11] BOLLEGALA D, MATSUO Y, ISHIZUKA M. Measuring semantic similarity between words using web search engines [C]∥ Proceedings of the 16th International Conference on World Wide Web. Banff: ACM, 2007: 757-765.
[12] EFRON M, ORGANISCIAK P, FENLON K. Improving retrieval of short texts through document expansion [C]∥ Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval. Portland: ACM, 2012: 911-920.
[13] HALL M, FRANK E, HOLMES G, et al. The WEKA data mining software: an update [J]. ACM SIGKDD Explorations Newsletter, 2009, 11(1): 10-18.
|
|
Viewed |
|
|
|
Full text
|
|
|
|
|
Abstract
|
|
|
|
|
Cited |
|
|
|
|
|
Shared |
|
|
|
|
|
Discussed |
|
|
|
|