Please wait a minute...
浙江大学学报(工学版)
计算机技术﹑电信技术     
受限玻尔兹曼机的新混合稀疏惩罚机制
刘凯1, 张立民1, 张超2
1. 海军航空工程学院 电子信息工程学系, 山东 烟台 264001; 2. 南海舰队装备部 军械处, 广东 湛江 524001
New hybrid sparse penalty mechanism of restricted Boltzmann machine
LIU Kai1, ZHANG Li-min1, ZHANG Chao2
1. Department of Electronic and Information Engineering, Naval Aeronautical and Astronautical University, Yantai 264001, China; 2. Ordnance Branch, Equipment Department of South China Sea Fleet, Zhanjiang 524001, China
 全文: PDF(1691 KB)   HTML
摘要:

为解决受限玻尔兹曼机(RBM)在学习过程中出现的特征同质化问题,在RBM已有的稀疏模型基础上提出新的混合稀疏惩罚机制(HSPM).鉴于隐单元之间存在的统计相关性,该机制通过在RBM训练过程中引入交叉熵稀疏惩罚因子,实现对RBM的初步处理;按照基于RBM连接权值列相似性的自适应分组策略,构建稀疏组RBM,并按照稀疏组受限玻尔兹曼机(SGRBM)的形式继续进行隐单元稀疏化.实验结果表明:HSPM能够有效解决RBM特征同质化问题,在隐单元的稀疏程度上优于以往的稀疏惩罚因子,可以整体提高RBM的特征提取能力,并可以成功应用于深度玻尔兹曼机(DBM)的训练.

Abstract:

A new hybrid sparse penalty mechanism (HSPM) was proposed to resolve the features homogenization problem of restricted Boltzmann machine (RBM). HSPM was based on the existing sparse restricted Boltzmann machine (SRBM). Since the statistical correlation among hidden units, a cross-entropy factor to optimize the training of RBMs was first implemented by HSPM. Then, hidden units were grouped according to adaptive grouping strategy based on the column similarity of connection weights. Finally, hidden units sparse processing was carried out in the form of sparse group restricted Boltzmann machine (SGRBM). The experimental results confirmed that HSPM could effectively resolve the feature homogenization problem of RBM and was better than ever sparse penalty factor on degree of signals sparsity. HSPM can improve the feature extraction capability of RBM and be applied to the training of deep Boltzmann machine (DBM) successfully.

出版日期: 2015-06-01
:  TP 391  
基金资助:

国家自然科学基金资助项目(61032001)

通讯作者: 张立民,男,教授     E-mail: iamzlm@163.com
作者简介: 刘凯(1986—),男,博士生,从事智能信息处理和军用仿真技术研究. E-mail: wendao_2008@163.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章  

引用本文:

刘凯, 张立民, 张超. 受限玻尔兹曼机的新混合稀疏惩罚机制[J]. 浙江大学学报(工学版), 10.3785/j.issn.1008-973X.2015.06.010.

LIU Kai1, ZHANG Li-min1, ZHANG Chao2. New hybrid sparse penalty mechanism of restricted Boltzmann machine. JOURNAL OF ZHEJIANG UNIVERSITY (ENGINEERING SCIENCE), 10.3785/j.issn.1008-973X.2015.06.010.

链接本文:

http://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2015.06.010        http://www.zjujournals.com/eng/CN/Y2015/V49/I6/1070

[1] 刘建伟,刘媛,罗雄麟.玻尔兹曼机研究进展[J].计算机研究与发展,2014, 51(1): 1-16.
LIU Jian-wei, LIU Yuan, LUO Xiong-lin. Research and development on boltzmann machine [J]. Journal of Computer Research and Development, 2014, 51(1): 1-16.
[2] 陈宇,郑德权,赵铁军.基于Deep Belief Nets的中文名实体关系抽取 [J].软件学报, 2012, 23(10): 2572-2585.
CHEN Yu, ZHENG De-quan, ZHAO Tie-jun. Chinese relation extraction based on deep belief nets [J]. Journal of Software, 2012, 23(10): 2572-2585.
[3] MOHAMED A, DAHL G E, HINTON G E. Acoustic modeling using deep belief networks [J]. IEEE Transactions on Audio, Speech, and Language Processing, 2012, 20(1): 14-22.
[4] SCHMAH T, HINTON G E, ZEMEL R, et al. Generative versus discriminative training of RBMs for classification of fMRI images [C]∥ Proceedings of the 22rd Annual Conference on Neural Information Processing Systems. Whistler: NIPS, 2008: 1409-1416.
[5] MEMISEVIC R, HINTON G E. Unsupervised learning of image transformations [C]∥ Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2007: 18.
[6] TAYLOR G W, SIGAL L, FLEET D J, et al. Dynamical binary latent variable models for 3D human pose tracking [C]∥ Proceedings of 2010 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2010: 631-638.
[7] 林妙真.基于深度学习的人脸识别研究[D].大连:大连理工大学,2013: 24-69.
LIN Miao-zhen. Research on face recognition based on deep learning [D]. Dalian: Dalian University of Technology, 2013: 24-69.
[8] 罗恒.基于协同过滤视角的受限玻尔兹曼机研究[D].上海,上海交通大学,2011: 70-79.
LUO Heng. Restricted Boltzmann machines: a collaborative filtering perspective [D]. Shanghai: Shanghai Jiao Tong University, 2011: 70-79.
[9] LEE H, EKANADHAM C, NG A. Sparse deep belief net model for visual area V2 [C]∥ Proceedings of the 21rd Annual Conference on Neural Information Processing Systems. Vancouver: NIPS, 2007: 873-880.
[10] HINTON G E. A practical guide to training restricted Boltzmann machines [R]. Canada, Toronto University: Machine Learning Group, 2010.
[11] LUO H, CHANG Y N, RUI M S, et al. Sparse group restricted boltzmann machines [C]∥ Proceedings of 25th Conference on Innovative Applications of Artificial Intelligence. San Francisco : AAAI, 2011: 1207-1216.
[12] GUO R, QI H. Partially-sparse restricted boltzmann machine for background modeling and subtraction [C]∥ Proceedings of 2013 IEEE Conference on Machine Learning and Applications. Piscataway: IEEE, 2013: 209-214.
[13] 吴一全,张晓杰,吴诗婳.2维对称交叉熵图像阈值分割[J].中国图像图形学报,2011,11(8): 122-126.
WU Yi-quan, ZHANGE Xiao-jie, WU Shi-hua. Two-dimensional symmetric cross-entropy image thresholding [J]. Journal of Image and Graphics, 2011, 11(8): 122-126.
[14] BENGIO Y. Learning deep architectures for AI [J]. Foundations and Trends in Machine Learning, 2009, 2(1): 11-27.
[15] LENG B, ZHANG X, YAO M, et al. A 3D model recognition mechanism based on deep Boltzmann machines [J]. Neuro Computing, 2015, 151(2): 593-602.
[16] SUSSKIND J. The Toronto face database [R]. Canada, Toronto University: Machine Learning Group, 2011.
[17] CHANG C C, LIN C J. LIBSVM: a library for support vector machines [J]. ACM Transactions on Intelligent Systems and Technology, 2011, 2(3): 27-33.
[18] KUREMOTO T, KIMURA S, KOBAYASHI K, et al. Time series forecasting using a deep belief network with restricted Boltzmann machines [J]. Neuro computing, 2014, 137(5): 47-56.
[19] SRIVASTAVA N, SALAKHUTDINOV R. Multimodal learning with deep Boltzmann machines [J]. Journal of Machine Learning Research, 2014, 15(4): 2949-2980.
[20] SALAKHUTDINOV R, HINTON G E. Deep Boltzmann machines [C]∥ Proceedings of 12th International Conference on Artificial Intelligence and Statistics. Florida: AISTATS, 2009: 448-455.
[1] 何雪军, 王进, 陆国栋, 刘振宇, 陈立, 金晶. 基于三角网切片及碰撞检测的工业机器人三维头像雕刻[J]. 浙江大学学报(工学版), 2017, 51(6): 1104-1110.
[2] 王桦, 韩同阳, 周可. 公安情报中基于关键图谱的群体发现算法[J]. 浙江大学学报(工学版), 2017, 51(6): 1173-1180.
[3] 尤海辉, 马增益, 唐义军, 王月兰, 郑林, 俞钟, 吉澄军. 循环流化床入炉垃圾热值软测量[J]. 浙江大学学报(工学版), 2017, 51(6): 1163-1172.
[4] 毕晓君, 王佳荟. 基于混合学习策略的教与学优化算法[J]. 浙江大学学报(工学版), 2017, 51(5): 1024-1031.
[5] 穆晶晶, 赵昕玥, 何再兴, 张树有. 基于凹凸变换与圆周拟合的重叠气泡轮廓重构[J]. 浙江大学学报(工学版), 2017, 51(4): 714-721.
[6] 黄正宇, 蒋鑫龙, 刘军发, 陈益强, 谷洋. 基于融合特征的半监督流形约束定位方法[J]. 浙江大学学报(工学版), 2017, 51(4): 655-662.
[7] 蒋鑫龙, 陈益强, 刘军发, 忽丽莎, 沈建飞. 面向自闭症患者社交距离认知的可穿戴系统[J]. 浙江大学学报(工学版), 2017, 51(4): 637-647.
[8] 王亮, 於志文, 郭斌. 基于双层多粒度知识发现的移动轨迹预测模型[J]. 浙江大学学报(工学版), 2017, 51(4): 669-674.
[9] 廖苗, 赵于前, 曾业战, 黄忠朝, 张丙奎, 邹北骥. 基于支持向量机和椭圆拟合的细胞图像自动分割[J]. 浙江大学学报(工学版), 2017, 51(4): 722-728.
[10] 戴彩艳, 陈崚, 李斌, 陈伯伦. 复杂网络中的抽样链接预测[J]. 浙江大学学报(工学版), 2017, 51(3): 554-561.
[11] 刘磊, 杨鹏, 刘作军. 采用多核相关向量机的人体步态识别[J]. 浙江大学学报(工学版), 2017, 51(3): 562-571.
[12] 郭梦丽, 达飞鹏, 邓星, 盖绍彦. 基于关键点和局部特征的三维人脸识别[J]. 浙江大学学报(工学版), 2017, 51(3): 584-589.
[13] 王海军, 葛红娟, 张圣燕. 基于核协同表示的快速目标跟踪算法[J]. 浙江大学学报(工学版), 2017, 51(2): 399-407.
[14] 张亚楠, 陈德运, 王莹洁, 刘宇鹏. 基于增量图形模式匹配的动态冷启动推荐方法[J]. 浙江大学学报(工学版), 2017, 51(2): 408-415.
[15] 刘宇鹏, 乔秀明, 赵石磊, 马春光. 统计机器翻译中大规模特征的深度融合[J]. 浙江大学学报(工学版), 2017, 51(1): 46-56.