Please wait a minute...
浙江大学学报(理学版)  2021, Vol. 48 Issue (1): 69-73    DOI: 10.3785/j.issn.1008-9497.2021.01.010
数学与计算机科学     
基于多尺度特征提取网络的图像美学量化评分方法
王欣1, 穆绍硕2, 陈华锋2
1.北京中盾安全技术开发公司,北京 100044
2.浙江传媒学院 媒体工程学院,浙江 杭州 310018
Quantitative scoring method of image aesthetics based on multi-scale feature extraction network
WANG Xin1, MU Shaoshuo2, CHEN Huafeng2
1.Beijing Zhongdun Security Technology Development Company, Beijing 100044
2.School of Media Engineering, Communication University of Zhejiang, Hangzhou 310018, China
 全文: PDF(2318 KB)   HTML  
摘要: 提出了一种基于多尺度特征提取网络的图像美学客观量化评分方法,该模型主要由多个多尺度特征提取单元级联组成,每个单元包含由3个不同卷积核组成的特征提取层、融合层和映射层。特征提取层通过联合图像的全局视图和局部视图组成网络输入端,在输出端以EMD函数为损失函数,输出分布为1~10分的概率密度质量函数,并以分布均值作为图像美学量化值。实验证明,本文方法具有可行性和有效性,解决了传统方法只进行美感二进制等级分类的问题,给出了(模拟人类思维对)图像的客观量化评分;同时在AVA数据集上获得了优于几种主流算法的分类准确度。
关键词: 图像美学多尺度特征EMD损失函数量化评分    
Abstract: An objective quantitative scoring method of image aesthetics is proposed based on multi-scale feature extraction network.The proposed model mainly comprises several multi-scale feature extraction units,each of which includes a set of feature extraction layers with different convolution kernels,a fusion layer and a mapping layer.The feature extraction layer combines the global view and the local view of the image to form the input of the network.The EMD function is used as the loss function in the softmax layer.The output is a probability density mass function from 1 to 10,and the mean is used as objective qualitative score of picture quality.Experiments show that the proposed algorithm is feasible and effective,in particular,it solves the problem that the traditional method obtains only the binary classification of aesthetic,and the classification accuracy of AVA dataset is better than that of several mainstream algorithms.
Key words: quantitative scoring    EMD loss function    multi-scale feature    image aesthetics
收稿日期: 2019-12-01 出版日期: 2021-01-20
CLC:  TP 391  
通讯作者: ORCID:http://orcid.org/0000-0001-5289-5680,E-mail:wwcucu123@163.com.     E-mail: wwcucu123@163.com
作者简介: 王欣(1972—),ORCID:http://orcid.org/0000-0002-0764-3813,男,博士,副研究员,主要从事大数据和人工智能研;
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
RSS
作者相关文章  
王欣
穆绍硕
陈华锋

引用本文:

王欣, 穆绍硕, 陈华锋. 基于多尺度特征提取网络的图像美学量化评分方法[J]. 浙江大学学报(理学版), 2021, 48(1): 69-73.

WANG Xin, MU Shaoshuo, CHEN Huafeng. Quantitative scoring method of image aesthetics based on multi-scale feature extraction network. Journal of Zhejiang University (Science Edition), 2021, 48(1): 69-73.

链接本文:

https://www.zjujournals.com/sci/CN/10.3785/j.issn.1008-9497.2021.01.010        https://www.zjujournals.com/sci/CN/Y2021/V48/I1/69

1 BHATTACHARYA S,SUKTHANKAR R,SHAH M.A framework for photo-quality assessment and enhancement based on visual aesthetics[C]//Proceedings of the 18th ACM International Conference on Multimedia.Firenze:ACM,2010:271-280. DOI:10.1145/1873951.1873990
2 DHAR S,ORDONEZ V,BERG T.High level describable attributes for predicting aesthetics and interestingness[C]//The 24th IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Colorado:IEEE,2011:1657-1664. DOI:10.1109/CVPR.2011.5995467
3 LITIAN S,TOSHIHIKO Y.Photo aesthetic quality estimation using visual complexity features[J].Multimedia Tools and Applications,2018,77(5):5189-5213.
4 TANG X O,LUO W,WANG X G.Content-based photo quality assessment[J].IEEE Transactions on Multimedia,2013,15(8):1930-1943.
5 KARAYEV S,HERTZMANN A,WINNERMOLLER H,et al. Recognizing image style[C]//Proceedings of the British Machine Vision Conference (BMVC).Guildford:BMVA Press,2014. DOI:10.5244/C.28.122
6 王伟凝,王励,赵明权,等.基于并行深度卷积神经网络的图像美感分类[J].自动化学报,2016,42(6):904-914. DOI:10.16383/j.aas.2016.c150718 WANG W N,WANG L,ZHAO M Q,et al.Image aesthetic classification using parallel deep convolutional neural networks[J].Acta Automatica Sinica,2016,42(6):904-914. DOI:10.16383/j.aas.2016.c150718
7 李素梅,常永莉,段志成.基于卷积神经网络的立体图像舒适度客观评价[J].光学学报,2018,38(6):0610003. DOI:10.3788/AOS201838.0610003 LI S M,CHANG Y L,DUAN Z C.Objective assessment of stereoscopic image comfort based on convolutional neural network[J]. Acta Optica Sinica,2018,38(6):0610003. DOI:10.3788/AOS201838. 0610003
8 SRIVASTAVA N,HINTON G E,KRIZHVSKY A,et al.Dropout:A simple way to prevent neural networks from overfitting[J].Journal of Machine Learning Research,2014,15(1):1929-1958.
9 DONG Z,SHEN X,LI H Q,et al.Photo quality assessment with DCNN that understands image well[C]//The 21st International Conference on Multimedia Modeling.Cham:Springer,2015:524-535. DOI:10.1007/978-3-319-14442-9_57
10 DONG Z,TIAN X M.Multi-level photo quality assessment with multi-view features[J].Neurocomputing,2015,168:308-319. DOI:10.1016/j.neucom.2015.05.095
11 LI Y,PU Y,XU D,et al.Image aesthetic quality evaluation using convolution neural network embedded fine-tune[J]. Optoelectronics Letters,2017(6):471-475. DOI:10.1007/978-981-10-7302-1_23
12 LU X,LIN Z,JIN H L,et al.RAPID:Rating pictorial aesthetics using deep learning[C]//The 22nd ACM International Conference on Multimedia. New York:ACM,2014:457-466.
13 KAO Y,HE R,HUANG K.Visual Aesthetic Quality Assessment with Multi-Task Deep Learning[EB/OL].[2016-08-16]. http://arxiv.org/abs/1604.04970
14 MAI L,JIN H,LIU F.Composition-preserving deep photo aesthetics assessment[C]//2016 IEEE Conference on Computer Vision and Pattern Recognitio. Las Vegas:IEEE,2016:497-506.
15 LU X,LIN Z,JIN H L,et al.Rating image aesthetics using deep learning[J].IEEE Transactions on Multimedia,2015,11(17):2021-2034.
16 HOU L,YU C P,SAMARAS M.Squared Earth Mover’s Distance-Based Loss for Training Deep Neural Networks[EB/OL].[2106-11-17].https://arxiv.org/abs/1611.05916.
17 JIN X,WU L,LI X D,et al.Predicting aesthetic score distribution through cumulative Jensen-Shannon divergence[C]//Conference on Artificial Intelligence (AAAI). New Orleans:Association for the Advancement of Artificial Intelligence,2018.
18 KONG S,SHEN X,LIN Z,et al.Photo aesthetics ranking network with attributes and content adaptation[C]//Proceedings of 14th European Conference on Computer Vision. Heidelberg:Springer-Verlag,2016:662-679. DOI:10.1007/978-3-319-46448-0_40
19 FANG H,CUI C,DENG X,et al.Image aesthetic distribution prediction with fully convolutional network[C]//The 24th International Conference on Multimedia Modeling.Cham:Springer,2018:267-278.
20 WANG Z,CHANG S,DOLCOS F,et al.Brain-Inspired Deep Networks for Image Aesthetics Assessment[EB/OL].[2016-01-16].http://arxiv.org/abs/1601.04155v2.
21 CHANG KY,LU K H,CHEN C S.Aesthetic critiques generation for photos[C]//2017 IEEE International Conference on Computer Vision (ICCV).Venice:IEEE,2017:3534-3543. DOI:10.1109/ICCV.2017.380
22 TALEBI H,PEYMAN M.NIMA:Neural image assessment[J].IEEE Transactions on Image Processing,2018,27(8):3998-4011.
[1] 方于华,叶枫. MFDC-Net:一种融合多尺度特征和注意力机制的乳腺癌病理图像分类算法[J]. 浙江大学学报(理学版), 2023, 50(4): 455-464.