Please wait a minute...
浙江大学学报（理学版）
Journal of Zhejiang University (Science Edition)  2021, Vol. 48 Issue (1): 69-73    DOI: 10.3785/j.issn.1008-9497.2021.01.010
Mathematics and Computer Science     
Quantitative scoring method of image aesthetics based on multi-scale feature extraction network
WANG Xin1, MU Shaoshuo2, CHEN Huafeng2
1.Beijing Zhongdun Security Technology Development Company, Beijing 100044
2.School of Media Engineering, Communication University of Zhejiang, Hangzhou 310018, China
Download: HTML ( )    PDF(2318KB)
Export: BibTeX | EndNote (RIS)      

Abstract  An objective quantitative scoring method of image aesthetics is proposed based on multi-scale feature extraction network.The proposed model mainly comprises several multi-scale feature extraction units,each of which includes a set of feature extraction layers with different convolution kernels,a fusion layer and a mapping layer.The feature extraction layer combines the global view and the local view of the image to form the input of the network.The EMD function is used as the loss function in the softmax layer.The output is a probability density mass function from 1 to 10,and the mean is used as objective qualitative score of picture quality.Experiments show that the proposed algorithm is feasible and effective,in particular,it solves the problem that the traditional method obtains only the binary classification of aesthetic,and the classification accuracy of AVA dataset is better than that of several mainstream algorithms.

Key words： quantitative scoring      EMD loss function      multi-scale feature      image aesthetics     
Received: 01 December 2019      Published: 20 January 2021
CLC:  TP 391  
Service
E-mail this article
Add to my bookshelf
Add to citation manager
E-mail Alert
RSS
Articles by authors
WANG Xin
MU Shaoshuo
CHEN Huafeng
Cite this article:

WANG Xin, MU Shaoshuo, CHEN Huafeng. Quantitative scoring method of image aesthetics based on multi-scale feature extraction network. Journal of Zhejiang University (Science Edition), 2021, 48(1): 69-73.

URL:

https://www.zjujournals.com/sci/EN/Y2021/V48/I1/69


基于多尺度特征提取网络的图像美学量化评分方法

提出了一种基于多尺度特征提取网络的图像美学客观量化评分方法，该模型主要由多个多尺度特征提取单元级联组成，每个单元包含由3个不同卷积核组成的特征提取层、融合层和映射层。特征提取层通过联合图像的全局视图和局部视图组成网络输入端，在输出端以EMD函数为损失函数，输出分布为1~10分的概率密度质量函数，并以分布均值作为图像美学量化值。实验证明，本文方法具有可行性和有效性，解决了传统方法只进行美感二进制等级分类的问题，给出了（模拟人类思维对）图像的客观量化评分；同时在AVA数据集上获得了优于几种主流算法的分类准确度。

关键词： 图像美学,  多尺度特征,  EMD损失函数,  量化评分 
1 BHATTACHARYA S，SUKTHANKAR R，SHAH M.A framework for photo-quality assessment and enhancement based on visual aesthetics［C］//Proceedings of the 18th ACM International Conference on Multimedia.Firenze：ACM，2010：271-280. DOI：10.1145/1873951.1873990
2 DHAR S，ORDONEZ V，BERG T.High level describable attributes for predicting aesthetics and interestingness［C］//The 24th IEEE Conference on Computer Vision and Pattern Recognition （CVPR）.Colorado：IEEE，2011：1657-1664. DOI：10.1109/CVPR.2011.5995467
3 LITIAN S，TOSHIHIKO Y.Photo aesthetic quality estimation using visual complexity features［J］.Multimedia Tools and Applications，2018，77（5）：5189-5213.
4 TANG X O，LUO W，WANG X G.Content-based photo quality assessment［J］.IEEE Transactions on Multimedia，2013，15（8）：1930-1943.
5 KARAYEV S，HERTZMANN A，WINNERMOLLER H，et al. Recognizing image style［C］//Proceedings of the British Machine Vision Conference （BMVC）.Guildford：BMVA Press，2014. DOI：10.5244/C.28.122
6 王伟凝，王励，赵明权，等.基于并行深度卷积神经网络的图像美感分类［J］.自动化学报，2016，42（6）：904-914. DOI：10.16383/j.aas.2016.c150718 WANG W N，WANG L，ZHAO M Q，et al.Image aesthetic classification using parallel deep convolutional neural networks［J］.Acta Automatica Sinica，2016，42（6）：904-914. DOI：10.16383/j.aas.2016.c150718
7 李素梅，常永莉，段志成.基于卷积神经网络的立体图像舒适度客观评价［J］.光学学报，2018，38（6）：0610003. DOI：10.3788/AOS201838.0610003 LI S M，CHANG Y L，DUAN Z C.Objective assessment of stereoscopic image comfort based on convolutional neural network［J］. Acta Optica Sinica，2018，38（6）：0610003. DOI：10.3788/AOS201838. 0610003
8 SRIVASTAVA N，HINTON G E，KRIZHVSKY A，et al.Dropout：A simple way to prevent neural networks from overfitting［J］.Journal of Machine Learning Research，2014，15（1）：1929-1958.
9 DONG Z，SHEN X，LI H Q，et al.Photo quality assessment with DCNN that understands image well［C］//The 21st International Conference on Multimedia Modeling.Cham：Springer，2015：524-535. DOI：10.1007/978-3-319-14442-9_57
10 DONG Z，TIAN X M.Multi-level photo quality assessment with multi-view features［J］.Neurocomputing，2015，168：308-319. DOI：10.1016/j.neucom.2015.05.095
11 LI Y，PU Y，XU D，et al.Image aesthetic quality evaluation using convolution neural network embedded fine-tune［J］. Optoelectronics Letters，2017（6）：471-475. DOI：10.1007/978-981-10-7302-1_23
12 LU X，LIN Z，JIN H L，et al.RAPID：Rating pictorial aesthetics using deep learning［C］//The 22nd ACM International Conference on Multimedia. New York：ACM，2014：457-466.
13 KAO Y，HE R，HUANG K.Visual Aesthetic Quality Assessment with Multi-Task Deep Learning［EB/OL］.［2016-08-16］. http：//arxiv.org/abs/1604.04970
14 MAI L，JIN H，LIU F.Composition-preserving deep photo aesthetics assessment［C］//2016 IEEE Conference on Computer Vision and Pattern Recognitio. Las Vegas：IEEE，2016：497-506.
15 LU X，LIN Z，JIN H L，et al.Rating image aesthetics using deep learning［J］.IEEE Transactions on Multimedia，2015，11（17）：2021-2034.
16 HOU L，YU C P，SAMARAS M.Squared Earth Mover’s Distance-Based Loss for Training Deep Neural Networks［EB/OL］.［2106-11-17］.https：//arxiv.org/abs/1611.05916.
17 JIN X，WU L，LI X D，et al.Predicting aesthetic score distribution through cumulative Jensen-Shannon divergence［C］//Conference on Artificial Intelligence （AAAI）. New Orleans：Association for the Advancement of Artificial Intelligence，2018.
18 KONG S，SHEN X，LIN Z，et al.Photo aesthetics ranking network with attributes and content adaptation［C］//Proceedings of 14th European Conference on Computer Vision. Heidelberg：Springer-Verlag，2016：662-679. DOI：10.1007/978-3-319-46448-0_40
19 FANG H，CUI C，DENG X，et al.Image aesthetic distribution prediction with fully convolutional network［C］//The 24th International Conference on Multimedia Modeling.Cham：Springer，2018：267-278.
20 WANG Z，CHANG S，DOLCOS F，et al.Brain-Inspired Deep Networks for Image Aesthetics Assessment［EB/OL］.［2016-01-16］.http：//arxiv.org/abs/1601.04155v2.
21 CHANG KY，LU K H，CHEN C S.Aesthetic critiques generation for photos［C］//2017 IEEE International Conference on Computer Vision （ICCV）.Venice：IEEE，2017：3534-3543. DOI：10.1109/ICCV.2017.380
22 TALEBI H，PEYMAN M.NIMA：Neural image assessment［J］.IEEE Transactions on Image Processing，2018，27（8）：3998-4011.
[1] Yuhua FANG,Feng YE. MFDC-Net: A breast cancer pathological image classification algorithm incorporating multi-scale feature fusion and attention mechanism[J]. Journal of Zhejiang University (Science Edition), 2023, 50(4): 455-464.