Please wait a minute...
浙江大学学报(工学版)
浙江大学学报(工学版)  2019, Vol. 53 Issue (2): 373-381    DOI: 10.3785/j.issn.1008-973X.2019.02.021
计算机与控制工程     
Attention Res-Unet: 一种高效阴影检测算法
董月(),冯华君(),徐之海,陈跃庭,李奇*()
浙江大学 现代光学仪器国家重点实验室，浙江 杭州 310027
Attention Res-Unet: an efficient shadow detection algorithm
Yue DONG(),Hua-jun FENG(),Zhi-hai XU,Yue-ting CHEN,Qi LI*()
State Key Laboratory of Optical Instrumentation, Zhejiang University, Hangzhou 310027, China
 全文: PDF(1545 KB)   HTML
摘要：

图像中阴影像素的存在会导致图像内容的不确定性，对计算机视觉任务有害，因此常将阴影检测作为计算机视觉算法的预处理步骤. 提出全新的阴影检测网络结构，通过结合输入图像中包含的语义信息和像素之间的关联，提升网络性能. 使用预训练后的深层网络ResNeXt101作为特征提取前端，提取图像的语义信息，并结合U-net的设计思路，搭建网络结构，完成特征层的上采样过程. 在输出层之前使用非局部操作，为每一个像素提供全局信息，建立像素与像素之间的联系. 设计注意力生成模块和注意力融合模块，进一步提高检测准确率. 分别在SBU、UCF这2个阴影检测数据集上进行验证，实验结果表明，所提方法的目视效果及客观指标皆优于此前最优方法所得结果，在2个数据集上的平均检测错误率分别降低14.4%和14.9%.
关键词： 阴影检测;  特征提取;  语义信息;  像素关联;  非局部操作;  注意力机制;  卷积神经网络(CNN)    
Abstract:

Shadow pixels in images can lead to the uncertainty of image content, which is harmful to computer vision tasks. Therefore, shadow detection is often used as a preprocessing step of computer vision algorithm. A shadow detection network was proposed by combining semantic information contained in input images and correlation between pixels. Pre-trained deep network ResNeXt101 was used as feature extraction front-end module to extract semantic information of the image. The baseline structure of the network was built to up-sample feature layers, encouraged by the design idea of U-Net. Non-local operations were added before the output layer to provide global information for each pixel and establish the relationship between pixels. At the same time, an attention generation module and an attention fusion module were developed to further improve shadow detection accuracy. Two common shadow detection datasets named SBU and UCF were utilized for verification. Experiment results showed that the proposed network outperformed previous methods in both visual effect and objective indicator. The proposed network showed 14.4% reduction on SBU and 14.9% reduction on UCF for the balance error rate, compared with the state-of-the-art framework.
Key words: shadow detection    feature extraction    semantic information    pixel correlation    non-local    attention module    convolutional neural network (CNN)
收稿日期: 2018-10-14 出版日期: 2019-02-21
CLC:  TP 391  
通讯作者: 李奇     E-mail: 21730058@zju.edu.cn;fenghj@zju.edu.cn;liqi@zju.edu.cn
作者简介: 董月(1995—)，男，硕士生，从事图像处理研究. orcid.org/0000-0003-0433-473X. E-mail: 21730058@zju.edu.cn
服务　　
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章　　
董月
冯华君
徐之海
陈跃庭
李奇

引用本文:

董月,冯华君,徐之海,陈跃庭,李奇. Attention Res-Unet: 一种高效阴影检测算法[J]. 浙江大学学报(工学版), 2019, 53(2): 373-381.

Yue DONG,Hua-jun FENG,Zhi-hai XU,Yue-ting CHEN,Qi LI. Attention Res-Unet: an efficient shadow detection algorithm. Journal of ZheJiang University (Engineering Science), 2019, 53(2): 373-381.

链接本文:

http://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2019.02.021    或    http://www.zjujournals.com/eng/CN/Y2019/V53/I2/373

图 1  阴影成因示意图
图 2  阴影检测困难案例
图 3  Attention Res-Unet算法网络结构
图 4  Attention Res-Unet算法中注意力生成模块流程图
图 5  Attention Res-Unet算法中注意力融合模块流程图
图 6  Attention Res-Unet与scGAN[17]、patched-CNN[16]、stacked-CNN[15]检测结果的目视效果对比
方法 SBU[15] UCF[23]
BER SER NER BER SER NER
Attention Res-Unet 4.88 5.31 4.44 8.42 8.40 8.44
A+D-Net[18] 5.70 6.20 5.20 9.90 7.30 12.50
scGAN[17] 9.10 7.80 10.40 11.50 7.70 15.30
patched-CNN[16] 11.20 10.13 12.27 — — —
stacked-CNN[15] 11.00 9.60 12.50 13.00 9.00 17.10
表 1  Attention Res-Unet与A+D-Net[18]、scGAN[17]、patched-CNN[16]、stacked-CNN[15]的定量评价结果
方法 SBU[15]
BER SER NER
U-net 7.69 10.48 4.90
Res-Unet 6.56 7.40 5.73
Res-Unet+Attenion 5.33 5.88 4.77
Attention Res-Unet 4.88 5.31 4.44
表 2  Attention Res-Unet各模块作用验证
图 7  Attention Res-Unet与Res-Unet+Attenion、Res-Unet、U-net检测结果的目视效果对比
图 8  Attention Res-Unet的其他实验结果
图 9  Attention Res-Unet的失败应用案例
1 MURALI S, GOVINDAN V K, KALADY S A survey on shadow detection techniques in a single image[J]. Information Technology and Control, 2018, 47 (1): 75- 92
2 FINLAYSON G D, DREW M S, LU C. Intrinsic images by entropy minimization [C]// Computer Vision-ECCV. Prague: Springer, 2004: 582–595.
3 FINLAYSON G D, HORDLEY S D, LU C On the removal of shadows from images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28 (1): 59- 68
4 FINLAYSON G D, DREW M S, LU C Entropy minimization for shadow removal[J]. International Journal of Computer Vision, 2009, 85 (1): 35- 57
5 HOIEM D. Single-image shadow detection and removal using paired regions [C] // Computer Vision and Pattern Recognition. Colorado Springs: IEEE, 2011: 2033–2040.
6 GUO R, DAI Q, HOIEM D Paired regions for shadow detection and removal[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35 (12): 2956- 2967
7 VICENTE T F Y, YU C P, SAMARAS D. Single image shadow detection using multiple cues in a supermodular MRF [C] // British Machine Vision Conference. Bristol: British Machine Vision Association, 2013: 1–12.
8 YUAN X, EBNER M, WANG Z Single-image shadow detection and removal using local colour constancy computation[J]. Image Processing Iet, 2015, 9 (2): 118- 126
9 PANAGOPOULOS A, WANG C, SAMARAS D. Estimating shadows with the bright channel cue [C] // European Conference on Trends and Topics in Computer. Heraklion: Springer, 2010: 1–12.
10 MARYAM G, FATIMAH K, ABDULLAH L N Shadow detection using color and edge information[J]. Journal of Computer Science, 2013, 9 (11): 1575- 1588
11 SALVADOR E, CAVALLARO A, EBRAHIMI T. Cast shadow segmentation using invariant color features [J]. Computer Vision and Image Understanding, 2004, 95(2): 238-259.
12 DONG Q, LIU Y, ZHAO Q Detecting soft shadows in a single outdoor image: from local edge-based models to global constraints[J]. Computers and Graphics, 2014, 38 (1): 310- 319
13 TIAN J, QI X, QU L New spectrum ratio properties and features for shadow detection[J]. Pattern Recognition, 2016, 51 (C): 85- 96
14 KHAN S H, BENNAMOUN M, SOHEL F. Automatic feature learning for robust shadow detection [C] // Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 1939–1946.
15 VICENTE T F Y, HOU L, YU C P. Large-scale training of shadow detectors with noisily-annotated shadow examples [C] // Computer Vision-ECCV. Amsterdam: Springer, 2016: 816–832.
16 HOSSEINZADEH S, SHAKERI M, ZHANG H. Fast shadow detection from a single image using a patched convolutional neural network [EB/OL]. (2018-03-16)[2018-10-14]. http.//arxiv.org/abs/1709.09283.
17 NGUYEN V, VICENTE T F Y, ZHAO M. Shadow detection with conditional generative adversarial networks [C] // IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 4520–4528.
18 LE H, VICENTE T F Y, NGUYEN V. A+D-Net: shadow detection with adversarial shadow attenuation. [EB/OL]. (2018-07-27)[2018-10-14]. https://arxiv.org/abs/1712.01361.
19 XIE S, GIRSHICK R, DOLLAR P. Aggregated residual transformations for deep neural networks [C]// Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 5987–5995.
20 RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation [C] // International Conference on Medical Image Computing and Computer-assisted Intervention. Cham: Springer, 2015: 234–241.
21 WANG F, JIANG M, QIAN C. Residual attention network for image classification [C] // Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6450–6458.
22 WANG X, GIRSHICK R, GUPTA A. Non-local neural networks [EB/OL]. (2018-04-18)[2018-10-14]. https://arxiv.org/abs/1711.07971.
23 ZHU J, SAMUEL K G G, MASOOD S Z. Learning to recognize shadows in monochromatic natural images [C] // Computer Vision and Pattern Recognition. San Francisco: IEEE, 2010: 223–230.
24 IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift [C]// International Conference on Machine Learning. Lille: ACM, 2015: 448-456.
[1] 刘芳,汪震,刘睿迪,王锴. 基于组合损失函数的BP神经网络风力发电短期预测方法[J]. 浙江大学学报(工学版), 2021, 55(3): 594-600.
[2] 雍子叶,郭继昌,李重仪. 融入注意力机制的弱监督水下图像增强算法[J]. 浙江大学学报(工学版), 2021, 55(3): 555-562.
[3] 陈巧红,陈翊,李文书,贾宇波. 多尺度SE-Xception服装图像分类[J]. 浙江大学学报(工学版), 2020, 54(9): 1727-1735.
[4] 张彦楠,黄小红,马严,丛群. 基于深度学习的录音文本分类方法[J]. 浙江大学学报(工学版), 2020, 54(7): 1264-1271.
[5] 刘创,梁军. 基于注意力机制的车辆运动轨迹预测[J]. 浙江大学学报(工学版), 2020, 54(6): 1156-1163.
[6] 张岩,郭斌,王倩茹,张靖,於志文. SeqRec：基于长期偏好和即时兴趣的序列推荐模型[J]. 浙江大学学报(工学版), 2020, 54(6): 1177-1184.
[7] 杨萍,王丹,康子健,李童,付利华,余悦任. 基于模式识别和集成CNN-LSTM的阵发性房颤预测模型[J]. 浙江大学学报(工学版), 2020, 54(5): 1039-1048.
[8] 冯毅雄,李康杰,高一聪,郑浩,谭建荣. 基于特征与形貌重构的轴件表面缺陷检测方法[J]. 浙江大学学报(工学版), 2020, 54(3): 427-434.
[9] 乔美英,汤夏夏,闫书豪,史建柯. 基于改进稀疏滤波与深度网络融合的轴承故障诊断[J]. 浙江大学学报(工学版), 2020, 54(12): 2301-2309.
[10] 贾子钰,林友芳,张宏钧,王晶. 基于深度卷积神经网络的睡眠分期模型[J]. 浙江大学学报(工学版), 2020, 54(10): 1899-1905.
[11] 叶刚,李毅波,马逐曦,成杰. 基于ViBe的端到端铝带表面缺陷检测识别方法[J]. 浙江大学学报(工学版), 2020, 54(10): 1906-1914.
[12] 李红光,郭英,眭萍,齐子森. 基于时频特征的卷积神经网络跳频调制识别[J]. 浙江大学学报(工学版), 2020, 54(10): 1945-1954.
[13] 赵小虎,尹良飞,赵成龙. 基于全局?局部特征和自适应注意力机制的图像语义描述算法[J]. 浙江大学学报(工学版), 2020, 54(1): 126-134.
[14] 吕艳,张萌,姜吴昊,倪益华,钱小鸿. 采用卷积神经网络的老年人跌倒检测系统设计[J]. 浙江大学学报(工学版), 2019, 53(6): 1130-1138.
[15] 赫贵然,李奇,冯华君,徐之海,陈跃庭. 基于CNN特征提取的双焦相机连续数字变焦[J]. 浙江大学学报(工学版), 2019, 53(6): 1182-1189.