Please wait a minute...
浙江大学学报(工学版)  2026, Vol. 60 Issue (11): 2406-2417    DOI: 10.3785/j.issn.1008-973X.2026.11.010
计算机技术     
基于可变精度脉动阵列结构的卷积神经网络加速器
秦婷(),李钊*(),闫玉婷,杜亚飞
山东理工大学 计算机科学与技术学院,山东 淄博 255049
Convolutional neural network accelerator based on variable-precision systolic-array structure
Ting QIN(),Zhao LI*(),Yuting YAN,Yafei DU
School of Computer Science and Technology, Shandong University of Technology, Zibo 255049, China
 全文: PDF(908 KB)   HTML
摘要:

为了缓解现有方法中资源配置不合理与精度下降的问题,提出基于可变精度脉动阵列结构的CNN加速器设计方案. 在算法层面,引入基于信息熵的量化策略,评估各层信息含量并为网络分配INT4/INT8混合精度;在量化感知训练中加入KL散度损失,降低低比特量化带来的精度损失. 在硬件架构层面,构建用于执行该量化策略的专用现场可编程门阵列(FPGA)加速器;以输出固定式脉动阵列为计算核心,集成可变精度处理单元;为每个处理单元设计数字信号处理器(DSP)模块,实现并行双乘法运算以提升计算密度. 加速器采用先进先出(FIFO)滑动窗口缓存以提高片上数据复用率,将批归一化操作融入卷积计算以构建端到端处理流水线. 在Xilinx Kintex-7平台上实测所提加速器的性能. 结果表明,加速器的吞吐量、单张图片推理延迟和功耗分别为96 GOP/s、0.56 ms和13.54 W,该方案具备面向边缘应用场景的实现可行性.

关键词: 卷积神经网络(CNN);  混合精度量化;  定点量化;  脉动阵列;  现场可编程门阵列(FPGA)    
Abstract:

A CNN accelerator design based on a variable-precision systolic-array architecture was proposed to address inefficient resource provisioning and accuracy degradation in existing approaches. At the algorithm level, an entropy-based quantization strategy was introduced to evaluate the information content of each layer, and mixed-precision configurations (INT4/INT8) were assigned across the network. A KL-divergence loss term was incorporated into quantization-aware training to reduce the accuracy degradation caused by low-bit quantization. At the hardware-architecture level, a dedicated field-programmable gate array (FPGA) accelerator was developed to implement the proposed quantization scheme. An output-stationary systolic array was adopted as the computation core, and variable-precision processing elements were integrated into the accelerator. Each processing element was designed with a single digital signal processor (DSP) block and was configured to perform two parallel multiplications to improve computational density. Furthermore, a FIFO-based sliding-window buffer was employed to enhance on-chip data reuse. Batch normalization was fused with convolution to establish an efficient end-to-end processing pipeline. The performance of the proposed accelerator was evaluated on a Xilinx Kintex-7 platform. Results showed that the accelerator achieved a throughput of 96 GOP/s, a latency of 0.56 ms per-image, and a power consumption of 13.54 W, respectively. The proposed design demonstrates implementation feasibility for edge-deployment scenarios.

Key words: convolutional neural network (CNN)    mixed precision quantization    fixed-point quantization    systolic array    field-programmable gate array (FPGA)
收稿日期: 2025-12-05 出版日期: 2026-10-16
CLC:  TP 393  
基金资助: 国家重点研发计划资助项目(2022YFE0107300);山东省重点研发计划资助项目(2025TSGCCZZB0338).
通讯作者: 李钊     E-mail: qin_ting7749@163.com;lizhao_buaa@126.com
作者简介: 秦婷(2001—),女,硕士生,从事深度学习模型压缩研究. orcid.org/0009-0007-8170-6513. E-mail:qin_ting7749@163.com
服务  
把本文推荐给朋友
加入引用管理器
E-mail Alert
作者相关文章  
秦婷
李钊
闫玉婷
杜亚飞

引用本文:

秦婷,李钊,闫玉婷,杜亚飞. 基于可变精度脉动阵列结构的卷积神经网络加速器[J]. 浙江大学学报(工学版), 2026, 60(11): 2406-2417.

Ting QIN,Zhao LI,Yuting YAN,Yafei DU. Convolutional neural network accelerator based on variable-precision systolic-array structure. Journal of ZheJiang University (Engineering Science), 2026, 60(11): 2406-2417.

链接本文:

https://www.zjujournals.com/eng/CN/10.3785/j.issn.1008-973X.2026.11.010    或    https://www.zjujournals.com/eng/CN/Y2026/V60/I11/2406

图 1  基于信息熵的位宽分配流程图
图 2  ResNet20权重分布图
图 3  基于可变精度脉动阵列结构的加速器架构图
图 4  脉动阵列结构图
图 5  双乘法过程图
图 6  先进先出滑动窗口行缓存数据对齐结构图
图 7  集成可变精度处理单元与先进先出时序对齐组件的脉动阵列总体硬件结构图
图 8  适配不同卷积核尺寸的可配置先进先出滑动窗口缓存的数据对齐时序图
模型AccFP/%AccINT/%LRC
ResNet2074.4673.740.727.8×
ResNet5674.5674.090.4716.2×
表 1  CIFAR-100数据集上不同模型的性能量化实验结果
PpTop-1/%RC
3074.105.2×
5073.747.8×
7072.1510.2×
表 2  不同百分位阈值设置下ResNet20的性能对比实验结果
模型方法量化精度Top-1/%
ResNet20BaselineFP3274.46
PACTINT465.56
LSQINT471.37
LIMPQ4MP73.95
本研究INT4/874.01
ResNet56BaselineFP3274.10
PACTINT470.35
LSQINT473.56
LIMPQ4MP73.80
本研究INT4/874.06
表 3  CIFAR-100数据集上不同方法的Top-1准确率对比
实验平台硬件型号计算配置存储配置软件环境
CPUIntel(R) Xeon(R) Gold 6242R20 Cores/40 Threads64 GB DDR4MKL Enabled
GPUNVIDIA RTX 2070 SUPER2560 CUDA Cores8 GB GDDR6CUDA/cuDNN Enabled
FPGAXilinx Kintex-7 (XC7K325T)28 nm/840 DSPs1 GB DDR3Vivado 2018.2
表 4  测试平台配置表
实验平台f/MHzt/msP/WS
CPU31743.34212.001.00
GPU16051.52202.002.20
FPGA1000.5613.545.96
表 5  不同平台下ResNet20单张图像推理性能对比
模块LUTFFDSPBRAM
仲裁器37238101.0
控制器30351300.5
乘累加模块280363205326277.0
池化模块31540500.5
临时累加模块42347801
表 6  加速器各模块资源使用统计
资源类型资源总量已用资源η/%
LUT203 80029 42917.12
FF407 60033 8309.84
DSP84026229.11
BRAM4458016.00
表 7  ResNet20加速器总体资源使用情况
方案实验平台量化精度f/MHzt/msT/(GOP·s?1)EER/(GOP·W?1)
文献[30]Zynq ZU7EVFP321002.9161.4611.80
文献[31]Kintex 7INT6-81000.47179.0013.44
文献[32]Zynq 7020FP1615442.00——
文献[33]Zynq MpSoCINT83001.53130.0031.71
文献[34]ZYNQ-7020FP16100——7.39
本研究Kintex-7(ResNet20)INT4/81000.5696.007.09
本研究Kintex-7(ResNet56)INT4/81001.2396.007.10
表 8  加速器性能对比实验结果
1 陈江浩, 杨军 结合深度可分离卷积的多源遥感融合影像目标检测[J]. 浙江大学学报: 工学版, 2025, 59 (12): 2545- 2555
CHEN Jianghao, YANG Jun Object detection for multi-source remote sensing fused images based on depthwise separable convolution[J]. Journal of Zhejiang University: Engineering Science, 2025, 59 (12): 2545- 2555
2 AHMED W, KOMMINENI V K, KÖNIG-RIES B, et al Evaluating the method reproducibility of deep learning models in biodiversity research[J]. PeerJ Computer Science, 2025, 11: e2618
3 BHARATHI V, RANJITHA M An optimized deep learning method for automatic translator for ancient hieroglyphic language from scanned images to English text[J]. SN Computer Science, 2025, 6 (6): 639
doi: 10.1007/s42979-025-04165-0
4 EL-TAHER A, AZAB S, MOHAMMED A. From artifact to text: a novel deep learning approach for ancient Egyptian language processing [C]// Proceedings of the Intelligent Methods, Systems, and Applications. Giza: IEEE, 2025: 434–441.
5 JHA K, JAIN A, SRIVASTAVA S A challenge-response based authentication approach for multimodal biometric system using deep learning techniques[J]. Scalable Computing: Practice and Experience, 2025, 26 (5): 2118- 2129
6 CHANDRABANSHI V, DOMNIC S Leveraging 3D-CNN and graph neural network with attention mechanism for visual speech recognition[J]. Signal, Image and Video Processing, 2025, 19 (10): 844
doi: 10.1007/s11760-025-04438-4
7 NAGEL M, FOURNARAKIS M, AMJAD R, et al. A white paper on neural network quantization [EB/OL]. (2021–06–15) [2025–11–10]. https://arxiv.org/pdf/2106.08295.
8 HAN S, MAO H, DALLY W J. Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding [EB/OL]. (2016–02–15) [2025–11–10]. https://arxiv.org/pdf/1510.00149.
9 SAHA R, SAGAN N, SRIVASTAVA V, et al. Compressing large language models using low rank and low precision decomposition [C]// Proceedings of the 38th International Conference on Neural Information Processing Systems. [S.l.]: Curran Associates Inc. , 2024: 88981–89018.
10 WANG S, KANG Y Gradient distribution-aware INT8 training for neural networks[J]. Neurocomputing, 2023, 541: 126269
11 DONG Z, YAO Z, GHOLAMI A, et al. HAWQ: Hessian aware quantization of neural networks with mixed-precision [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2020: 293–302.
12 DONG Z, YAO Z, CAI Y, et al. HAWQ-V2: Hessian aware trace-weighted quantization of neural networks [C]// 34th Conference on Neural Information Processing Systems. Vancouver: [s.n.], 2020: 1–12.
13 KLOBERDANZ E, LE W. MixQuant: mixed precision quantization with a bit-width optimization search [EB/OL]. (2023–09–29)[2025–11–10]. https://arxiv.org/pdf/2309.17341.
14 TANG C, OUYANG K, WANG Z, et al. Mixed-precision neural network quantization via learned layer-wise importance [C]// Computer Vision – ECCV 2022. Cham: Springer, 2022: 259–275.
15 MA Y, JIN T, ZHENG X, et al OMPQ: orthogonal mixed precision quantization[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37 (7): 9029- 9037
16 SHANNON C E A mathematical theory of communication[J]. The Bell System Technical Journal, 1948, 27 (3): 379- 423
17 BOUDJADAR J, ISLAM S U, BUYYA R Dynamic FPGA reconfiguration for scalable embedded artificial intelligence (AI): a co-design methodology for convolutional neural networks (CNN) acceleration[J]. Future Generation Computer Systems, 2025, 169: 107777
18 ZHANG C, PRASANNA V. Frequency domain acceleration of convolutional neural networks on CPU-FPGA shared memory system [C]// Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. Monterey: ACM, 2017: 35–44.
19 ZENG H, CHEN R, ZHANG C, et al. A framework for generating high throughput CNN implementations on FPGAs [C]// Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. Monterey: ACM, 2018: 117–126.
20 WINOGRAD S. Arithmetic complexity of computations [M]. Philadelphia: Society for Industrial and Applied Mathematics, 1980.
21 PODILI A, ZHANG C, PRASANNA V. Fast and efficient implementation of convolutional neural networks on FPGA [C]// Proceedings of the IEEE 28th International Conference on Application-specific Systems, Architectures and Processors. Seattle: IEEE, 2017: 11–18.
22 LU L, LIANG Y, XIAO Q, et al. Evaluating fast algorithms for convolutional neural networks on FPGAs [C]// Proceedings of the IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines. [S.l.]: IEEE, 2017: 101–108.
23 ZHANG X, SONG X, LIU S, et al MRSA: a multi-mode reconfigurable systolic array architecture based on channel-wise stationary dataflow for CNNs[J]. Microelectronics Journal, 2026, 168: 106983
24 JOSE A, JACOB J. Novel FPGA based systolic array design for dynamic workflows [C]// Proceedings of the 2nd International Conference on Trends in Engineering Systems and Technologies. Ernakulam: IEEE, 2025: 1–5.
25 YE H, ZHENG X, HUA S, et al. Systolic array-based CNN accelerator for gesture recognition [EB/OL]. [2025–11–10]. https://www.cs.columbia.edu/~sedwards/classes/2025/4840-spring/proposals/Systolic.pdf.
26 XU H, LI Z, JIN Z, et al. Effective and efficient mixed precision quantization of speech foundation models [C]// Proceedings of the ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing. Hyderabad: IEEE, 2025: 1–5.
27 KUMAR N, NARANG A, LALL B Kullback–Leibler divergence-based regularized normalization for low-resource tasks[J]. IEEE Transactions on Artificial Intelligence, 2024, 5 (6): 2638- 2650
28 CHOI J, WANG Z, VENKATARAMANI S, et al. PACT: parameterized clipping activation for quantized neural networks [EB/OL]. (2018–07–17) [2025–11–10]. https://arxiv.org/pdf/1805.06085.
29 ESSER S K, MCKINSTRY J L, BABLANI D, et al. Learned step size quantization: US11423054 [P]. 2023–11–21.
30 ISIK M, INADAGBO K, AKTAS H. Design optimization for high-performance computing using FPGA [C]// Information Management and Big Data. Cham: Springer, 2024: 142–156.
31 HE Y, LIU W, TAHIR M, et al Research on efficient CNN acceleration through mixed precision quantization: a comprehensive methodology[J]. International Journal of Advanced Computer Science and Applications, 2023, 14 (12): 806- 817
32 ABTAHI T, SHEA C, KULKARNI A, et al Accelerating convolutional neural network with FFT on embedded hardware[J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2018, 26 (9): 1737- 1749
33 画芊昊, 李博, 杜宸罡 基于FPGA的深度可分离卷积加速器研究[J]. 计算机测量与控制, 2024, 32 (5): 267- 273
HUA Qianhao, LI Bo, DU Chengang Research on deep separable convolution accelerator based on FPGA[J]. Computer Measurement and Control, 2024, 32 (5): 267- 273
34 李珍琪, 王强, 齐星云, 等 轻量化卷积神经网络硬件加速设计及FPGA实现[J]. 计算机工程与科学, 2025, 47 (4): 582- 591
LI Zhenqi, WANG Qiang, QI Xingyun, et al Design and FPGA implementation of lightweight convolutional neural network hardware acceleration[J]. Computer Engineering and Science, 2025, 47 (4): 582- 591
[1] 王立红,刘新倩,李静,冯志全. 基于联邦学习和时空特征融合的网络入侵检测方法[J]. 浙江大学学报(工学版), 2025, 59(6): 1201-1210.
[2] 安春丽,张碧玲,赵国安,王博,刘岩. 基于FFT-CNN-GCN的电网故障诊断[J]. 浙江大学学报(工学版), 2025, 59(10): 2205-2212.
[3] 耿志强,陈威,马波,韩永明. 基于连续小波卷积神经网络的轴承智能故障诊断方法[J]. 浙江大学学报(工学版), 2024, 58(10): 2069-2075.
[4] 莫仁鹏,司小胜,李天梅,朱旭. 基于多尺度特征与注意力机制的轴承寿命预测[J]. 浙江大学学报(工学版), 2022, 56(7): 1447-1456.
[5] 陈巧红,陈翊,李文书,贾宇波. 多尺度SE-Xception服装图像分类[J]. 浙江大学学报(工学版), 2020, 54(9): 1727-1735.
[6] 张彦楠,黄小红,马严,丛群. 基于深度学习的录音文本分类方法[J]. 浙江大学学报(工学版), 2020, 54(7): 1264-1271.
[7] 杨萍,王丹,康子健,李童,付利华,余悦任. 基于模式识别和集成CNN-LSTM的阵发性房颤预测模型[J]. 浙江大学学报(工学版), 2020, 54(5): 1039-1048.
[8] 叶刚,李毅波,马逐曦,成杰. 基于ViBe的端到端铝带表面缺陷检测识别方法[J]. 浙江大学学报(工学版), 2020, 54(10): 1906-1914.
[9] 李红光,郭英,眭萍,齐子森. 基于时频特征的卷积神经网络跳频调制识别[J]. 浙江大学学报(工学版), 2020, 54(10): 1945-1954.
[10] 贾子钰,林友芳,张宏钧,王晶. 基于深度卷积神经网络的睡眠分期模型[J]. 浙江大学学报(工学版), 2020, 54(10): 1899-1905.
[11] 吕艳,张萌,姜吴昊,倪益华,钱小鸿. 采用卷积神经网络的老年人跌倒检测系统设计[J]. 浙江大学学报(工学版), 2019, 53(6): 1130-1138.
[12] 赫贵然,李奇,冯华君,徐之海,陈跃庭. 基于CNN特征提取的双焦相机连续数字变焦[J]. 浙江大学学报(工学版), 2019, 53(6): 1182-1189.
[13] 董月,冯华君,徐之海,陈跃庭,李奇. Attention Res-Unet: 一种高效阴影检测算法[J]. 浙江大学学报(工学版), 2019, 53(2): 373-381.
[14] 王敬昌,陈岭,余珊珊,蒋晨书,吴勇. 基于门控循环单元的多因素感知短期游客人数预测模型[J]. 浙江大学学报(工学版), 2019, 53(12): 2357-2364.
[15] 李浩,赵文杰,韩波. 基于滤波器裁剪的卷积神经网络加速算法[J]. 浙江大学学报(工学版), 2019, 53(10): 1994-2002.