Please wait a minute...
Journal of ZheJiang University (Engineering Science)  2026, Vol. 60 Issue (11): 2406-2417    DOI: 10.3785/j.issn.1008-973X.2026.11.010
    
Convolutional neural network accelerator based on variable-precision systolic-array structure
Ting QIN(),Zhao LI*(),Yuting YAN,Yafei DU
School of Computer Science and Technology, Shandong University of Technology, Zibo 255049, China
Download: HTML     PDF(908KB) HTML
Export: BibTeX | EndNote (RIS)      

Abstract  

A CNN accelerator design based on a variable-precision systolic-array architecture was proposed to address inefficient resource provisioning and accuracy degradation in existing approaches. At the algorithm level, an entropy-based quantization strategy was introduced to evaluate the information content of each layer, and mixed-precision configurations (INT4/INT8) were assigned across the network. A KL-divergence loss term was incorporated into quantization-aware training to reduce the accuracy degradation caused by low-bit quantization. At the hardware-architecture level, a dedicated field-programmable gate array (FPGA) accelerator was developed to implement the proposed quantization scheme. An output-stationary systolic array was adopted as the computation core, and variable-precision processing elements were integrated into the accelerator. Each processing element was designed with a single digital signal processor (DSP) block and was configured to perform two parallel multiplications to improve computational density. Furthermore, a FIFO-based sliding-window buffer was employed to enhance on-chip data reuse. Batch normalization was fused with convolution to establish an efficient end-to-end processing pipeline. The performance of the proposed accelerator was evaluated on a Xilinx Kintex-7 platform. Results showed that the accelerator achieved a throughput of 96 GOP/s, a latency of 0.56 ms per-image, and a power consumption of 13.54 W, respectively. The proposed design demonstrates implementation feasibility for edge-deployment scenarios.



Key words: convolutional neural network (CNN)      mixed precision quantization      fixed-point quantization      systolic array      field-programmable gate array (FPGA)     
Received: 05 December 2025      Published: 16 October 2026
CLC:  TP 393  
Fund:  国家重点研发计划资助项目(2022YFE0107300);山东省重点研发计划资助项目(2025TSGCCZZB0338).
Corresponding Authors: Zhao LI     E-mail: qin_ting7749@163.com;lizhao_buaa@126.com
Cite this article:

Ting QIN,Zhao LI,Yuting YAN,Yafei DU. Convolutional neural network accelerator based on variable-precision systolic-array structure. Journal of ZheJiang University (Engineering Science), 2026, 60(11): 2406-2417.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2026.11.010     OR     https://www.zjujournals.com/eng/Y2026/V60/I11/2406


基于可变精度脉动阵列结构的卷积神经网络加速器

为了缓解现有方法中资源配置不合理与精度下降的问题,提出基于可变精度脉动阵列结构的CNN加速器设计方案. 在算法层面,引入基于信息熵的量化策略,评估各层信息含量并为网络分配INT4/INT8混合精度;在量化感知训练中加入KL散度损失,降低低比特量化带来的精度损失. 在硬件架构层面,构建用于执行该量化策略的专用现场可编程门阵列(FPGA)加速器;以输出固定式脉动阵列为计算核心,集成可变精度处理单元;为每个处理单元设计数字信号处理器(DSP)模块,实现并行双乘法运算以提升计算密度. 加速器采用先进先出(FIFO)滑动窗口缓存以提高片上数据复用率,将批归一化操作融入卷积计算以构建端到端处理流水线. 在Xilinx Kintex-7平台上实测所提加速器的性能. 结果表明,加速器的吞吐量、单张图片推理延迟和功耗分别为96 GOP/s、0.56 ms和13.54 W,该方案具备面向边缘应用场景的实现可行性.


关键词: 卷积神经网络(CNN),  混合精度量化,  定点量化,  脉动阵列,  现场可编程门阵列(FPGA) 
Fig.1 Flowchart for bit-width allocation based on entropy
Fig.2 ResNet20 weight distribution
Fig.3 Architecture diagram of accelerator based on variable-precision systolic-array structure
Fig.4 Diagram of systolic-array structure
Fig.5 Double multiplication process diagram
Fig.6 Data alignment structure in FIFO-based sliding-window line buffer
Fig.7 Overall hardware architecture of systolic array integrating reconfigurable precision processing elements and FIFO based timing alignment modules
Fig.8 Timing diagram of data alignment for configurable FIFO sliding window buffer adapting to different kernel sizes
模型AccFP/%AccINT/%LRC
ResNet2074.4673.740.727.8×
ResNet5674.5674.090.4716.2×
Tab.1 Experimental results on quantization performance for different models on CIFAR-100 dataset
PpTop-1/%RC
3074.105.2×
5073.747.8×
7072.1510.2×
Tab.2 Performance comparison of ResNet20 under different percentile threshold
模型方法量化精度Top-1/%
ResNet20BaselineFP3274.46
PACTINT465.56
LSQINT471.37
LIMPQ4MP73.95
本研究INT4/874.01
ResNet56BaselineFP3274.10
PACTINT470.35
LSQINT473.56
LIMPQ4MP73.80
本研究INT4/874.06
Tab.3 Top-1 accuracy comparison of different methods on CIFAR-100 dataset
实验平台硬件型号计算配置存储配置软件环境
CPUIntel(R) Xeon(R) Gold 6242R20 Cores/40 Threads64 GB DDR4MKL Enabled
GPUNVIDIA RTX 2070 SUPER2560 CUDA Cores8 GB GDDR6CUDA/cuDNN Enabled
FPGAXilinx Kintex-7 (XC7K325T)28 nm/840 DSPs1 GB DDR3Vivado 2018.2
Tab.4 Test platform configuration
实验平台f/MHzt/msP/WS
CPU31743.34212.001.00
GPU16051.52202.002.20
FPGA1000.5613.545.96
Tab.5 Comparison of ResNet20 single-image inference performance across different platforms
模块LUTFFDSPBRAM
仲裁器37238101.0
控制器30351300.5
乘累加模块280363205326277.0
池化模块31540500.5
临时累加模块42347801
Tab.6 Resource usage statistics for each accelerator module
资源类型资源总量已用资源η/%
LUT203 80029 42917.12
FF407 60033 8309.84
DSP84026229.11
BRAM4458016.00
Tab.7 Overall resource usage of ResNet20 accelerator
方案实验平台量化精度f/MHzt/msT/(GOP·s?1)EER/(GOP·W?1)
文献[30]Zynq ZU7EVFP321002.9161.4611.80
文献[31]Kintex 7INT6-81000.47179.0013.44
文献[32]Zynq 7020FP1615442.00——
文献[33]Zynq MpSoCINT83001.53130.0031.71
文献[34]ZYNQ-7020FP16100——7.39
本研究Kintex-7(ResNet20)INT4/81000.5696.007.09
本研究Kintex-7(ResNet56)INT4/81001.2396.007.10
Tab.8 Experimental results on accelerator performance comparison
[1]   陈江浩, 杨军 结合深度可分离卷积的多源遥感融合影像目标检测[J]. 浙江大学学报: 工学版, 2025, 59 (12): 2545- 2555
CHEN Jianghao, YANG Jun Object detection for multi-source remote sensing fused images based on depthwise separable convolution[J]. Journal of Zhejiang University: Engineering Science, 2025, 59 (12): 2545- 2555
[2]   AHMED W, KOMMINENI V K, KÖNIG-RIES B, et al Evaluating the method reproducibility of deep learning models in biodiversity research[J]. PeerJ Computer Science, 2025, 11: e2618
[3]   BHARATHI V, RANJITHA M An optimized deep learning method for automatic translator for ancient hieroglyphic language from scanned images to English text[J]. SN Computer Science, 2025, 6 (6): 639
doi: 10.1007/s42979-025-04165-0
[4]   EL-TAHER A, AZAB S, MOHAMMED A. From artifact to text: a novel deep learning approach for ancient Egyptian language processing [C]// Proceedings of the Intelligent Methods, Systems, and Applications. Giza: IEEE, 2025: 434–441.
[5]   JHA K, JAIN A, SRIVASTAVA S A challenge-response based authentication approach for multimodal biometric system using deep learning techniques[J]. Scalable Computing: Practice and Experience, 2025, 26 (5): 2118- 2129
[6]   CHANDRABANSHI V, DOMNIC S Leveraging 3D-CNN and graph neural network with attention mechanism for visual speech recognition[J]. Signal, Image and Video Processing, 2025, 19 (10): 844
doi: 10.1007/s11760-025-04438-4
[7]   NAGEL M, FOURNARAKIS M, AMJAD R, et al. A white paper on neural network quantization [EB/OL]. (2021–06–15) [2025–11–10]. https://arxiv.org/pdf/2106.08295.
[8]   HAN S, MAO H, DALLY W J. Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding [EB/OL]. (2016–02–15) [2025–11–10]. https://arxiv.org/pdf/1510.00149.
[9]   SAHA R, SAGAN N, SRIVASTAVA V, et al. Compressing large language models using low rank and low precision decomposition [C]// Proceedings of the 38th International Conference on Neural Information Processing Systems. [S.l.]: Curran Associates Inc. , 2024: 88981–89018.
[10]   WANG S, KANG Y Gradient distribution-aware INT8 training for neural networks[J]. Neurocomputing, 2023, 541: 126269
[11]   DONG Z, YAO Z, GHOLAMI A, et al. HAWQ: Hessian aware quantization of neural networks with mixed-precision [C]// Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2020: 293–302.
[12]   DONG Z, YAO Z, CAI Y, et al. HAWQ-V2: Hessian aware trace-weighted quantization of neural networks [C]// 34th Conference on Neural Information Processing Systems. Vancouver: [s.n.], 2020: 1–12.
[13]   KLOBERDANZ E, LE W. MixQuant: mixed precision quantization with a bit-width optimization search [EB/OL]. (2023–09–29)[2025–11–10]. https://arxiv.org/pdf/2309.17341.
[14]   TANG C, OUYANG K, WANG Z, et al. Mixed-precision neural network quantization via learned layer-wise importance [C]// Computer Vision – ECCV 2022. Cham: Springer, 2022: 259–275.
[15]   MA Y, JIN T, ZHENG X, et al OMPQ: orthogonal mixed precision quantization[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2023, 37 (7): 9029- 9037
[16]   SHANNON C E A mathematical theory of communication[J]. The Bell System Technical Journal, 1948, 27 (3): 379- 423
[17]   BOUDJADAR J, ISLAM S U, BUYYA R Dynamic FPGA reconfiguration for scalable embedded artificial intelligence (AI): a co-design methodology for convolutional neural networks (CNN) acceleration[J]. Future Generation Computer Systems, 2025, 169: 107777
[18]   ZHANG C, PRASANNA V. Frequency domain acceleration of convolutional neural networks on CPU-FPGA shared memory system [C]// Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. Monterey: ACM, 2017: 35–44.
[19]   ZENG H, CHEN R, ZHANG C, et al. A framework for generating high throughput CNN implementations on FPGAs [C]// Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays. Monterey: ACM, 2018: 117–126.
[20]   WINOGRAD S. Arithmetic complexity of computations [M]. Philadelphia: Society for Industrial and Applied Mathematics, 1980.
[21]   PODILI A, ZHANG C, PRASANNA V. Fast and efficient implementation of convolutional neural networks on FPGA [C]// Proceedings of the IEEE 28th International Conference on Application-specific Systems, Architectures and Processors. Seattle: IEEE, 2017: 11–18.
[22]   LU L, LIANG Y, XIAO Q, et al. Evaluating fast algorithms for convolutional neural networks on FPGAs [C]// Proceedings of the IEEE 25th Annual International Symposium on Field-Programmable Custom Computing Machines. [S.l.]: IEEE, 2017: 101–108.
[23]   ZHANG X, SONG X, LIU S, et al MRSA: a multi-mode reconfigurable systolic array architecture based on channel-wise stationary dataflow for CNNs[J]. Microelectronics Journal, 2026, 168: 106983
[24]   JOSE A, JACOB J. Novel FPGA based systolic array design for dynamic workflows [C]// Proceedings of the 2nd International Conference on Trends in Engineering Systems and Technologies. Ernakulam: IEEE, 2025: 1–5.
[25]   YE H, ZHENG X, HUA S, et al. Systolic array-based CNN accelerator for gesture recognition [EB/OL]. [2025–11–10]. https://www.cs.columbia.edu/~sedwards/classes/2025/4840-spring/proposals/Systolic.pdf.
[26]   XU H, LI Z, JIN Z, et al. Effective and efficient mixed precision quantization of speech foundation models [C]// Proceedings of the ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing. Hyderabad: IEEE, 2025: 1–5.
[27]   KUMAR N, NARANG A, LALL B Kullback–Leibler divergence-based regularized normalization for low-resource tasks[J]. IEEE Transactions on Artificial Intelligence, 2024, 5 (6): 2638- 2650
[28]   CHOI J, WANG Z, VENKATARAMANI S, et al. PACT: parameterized clipping activation for quantized neural networks [EB/OL]. (2018–07–17) [2025–11–10]. https://arxiv.org/pdf/1805.06085.
[29]   ESSER S K, MCKINSTRY J L, BABLANI D, et al. Learned step size quantization: US11423054 [P]. 2023–11–21.
[30]   ISIK M, INADAGBO K, AKTAS H. Design optimization for high-performance computing using FPGA [C]// Information Management and Big Data. Cham: Springer, 2024: 142–156.
[31]   HE Y, LIU W, TAHIR M, et al Research on efficient CNN acceleration through mixed precision quantization: a comprehensive methodology[J]. International Journal of Advanced Computer Science and Applications, 2023, 14 (12): 806- 817
[32]   ABTAHI T, SHEA C, KULKARNI A, et al Accelerating convolutional neural network with FFT on embedded hardware[J]. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2018, 26 (9): 1737- 1749
[33]   画芊昊, 李博, 杜宸罡 基于FPGA的深度可分离卷积加速器研究[J]. 计算机测量与控制, 2024, 32 (5): 267- 273
HUA Qianhao, LI Bo, DU Chengang Research on deep separable convolution accelerator based on FPGA[J]. Computer Measurement and Control, 2024, 32 (5): 267- 273
[34]   李珍琪, 王强, 齐星云, 等 轻量化卷积神经网络硬件加速设计及FPGA实现[J]. 计算机工程与科学, 2025, 47 (4): 582- 591
LI Zhenqi, WANG Qiang, QI Xingyun, et al Design and FPGA implementation of lightweight convolutional neural network hardware acceleration[J]. Computer Engineering and Science, 2025, 47 (4): 582- 591
[1] Lihong WANG,Xinqian LIU,Jing LI,Zhiquan FENG. Network intrusion detection method based on federated learning and spatiotemporal feature fusion[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(6): 1201-1210.
[2] Chunli AN,Biling ZHANG,Guoan ZHAO,Bo WANG,Yan LIU. Power grid fault diagnosis based on FFT-CNN-GCN[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(10): 2205-2212.
[3] Zhiqiang GENG,Wei CHEN,Bo MA,Yongming HAN. Bearing intelligent fault diagnosis method based on continuous wavelet convolutional neural network[J]. Journal of ZheJiang University (Engineering Science), 2024, 58(10): 2069-2075.
[4] Ren-peng MO,Xiao-sheng SI,Tian-mei LI,Xu ZHU. Bearing life prediction based on multi-scale features and attention mechanism[J]. Journal of ZheJiang University (Engineering Science), 2022, 56(7): 1447-1456.
[5] Qiao-hong CHEN,YI CHEN,Wen-shu Li,Yu-bo JIA. Clothing image classification based on multi-scale SE-Xception[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(9): 1727-1735.
[6] Ping YANG,Dan WANG,Zi-jian KAGN,Tong LI,Li-hua FU,Yue-ren YU. Prediction model of paroxysmal atrial fibrillation based on pattern recognition and ensemble CNN-LSTM[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(5): 1039-1048.
[7] Hong-guang LI,Ying GUO,Ping SUI,Zi-sen QI. Frequency hopping modulation recognition of convolutional neural network based on time-frequency characteristics[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1945-1954.
[8] Gang YE,Yi-bo LI,Zhu-xi MA,Jie CHENG. End-to-end aluminum strip surface defects detection and recognition method based on ViBe[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1906-1914.
[9] Zi-yu JIA,You-fang LIN,Hong-jun ZHANG,Jing WANG. Sleep stage classification model based ondeep convolutional neural network[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(10): 1899-1905.
[10] Yan LV,Meng ZHANG,Wu-hao JIANG,Yi-hua NI,Xiao-hong QIAN. Design of elderly fall detection system using CNN[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(6): 1130-1138.
[11] Gui-ran HE,Qi LI,Hua-jun FENG,Zhi-hai XU,Yue-ting CHEN. Dual-focal camera continuous digital zoom based onCNN and feature extraction[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(6): 1182-1189.
[12] Yue DONG,Hua-jun FENG,Zhi-hai XU,Yue-ting CHEN,Qi LI. Attention Res-Unet: an efficient shadow detection algorithm[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(2): 373-381.
[13] Jing-chang WANG,Ling CHEN,Shan-shan YU,Chen-shu JIANG,Yong WU. Multi-factor perceived short-term tourist number prediction model based on gated recurrent unit[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(12): 2357-2364.
[14] Hao LI,Wen-jie ZHAO,Bo HAN. Convolutional neural network acceleration algorithm based on filters pruning[J]. Journal of ZheJiang University (Engineering Science), 2019, 53(10): 1994-2002.
[15] ZHOU Jia-li, CHEN Yi-jun, WU Min. Image acquisition and preprocessing method based on FPGA monitor[J]. Journal of ZheJiang University (Engineering Science), 2018, 52(2): 398-405.