Offline small data-driven evolutionary algorithm based on multi-kernel data synthesis

doi:10.3785/j.issn.1008-973X.2025.02.006

Journal of ZheJiang University (Engineering Science)

2025, Vol. 59

Issue (2): 278-288 DOI: 10.3785/j.issn.1008-973X.2025.02.006

Offline small data-driven evolutionary algorithm based on multi-kernel data synthesis

Erchao LI(

),Yun LIU

College of Electrical and Information Engineering, Lanzhou University of Technology, Lanzhou 730050, China

Download:

HTML

PDF(838KB) HTML
Export: BibTeX | EndNote (RIS)

Abstract

An offline data-driven evolutionary algorithm based on multi-kernel data synthesis (DDEA-MKDS) was proposed to enhance the performance of such algorithms in small data scenarios and weaken the dependence of surrogate model on data set size. The empirical formula and traversal method were used to calculate the optimal number of hidden layer nodes for the offline data set in order to simplify model structure by considering that the surrogate model is prone to overfitting due to small data. Three radial basis networks with different kernel functions were trained to generate synthetic data in order to make up for the lack of data. A part of synthetic data was selected by roulette to combine with original data, and new data set was used to train the surrogate model. The experimental results showed that DDEA-MKDS had good performance under the condition of small data by comparing with five states of the art offline data-driven evolutionary algorithms on six single objective benchmark problems, and its efficiency was significantly better than other algorithms.

Key words： offline data-driven evolutionary algorithm small data surrogate model hidden layer node synthetic data

Received: 13 December 2023 Published: 11 February 2025

CLC:

TP 181

Fund: 国家自然科学基金资助项目（62063019）；甘肃省自然科学基金资助项目（24JRRA173，22JR5RA241）.

	Service
	E-mail this article
	Add to my bookshelf
	Add to citation manager
	E-mail Alert
	RSS
	Articles by authors
	Erchao LI
	Yun LIU

Cite this article:

Erchao LI,Yun LIU. Offline small data-driven evolutionary algorithm based on multi-kernel data synthesis. Journal of ZheJiang University (Engineering Science), 2025, 59(2): 278-288.

URL:

https://www.zjujournals.com/eng/10.3785/j.issn.1008-973X.2025.02.006 OR https://www.zjujournals.com/eng/Y2025/V59/I2/278

基于多核数据合成的离线小数据驱动的进化算法

为了增强离线数据驱动的进化算法在小数据情景中的表现, 削弱代理模型对数据集规模的依赖, 提出基于多核数据合成的离线小数据驱动的进化算法(DDEA-MKDS). 考虑到代理模型易因小数据陷入过拟合, 通过经验公式与遍历法找出针对离线数据集的最优隐含层节点数，以简化模型结构. 为了弥补数据量的不足, 训练了3个不同核函数的径向基网络生成合成数据, 通过轮盘赌法选择其中的部分数据与原数据集合并, 使用新数据集训练代理模型. 将DDEA-MKDS与其他5种流行的离线数据驱动的进化算法在6个单目标基准测试问题上进行对比, 实验结果表明, 所提算法在数据量极小的条件下能够取得良好的效果, 寻优效率显著优于其他算法.

关键词： 离线数据驱动, 进化算法, 小数据, 代理模型, 隐含层节点, 合成数据

Fig.1 Structure of radial basis function network

Fig.2 Flow chart of DDEA-MKDS

问题	d	DDEA-MKDS	TT-DDEA	SRK-DDEA	CC-DDEA	CL-DDEA	DDEA-SE
Ellipsoid	10	0.79±0.73	4.52±6.36(+)	1.91±2.52(≈)	1.91±1.12(+)	114.41±97.15(+)	2.81±1.38(+)
	30	5.58±2.99	18.68±14.99(+)	11.40±11.59(+)	9.05±5.60(+)	451.17±394.59(+)	48.57±21.00(+)
	50	27.88±9.13	110.19±84.30(+)	23.99±14.62(?)	32.19±12.61(+)	1181.39±1035.79(+)	195.13±57.46(+)
	100	252.72±52.58	1380.95±1237.13(+)	269.84±54.97(≈)	49.60±22.66(?)	4908.28±5817.24(+)	1399.45±377.21(+)
Rosenbrock	10	16.22±7.00	20.84±10.33(≈)	23.90±12.02(+)	31.33±15.88(+)	941.78±796.76(+)	31.81±14.69(+)
	30	39.82±9.80	57.50±25.14(+)	65.48±29.53(+)	61.12±16.51(+)	1170.28±1082.81(+)	80.95±28.13(+)
	50	69.24±18.95	125.76±37.03(+)	85.74±20.65(≈)	104.91±51.58(+)	1298.44±1265.39(+)	217.48±61.36(+)
	100	209.61±36.64	373.37±68.49(+)	189.10±24.13(≈)	178.94±42.90(≈)	1907.06±1789.96(+)	714.23±121.78(+)
Ackley	10	5.78±1.73	9.71±4.75(+)	6.42±1.90(≈)	7.57±2.24(+)	10.27±4.71(≈)	6.35±1.63(≈)
	30	4.25±0.69	6.91±1.98(+)	6.10±2.17(+)	8.19±4.71(+)	16.19±2.45(+)	8.96±1.35(+)
	50	4.73±0.47	7.63±0.81(+)	5.94±1.65(≈)	5.73±1.98(≈)	11.51±3.09(+)	10.25±0.86(+)
	100	7.20±0.57	11.02±2.01(+)	7.28±0.60(≈)	4.95±0.85(?)	13.01±4.70(+)	11.59±0.52(+)
Levy	10	1.60±0.36	3.19±3.00(≈)	2.30±1.22(≈)	2.48±2.42(≈)	18.56±9.85(+)	2.95±2.53(≈)
	30	3.57±0.76	10.18±10.03(+)	4.39±1.33(≈)	4.02±0.87(≈)	51.71±39.59(+)	8.62±3.44(+)
	50	5.94±0.77	16.57±11.72(+)	6.01±1.06(≈)	10.30±4.54(+)	109.68±64.49(+)	20.91±6.25(+)
	100	13.70±3.59	106.97±66.70(+)	13.88±3.64(≈)	11.03±1.97(≈)	154.60±79.65(+)	73.42±17.79(+)
Griewank	10	1.21±0.28	3.51±3.64(+)	1.12±0.11(≈)	1.19±0.21(≈)	26.32±25.28(+)	2.26±0.54(+)
	30	3.32±1.00	5.72±3.12(+)	1.47±0.14(?)	1.70±0.33(?)	96.19±122.65(+)	12.34±4.19(+)
	50	6.24±1.81	9.23±6.83(≈)	3.04±0.51(?)	2.21±0.40(?)	136.93±114.01(+)	22.87±6.96(+)
	100	21.39±4.12	61.68±30.82(+)	18.56±3.96(≈)	3.61±0.74(?)	148.10±168.05(+)	90.66±20.69(+)
Rastrigin	10	42.23±26.46	58.47±28.90(≈)	64.47±35.35(+)	89.87±37.44(+)	138.64±34.53(+)	76.92±26.52(≈)
	30	95.41±61.47	242.85±162.46(+)	152.73±104.34(≈)	195.63±60.41(+)	322.84±68.42(+)	219.82±36.95(+)
	50	184.75±61.41	397.02±89.28(+)	245.34±101.93(+)	280.60±89.96(+)	570.56±47.71(+)	431.38±57.44(+)
	100	692.34±122.68	1061.33±190.85(+)	697.66±128.78(≈)	319.72±271.23(?)	1307.19±388.38(+)	953.94±51.30(+)
+/≈/?		NA	20/4/0	6/15/3	12/6/6	23/1/0	21/3/0
Friedman Rank		1.63	4.04	2.40	2.40	6.00	4.54

Tab.1 Optimization result of DDEA-MKDS and other comparison algorithms on six test problems

Fig.3 Convergence curves of DDEA-MKDS and other comparison algorithms on Ellipsoid and Rastrigin

Fig.4 Average running time of DDEA-MKDS and other comparison algorithms on all problems

Tab.2 Optimization result of DDEA-MKDS and other various algorithm on Ellipsoid and Rastrigin

Fig.5 Average running time of DDEA-MKDS and other various algorithm on all problems

Tab.3 Optimization result of DDEA-MKDS with different kernel function on Ellipsoid and Rastrigin

Tab.4 Average running time of DDEA-MKDS with different number of kernel function

Tab.5 Optimization result of DDEA-MKDS on Ellipsoid and Rastrigin when d_m takes different values

Fig.6 Variation trend of optimization result of DDEA-MKDS when d_m takes different values


[1]	JIN Y, WANG H, CHUGH T, et al Data-driven evolutionary optimization: an overview and case studies[J]. IEEE Transactions on Evolutionary Computation, 2019, 23 (3): 442- 458 doi: 10.1109/TEVC.2018.2869001

[2]	CHEN R, HE C, JIN Y, et al Model-based evolutionary algorithms: a short survey[J]. Complex and Intelligent Systems, 2018, 4 (4): 283- 292 doi: 10.1007/s40747-018-0080-1

[3]	HE C, TIAN Y, WANG H, et al A repository of real-world datasets for data-driven evolutionary multiobjective optimization[J]. Complex and Intelligent Systems, 2020, 6 (1): 189- 197 doi: 10.1007/s40747-019-00126-2

[4]	WANG H, JIN Y, JANSEN J O Data-driven surrogate-assisted multiobjective evolutionary optimization of a trauma system[J]. IEEE Transactions on Evolutionary Computation, 2016, 20 (6): 939- 952 doi: 10.1109/TEVC.2016.2555315

[5]	GUO D, CHAI T, DING J, et al. Small data driven evolutionary multi-objective optimization of fused magnesium furnaces [C]// IEEE Symposium Series on Computational Intelligence . Athens: IEEE, 2016: 1-8.

[6]	黄鹏飞. 离线数据驱动的进化优化研究[D]. 西安: 西安电子科技大学, 2021: 15-18, 46-47. HUANG Pengfei. A study on offline data-driven evolutionary optimization [D]. Xi’an: Xidian University, 2021: 15-18, 46-47.

[7]	HUANG P, WANG H, MA W. Stochastic ranking for offline data-driven evolutionary optimization using radial basis function networks with multiple kernels [C]// IEEE Symposium Series on Computational Intelligence . Xiamen: IEEE, 2019: 2050-2057.

[8]	CHENG R, JIN Y, NARUKAWA K, et al A multiobjective evolutionary algorithm using gaussian process-based inverse modeling[J]. IEEE Transactions on Evolutionary Computation, 2015, 19 (6): 838- 856 doi: 10.1109/TEVC.2015.2395073

[9]	梁正平, 黄锡均, 李燊钿, 等基于剪枝堆栈泛化的离线数据驱动进化优化[J]. 自动化学报, 2023, 49 (6): 1306- 1325 LIANG Zhengping, HUANG Xijun, LI Shentian, et al Offline data driven evolutionary optimization based on pruning stacked generalization[J]. Acta Automatica Sinica, 2023, 49 (6): 1306- 1325

[10]	ZHOU Z, ONG Y S, NGUYEN M H, et al. A study on polynomial regression and gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm [C]// IEEE Congress on Evolutionary Computation . Edinburgh: IEEE, 2005: 2832-2839.

[11]	HUANG P, WANG H, JIN Y Offline data-driven evolutionary optimization based on tri-training[J]. Swarm and Evolutionary Computation, 2021, 60: 100800 doi: 10.1016/j.swevo.2020.100800

[12]	CHUGH T, CHAKRABORTI N, SINDHYA K, et al A data-driven surrogate-assisted evolutionary algorithm applied to a many-objective blast furnace optimization problem[J]. Materials and Manufacturing Processes, 2017, 32 (10): 1172- 1178 doi: 10.1080/10426914.2016.1269923

[13]	LI J, ZHAN Z, ZHANG J Evolutionary computation for expensive optimization: a survey[J]. Machine Intelligence Research, 2022, 19 (1): 3- 23 doi: 10.1007/s11633-022-1317-4

[14]	CHUGH T, JIN Y, MIETTINEN K, et al A surrogate-assisted reference vector guided evolutionary algorithm for computationally expensive many-objective optimization[J]. IEEE Transactions on Evolutionary Computation, 2016, 22 (1): 129- 142

[15]	LIM D, ONG Y S, JIN Y, et al. A study on metamodeling techniques, ensembles, and multi-surrogates in evolutionary computation [C]// Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation . London: ACM, 2007: 1288-1295.

[16]	MEY A, LOOG M Improved generalization in semi-supervised learning: a survey of theoretical results[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45 (4): 4747- 4767 doi: 10.1109/TPAMI.2022.3198175

[17]	GÖNEN M, ALPAYDIN E Multiple kernel learning algorithms[J]. The Journal of Machine Learning Research, 2011, 12 (7): 2211- 2268

[18]	MA X, LI X, ZHANG Q, et al A survey on cooperative co-evolutionary algorithms[J]. IEEE Transactions on Evolutionary Computation, 2019, 23 (3): 421- 441 doi: 10.1109/TEVC.2018.2868770

[19]	WANG H, JIN Y, SUN C, et al Offline data-driven evolutionary optimization using selective surrogate ensembles[J]. IEEE Transactions on Evolutionary Computation, 2018, 23 (2): 203- 216

[20]	ZHOU Z, LI M Tri-training: exploiting unlabeled data using three classifiers[J]. IEEE Transactions on Knowledge and Data Engineering, 2005, 17 (11): 1529- 1541 doi: 10.1109/TKDE.2005.186

[21]	GOSAIN A, SACHDEVA K Materialized view selection for query performance enhancement using stochastic ranking based cuckoo search algorithm[J]. International Journal of Reliability, Quality and Safety Engineering, 2020, 27 (3): 2050008 doi: 10.1142/S0218539320500084

[22]	GONG Y, ZHONG Y, HUANG H. Offline data-driven optimization at scale: a cooperative coevolutionary approach [EB/OL]. (2023-12-04)[2024-01-30]. https://doi.org/10.1109/TEVC.2023.3338693.

[23]	HUANG H, GONG Y Contrastive learning: an alternative surrogate for offline data-driven evolutionary computation[J]. IEEE Transactions on Evolutionary Computation, 2023, 27 (2): 370- 384 doi: 10.1109/TEVC.2022.3170638

[24]	WANG R, ZHU F, ZHANG X, et al Training with scaled logits to alleviate class-level over-fitting in few-shot learning[J]. Neurocomputing, 2023, 522: 142- 151 doi: 10.1016/j.neucom.2022.12.011

[25]	ROHLOFF C T, KOHLI N, CHUNG S The impact of functional form complexity on model overfitting for nonlinear mixed-effects models[J]. Multivariate Behavioral Research, 2023, 58 (4): 723- 742 doi: 10.1080/00273171.2022.2119360

[26]	RASTEGAR R, HARIRI A A step forward in studying the compact genetic algorithm[J]. Evolutionary Computation, 2006, 14 (3): 277- 289 doi: 10.1162/evco.2006.14.3.277

[27]	王嵘冰, 徐红艳, 李波, 等 BP神经网络隐含层节点数确定方法研究[J]. 计算机技术与发展, 2018, 28 (4): 31- 35 WANG Rongbing, XU Hongyan, LI Bo, et al Research on method of determining hidden layer nodes in BP neural network[J]. Computer Technology and Development, 2018, 28 (4): 31- 35 doi: 10.3969/j.issn.1673-629X.2018.04.007

[28]	MAO Y, LIU C, XIAO D, et al Study of the magnetic properties of haematite based on spectroscopy and the IPSO-ELM neural network[J]. Journal of Sensors, 2018, 2018 (1): 1- 9

[29]	MIN M, CHEN X, LEI Y, et al A novel kernel-based extreme learning machine with incremental hidden layer nodes[J]. IFAC PapersOnLine, 2020, 53 (2): 11836- 11841 doi: 10.1016/j.ifacol.2020.12.695

[30]	TAO L, CAO T, WANG Q, et al Distribution adaptation and classification framework based on multiple kernel learning for motor imagery BCI illiteracy[J]. Sensors, 2022, 22 (17): 6572 doi: 10.3390/s22176572

[31]	PRICE S R, ANDERSEN D T, HAVENS T C, et al Kernel matrix-based heuristic multiple kernel learning[J]. Mathematics, 2022, 10 (12): 2026 doi: 10.3390/math10122026

[1]	Fei WU,Jiacheng CHEN,Wanliang WANG. Review on computational intelligence based on parallel computing[J]. Journal of ZheJiang University (Engineering Science), 2025, 59(1): 27-38.

[2]	Bo-ping YU,Gao-hua LI,Liang XIE,Fu-xin WANG. Dynamic stall optimization design of rotor airfoil based on surrogate model[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(4): 833-842.

[3]	ZHANG Xuan-wu, ZHENG Yao, YANG Bo-wei, ZHANG Ji-fa. Aerodynamic optimization design of airfoil configurations based on cascade feedforward neural network[J]. Journal of ZheJiang University (Engineering Science), 2017, 51(7): 1405-1411.

[4]	GUO Xiao fang, WANG Yu ping, DAI Cai. New hybrid decomposition many-objective evolutionary algorithm[J]. Journal of ZheJiang University (Engineering Science), 2016, 50(7): 1313-1321.

[5]	WANG Yun, FENG Yi-xiong, TAN Jian-rong, GAO Yi-cong. Multi-objective optimization method of flexible job-shop lot-splitting scheduling[J]. Journal of ZheJiang University (Engineering Science), 2011, 45(4): 719-726.

[6]	ZHAO Peng, FU Jian-zhong, LI Yang, CUI Shu-biao. Iterative optimization method for injection parameters based on surrogate model[J]. Journal of ZheJiang University (Engineering Science), 2011, 45(2): 197-200.

[7]	LIU Lin, SHU Xiao-Chi, HU Jun-Jiang, et al. Modification on non-dominated sorting genetic algorithm used for air powered engine design[J]. Journal of ZheJiang University (Engineering Science), 2009, 43(5): 907-910.

Viewed

Full text

Abstract

Cited

Shared

Discussed