AHPS²: An optimizer using adaptive heterogeneous particle swarms

Highlights

• Multiple heterogeneous swarms with each consisting a group of homogeneous particles are modeled.

• Nature’s law, “survival of the fittest”, is simulated by proposing the adaptive competition strategy with two models.

• Heterogeneous learning based on two complementary techniques are theoretically analyzed and experimentally tested.

• Adaptive competition strategy with immigration model is more effective in practice.

• The proposed method shows a better or comparable performance than the comparison algorithms over the numerical experiments.

Abstract

Particle swarm optimization (PSO) has suffered from premature convergence and lacked diversity for complex problems since its inception. An emerging advancement in PSO is multi-swarm PSO (MS-PSO) which is designed to increase the diversity of swarms. However, most MS-PSOs were developed for particular problems so their search capability on diverse landscapes is still less than satisfactory. Moreover, research on MS-PSO has so far treated the sub-swarms as cooperative groups with minimum competition (if not none). In addition, the size of each sub-swarm is set to be fixed which may encounter excessive computational cost. To address these issues, a novel optimizer using Adaptive Heterogeneous Particle SwarmS (AHPS²) is developed in this research. In AHPS², multiple heterogeneous swarms, each consisting of a group of homogenous particles having similar learning strategy, are introduced. Two complementary search techniques, comprehensive learning and a subgradient method, are studied. To best take advantage of the heterogeneous learning strategies, an adaptive competition strategy is proposed so the size of each swarm can be dynamically adjusted based on its group performance. The analyses of the swarm heterogeneity and the competition models are presented to validate the effectiveness. Furthermore, comparisons between AHPS² and state-of-the-art algorithms are grouped into three categories: 36 regular benchmark functions (30-dimensional), 20 large-scale benchmark functions (1000-dimensional) and 3 real-world problems. Experimental results show that AHPS² displays a better or comparable performance compared to the other swarm-based or evolutionary algorithms in terms of solution accuracy and statistical tests.

Introduction

Inspired by the natural behavior of fish schooling and bird flocking, Kennedy and Eberhart first proposed particle swarm optimization (PSO), a novel stochastic optimizer, in 1995 [28]. In PSO, a population of individuals, named as particles, is employed to explore the hyperspace and search for the global solution in a collaborative manner. The searching movement of each particle is guided by its best historical position in conjunction with the populations’ best historical position. Compared with evolutionary algorithms such as artificial bee colony, differential evolution and genetic algorithm, PSO has shown similar or even superior performance for both unimodal and multimodal optimization problems in terms of solution accuracy and computational efficiency [13]. As a result, PSO has been applied to various real-world problems including power systems, engineering optimization, and portfolio management [19], [40], [61], just to name a few.

The promise of PSO motivates researchers to further improve its performance. The particle velocity update formulation is modified to improve solution accuracy in [11], [52], [55]; different population topologies are studied to improve the information sharing in [29], [39]; new learning strategies for the particles are designed to avoid premature convergence in [4], [22], [23], [34], [37], [53], [54], [60]. Among these efforts, it is noted most PSO variants are developed to tackle one specific type of problem (e.g., unimodal or multimodal). Given this, the resulting performance on other types of problems is not guaranteed. For example, the PSO with comprehensive learning strategy (CLPSO) [34] performs well on complex multi-modal problems but converges slow on unimodal problems. The PSO obtaining a high convergence rate for unimodal problems tends to get trapped at local optima which limits its application to multi-modal problems [11]. To address the balance between the exploration and exploitation, a recent emerging research field is to increase the diversity of the population by introducing sub-swarms into PSO and has resulted in multi-swarm PSO (MS-PSO) [2], [15], [32], [33], [35], [36], [46], [47], [48], [49], [51], [70], [71], [73]. Compared to single-swarm PSO, MS-PSO has demonstrated the ability to explore the search space more thoroughly and is less likely to be trapped at local optima [71].

Research on MS-PSO mainly lies in two distinct directions: (1) multiple swarms with homogenous particles (same learning strategy, topologies). This is an extension of single-swarm PSO by deploying multiple swarms to the search space aiming to increase the diversity where each swarm may have different properties [2], [36], [49], [71], [73] and (2) each swarm may consist of different types of particles, known as heterogeneous swarms [7], [17], [18], [20], [31], [43], [45]. Research from both directions treats the swarms as cooperative groups with minimum competition (if not none) [35], [36], [49], [70], [72]. Furthermore, most MS-PSO variants are incapable of handling a diverse set of problems [35], [47], [48], [51], [70], [71], [72]. It is also worth pointing out that a majority of existing research treats the population size of each swarm fixed over the evolution process, which may not be computationally efficient and lead to a low convergence speed [7], [17], [32], [43].

In this research, we introduce an adaptive learning among competitive heterogeneous swarms, termed Adaptive Heterogeneous Particle SwarmS (AHPS²). There exist multiple heterogeneous swarms with each consisting of a group of homogenous particles having similar learning strategy. The adaptive competitions at the swarm level will trigger the size of the swarms to be dynamically adjusted based on the group performances. To comprehensively evaluate the performance of the proposed algorithm, AHPS² is compared with other state-of-the-art algorithms on three categories of experiments: (1) 36 30-dimensional benchmark problems with various properties, such as unimodal, multimodal, shifted, rotated, non-separable and ill-conditioned functions, are employed to test AHPS²’s performance for a diverse set of problems with common dimensionalities; (2) a set of large-scale optimization problems, i.e., CEC’2010 1000-dimensional testbed including 20 benchmark functions, are used to justify AHPS²’s scalability for high dimensional problems; (3) 3 real-world problems are employed to benchmark AHPS²’s applicability for practical problems. The numerical results demonstrate that AHPS² significantly improves the performance of PSO and outperforms most of the comparison algorithms during the experiments.

The rest of the paper is organized as follows. Section 2 provides an overview of PSO and multi-swarm variants. Section 3 introduces the proposed AHPS². Section 4 provides the analysis of two types of learning strategies to be implemented by the swarms followed by the validation experiments on the competition strategies. Comprehensive experiments are presented by Section 5. Finally, the conclusion is drawn in Section 6.