An analytical framework for high-speed hardware particle swarm optimization

https://doi.org/10.1016/j.micpro.2019.102949Get rights and content

Highlights

  • Develop a high-speed hardware core for PSO on FPGAs.

  • Develop a statistical framework that combines the characteristics of optimization algorithms and their implementations.

  • Validate the framework by studying a set of targeted benchmark evaluation functions.

  • Classify, rank, and evaluate PSO implementations according to combined indicators.

  • Identify the type of problems that can be efficiently optimized using the PSO based on heterogeneous characteristics.

Abstract

Engineering optimization techniques are computationally intensive and can challenge implementations on tightly-constrained embedded systems. Particle Swarm Optimization (PSO) is a well-known bio-inspired algorithm that is adopted in various applications, such as, transportation, robotics, energy, etc. In this paper, a high-speed PSO hardware processor is developed with focus on outperforming similar state-of-the-art implementations. In addition, the investigation comprises the development of an analytical framework that captures wide characteristics of optimization algorithm implementations, in hardware and software, using key simple and combined heterogeneous indicators. The framework proposes a combined Optimization Fitness Indicator that can classify the performance of PSO implementations when targeting different evaluation functions. The two targeted processing systems are Field Programmable Gate Arrays for hardware implementations and a high-end multi-core computer for software implementations. The investigation confirms the successful development of a PSO processor with appealing performance characteristics that outperforms recently presented implementations. The proposed hardware implementation attains 23,300 improvement ratio of execution times with an elliptic evaluation function. In addition, a speedup of 1777 times is achieved with a Shifted Schwefels function. Indeed, the developed framework successfully classifies PSO implementations according to multiple and heterogeneous properties for a variety of benchmark functions.

Introduction

Optimization is an important concept in the engineering domain [1], [2], [3], [4]. Indeed, all engineering applications involve some sort of optimization in order to realize the final product. Optimization involves reducing cost, power consumption, time delay or increases the yield, profit, quality of solution, etc. As engineering problems became more challenging; with properties such as large size, discontinuity, non-differentiability, non-linearity, multi-objectiveness, and mixed variable types; it becomes essential to develop new optimization paradigms that can reach acceptable solutions in less time.

Meta-heuristics is a substantial and considerably important optimization paradigm that can be used to tackle nowadays engineering problems. A major class of meta-heuristics is Population-based algorithms, which update a population of solutions over a number of iterations until some stopping condition is satisfied. Population-based algorithms are categorized based on the inspiration behind their population update scheme. These categories include Evolutionary Algorithms and Swarm Intelligence algorithms. Due to the heavy computational workload constituting of optimization with population-based algorithms, computational speed matters for solving many dynamic and real-time problems with Evolutionary Algorithms and Swarm Intelligence algorithms. For example, radio resource scheduling for recent generations of wireless communication networks requires a decision to be made within nano-seconds level, which is not viable with conventional computing infrastructures [5], [6]. Further research has been conducted over the past two decades in order to speed up the execution of such algorithms, whether by manipulating parameters, developing multiple/distributed versions or implementing them on specific hardware.

In this paper, we investigate the implementation of one Swarm Intelligence algorithm, namely Particle Swarm Optimization (PSO) [7], [8], on Field Programmable Gate Arrays (FPGAs) [9], [10], [11], [12]. The investigation benefits from the advancements in FPGA capabilities and aims at developing high-speed hardware cores. Such a hardware implementation enables embedding optimization algorithms in application to assist, or completely replace, a central processing component. The investigation is driven by the need for appealing performance characteristics, and the reliable operation within real-time applications, that hardware implementations can provide. Furthermore, the investigation comprises the development of a statistical analysis framework that captures wide characteristics of optimization computations. The proposed framework statistically combines the mathematical properties of optimization algorithms and their evaluation function complexities, besides, their implementations in HW and SW. To that end, the developed PSO implementations are employed to validate the proposed framework and verify its effectiveness in application. Indeed, limited work in the literature is found to analyze the performance of PSO implementations based on heterogeneous properties as in the proposed analysis framework. The general contribution of the paper is summarized as follows; a detailed discussion on the motivation, contributions, and objectives is presented in Section 2:

  • Develop efficient hardware cores for the PSO algorithm on FPGAs.

  • Develop an analytical framework that evaluates the fitness of optimization implementations based on heterogeneous performance indicators.

The rest of the paper is organized so that Section 2 present the motivation, research questions, and the paper contribution. In Section 3, a survey of the literature is presented. In Section 4, the processor design and implementation are presented. Section 5 introduces the statistical analysis model. A thorough performance evaluation is presented in Section 6. Section 7 concludes the paper and sets the ground for future work.

Section snippets

Research objectives

Challenges to optimization algorithms, including PSO, comprise performance characteristics (time, speed, efficiency, and complexity), storage requirements, reliability and accuracy, and first and foremost the dealing with the intrinsic sequential behavior of the algorithm. As related to PSO, the following research opportunities are highlighted:

  • Identify and investigate the performance aspects of SW implementations of PSO while targeting high-performance multi-core processors.

  • Develop efficient

Related work

The emergence of high-performance FPGAs enabled their use in computationally-intensive applications, such as optimization. Many recent investigations are identified in the literature to target implementing the PSO algorithm on FPGAs. In this section, state-of-the-art related work is identified and presented, while closely-related work is thoroughly addressed in Section 6.2. In presenting similar work, we focus on a variety of implementation-specific aspects, such as, target devices, the used

Hardware design

An informal and systematic approach is adopted to develop hardware cores for the PSO algorithm [1], [30]. The approach is informal in the sense that it does not rely on engineering formal methods [31]. In addition, the approach is systematic in the sense that its procedure can be reused to develop similar hardware solutions, however, the method is not yet automatic and does not include any code generations, compilations, or rapid prototyping of hardware circuits. Furthermore, the methodology is

Analytical model development

To present the proposed performance analysis model, we adopt the Generic Benchmark Model (GBM) of Damaj et al. [34]. The GBM comprises six elements that define the Goal, Inputs, Activities, Output, Outcomes, and the desired Performance profile of the performance analysis framework. The model captures the relationships among the resources, implementation, mathematical formulation, and the obtained results. The Goal defines the aim of the analysis framework. Moreover, the Input identifies the

Analysis and evaluation

The analysis of the developed PSO implementations are produced to serve for several evaluation purposes. Important implementation aspects are presented in Section 6.1. In addition, a set of thirteen BEFs, namely F1 through F13, is targeted to perfectly match the test-cases of the work presented in [13] and enable comparisons with similar work. To that end, the HW analysis is presented in Section 6.2 with comparisons to similar findings from closely-related work in the literature. A second set

Conclusion

Optimization is a key approach in engineering that enables effective solutions. PSO is a current and widely used heuristic; it is well-known for its effectiveness in application. In this paper, an on-chip PSO implementation is developed and mapped onto an FPGA. The developed processor significantly outperform the partitioned HW and SW implementations of [13] that target the same development board, FPGA, BEFs, and the execution parameters. Our proposed HW implementation achieves an EThw

Declaration of Competing Interest

Authors declare that they have no conflict of interest.

Issam Damaj Ph.D ME BE SMIEEE MASEE, is an Associate Professor of Electrical and Computer Engineering (ECE) at Beirut Arab University (BAU). At BAU, he is the Director of the Center for Quality Assurance. Before joining BAU, he spent 13 years in professorial ranks in higher education institutions in Kuwait (American University of Kuwait, AUK, 10 years) and Oman (Dhofar University, DU, 3 years). During his tenure, he published 69 technical papers and 9 book chapters-in addition to various short

References (38)

  • B. Shackleford et al.

    FPGA implementation of neighborhood-of-four cellular automata random number generators

    Proceedings of the 2002 ACM/SIGDA Tenth International Symposium on Field-programmable Gate Arrays

    (2002)
  • J.L. Hennessy et al.

    Computer Architecture: A Quantitative Approach

    (2011)
  • S.J. Kasbah et al.

    The Jacobi method in reconfigurable hardware

    World Congress on Engineering

    (2007)
  • M.E. Aydin et al.

    Multiuser scheduling on the LTE downlink with meta-heuristic approaches

    Phys. Commun.

    (2013)
  • J.L. Awange et al.

    Particle swarm optimization

    Mathematical Geosciences

    (2018)
  • I. Damaj et al.

    Mapping high-level algorithms onto massively parallel reconfigurable hardware

    IEEE International Conference of Computer Systems and Applications

    (2003)
  • S.M. Trimberger

    Three ages of FPGAs: A retrospective on the first thirty years of FPGA technology

    Proc. IEEE

    (2015)
  • J. de Fine Licht et al.

    Designing scalable FPGA architectures using high-level synthesis

    Proceedings of the 23rd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming

    (2018)
  • X. Zou et al.

    Parallel design of intelligent optimization algorithm based on FPGA

    Int. J. Adv. Manuf. Technol.

    (2018)
  • Cited by (0)

    Issam Damaj Ph.D ME BE SMIEEE MASEE, is an Associate Professor of Electrical and Computer Engineering (ECE) at Beirut Arab University (BAU). At BAU, he is the Director of the Center for Quality Assurance. Before joining BAU, he spent 13 years in professorial ranks in higher education institutions in Kuwait (American University of Kuwait, AUK, 10 years) and Oman (Dhofar University, DU, 3 years). During his tenure, he published 69 technical papers and 9 book chapters-in addition to various short papers and technical reports. His research interests include hardware design, smart cities, vehicular technology, and engineering education. He is an associate editor and a reviewer with publishers that include IEEE, Elsevier, and Springer. In addition, he is the recipient of various awards in mentoring, service, research, and academic high distinction. Dr. Damaj is a senior member of the IEEE. He maintains an academic website at https://www.idamaj.net.

    Mohamed Elshafei, ME BE, is a Ph.D. student in Software Engineering and a research assistant at Data-driven Analysis of Software Laboratory, Concordia University, Quebec, Canada. He received a Bachelor of Engineering in Computer Engineering from American University of Kuwait in 2013. In 2016, he received a Master of Science in Computer Engineering from Kuwait University. His search interests include artificial intelligence and machine learning.

    Mohammed El-Abd, Ph.D. ME BE SMIEEE, is an Associate Professor of Computer Engineering in the ECE Department at the American University of Kuwait (AUK). Dr. El-Abd has over 50 publications on the form of journal articles, book chapters, conference papers, and abstracts. His research interests include meta-heuristics, swarm and evolutionary intelligence, cooperative search, continuous and discrete optimization, large-scale optimization, robotics, and engineering education.

    Mehmet Emin Aydin, Ph.D. ME BE, is a Senior Lecturer in Computer Science at the Computer Science and Creative, University of West England, Bristol, UK. Prior to this post, he worked in academic and research positions for various universities including University of Bedfordshire, London South Bank University and University of Aberdeen. He is an editorial board member of a number of international peer-reviewed journals and have been serving as committee member of various international conferences. His research interests include parallel and distributed metaheuristics, wired/wireless network planning and optimization, combinatorial optimization, evolutionary computation and intelligent agents and multi agent systems.

    View full text