Abstract

The Estimation of Distribution Algorithm (EDA) model is an optimization procedure through learning and sampling a conditional probabilistic function. The use of conditional density function permits multivariate dependency modelling, which is not captured in a population-based representation, like the classical Genetic Algorithms. The Gaussian model is a simple and widely used model for density estimation. However, an assumption of normality is not realistic for many real-life problems. Alternatively, the maximum-entropy model can be used, which makes no assumption of a normal distribution. One disadvantage of the maximum-entropy model is the learning cost of its parameters. This paper proposes an Adaptive Estimated Maximum-Entropy Distribution (Adaptive MEED) model, which aims to reduce learning complexity of building a model. Adaptive MEED exploits the fact that samples have a low average fitness in the early stage, but they gradually converge to an optima towards the end of the search. Hence, it is not necessary to inference the model with a full account of observed constraints in the early stage of the search. The proposed model attempts to estimate the density function with a dynamic set of samples and active constraints. In addition, the proposed model includes a global sampling function to address the issue of a missing mutation operator. The ergodic convergence properties of the proposed model are discussed with the Markov Chain analysis. The preliminary experimental evaluation shows that the proposed model performs well against genetic algorithms on several clustering problems.

Keywords: Estimation of Distribution Algorithms (EDA); Genetic Algorithms (GA); Adaptive method; Global convergence; Clustering; Data mining

Article Outline

1. Introduction

2. Related work

3. Estimated maximum-entropy distribution method

3.1. Motivations

3.2. Maximum-entropy distribution

3.3. Procedure of the MEED model

3.4. Complexity analysis of MEED

4. The proposed adaptive MEED model

4.1. Adaptive control functions

4.2. The adaptive MEED model

4.3. Complexity analysis of the adaptive MEED

5. Markov chain analysis

6. Applications in data mining

7. Empirical study

8. Conclusion

Acknowledgements

References