Michael C. Burl
ABSTRACT
Physics-based simulation codes are widely used in science and engineering to model complex systems that would be infeasible to study otherwise. Such codes provide the highest-fidelity representation of system behavior, but are often so slow to run that insight into the system is limited. For example, conducting an exhaustive sweep over a d-dimensional input parameter space with k-steps along each dimension requires kd simulation trials (translating into kd CPU-days for one of our current simulations). An alternative is directed exploration in which the next simulation trials are cleverly chosen at each step. Given the results of previous trials, supervised learning techniques (SVM, KDE, GP) are applied to build up simplified predictive models of system behavior. These models are then used within an active learning framework to identify the most valuable trials to run next. Several active learning strategies are examined including a recently-proposed information-theoretic approach. Performance is evaluated on a set of thirteen synthetic oracles, which serve as surrogates for the more expensive simulations and enable the experiments to be replicated by other researchers.
- Baram, Y., El-Yaniv, R., & Luz, K. (2004). Online choice of active learning algorithms. J. Machine Learning Research, 5, 255--291. Google Scholar
Digital Library
- Burl, M. C., DeCoste, D., Enke, B. L., Mazzoni, D., Merline, W. J., & Scharenbroich, L. (2006). Automated knowledge discovery from simulators. SIAM Int. Conf. on Data Mining (pp. 82--93).Google ScholarCross Ref
- Chang, C.-C., & Lin, C.-J. (2001). LIBSVM: A library for support vector machines. http://www.csie.ntu.edu.tw/cjlin/libsvm.Google Scholar
- Cohn, D., Atlas, L., & Ladner, R. (1994). Improving generalization with active learning. Machine Learning, 15, 201--221. Google ScholarDigital Library
- Durda, D. D., Bottke, W. F., Enke, B. L., Merline, W. J., Asphaug, E., Richardson, D. C., & Leinhardt, Z. M. (2004). The formation of asteroid satellites in large impacts: results from numerical simulations. Icarus, 170, 243--257.Google ScholarCross Ref
- Gramacy, R. B., Lee, H. K., & Macready, W. G. (2004). Parameter space exploration with gaussian process trees. Int. Conf. on Machine Learning (pp. 353--360). Google ScholarDigital Library
- Guestrin, C., Krause, A., & Singh, A. P. (2005). Near-optimal sensor placements in gaussian processes. Int. Conf. on Machine Learning (pp. 265--272). Google ScholarDigital Library
- Holub, A., Perona, P., & Burl, M. C. (2008). Entropy-based active learning for object recognition. Proc. Online Learning for Classification (pp. 1--8).Google ScholarCross Ref
- Knuth, K., Erner, P., & Frasso, S. (2007). Designing intelligent instruments. MaxEnt, Amer. Inst. of Physics, Conf. Proc. 954 (pp. 203--211).Google Scholar
- Kwakernaak, H., & Sivan, R. (1972). Linear optimal control systems. Wiley-Interscience. Google ScholarDigital Library
- Lindenbaum, M., Markovitch, S., & Rusakov, D. (2004). Selective sampling for nearest neighbor classifiers. Machine Learning, 54, 125--152. Google ScholarDigital Library
- MacKay, D. (1992). Information-Based Objective Functions for Active Data Selection. Neural Computation, 4, 590--604. Google ScholarDigital Library
- Olivetti, E. (2008). Sampling strategies for expensive data. Doctoral dissertation, U. of Trento.Google Scholar
- Pfingsten, T. (2006). Bayesian active learning for sensitivity analysis. Lec. Notes in C.S., 4212, 353--364. Google ScholarDigital Library
- Platt, J. (1999). Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. Adv. in Large Margin Classifiers (pp. 61--74). MIT Press.Google Scholar
- Sacks, J., Welch, W., Mitchell, T., & Wynn, H. (1989). Design and Analysis of Computer Experiments. Statistical Science, 4, 409--423.Google ScholarCross Ref
- Tong, S., & Koller, D. (2001). Support vector machine active learning with applications to text classification. Machine Learning Research, 2, 45--66. Google ScholarDigital Library
- Vapnik, V. (1995). The nature of statistical learning theory. Springer, New York. Google ScholarDigital Library
- Veeramachaneni, S., Olivetti, E., & Avesani, P. (2006). Active sampling for detecting irrelevant features. Int. Conf. on Machine Learning (pp. 961--968). Google ScholarDigital Library
- Zadrozny, B. (2004). Learning and evaluating classifiers under sample selection bias. Int. Conf. on Machine Learning (p. 114). Google ScholarDigital Library
Index Terms
- Active learning for directed exploration of complex systems
Recommendations
Human active learning
NIPS'08: Proceedings of the 21st International Conference on Neural Information Processing SystemsWe investigate a topic at the interface of machine learning and cognitive science. Human active learning, where learners can actively query the world for information, is contrasted with passive learning from random examples. Furthermore, we compare ...
Conversing Learning: Active Learning and Active Social Interaction for Human Supervision in Never-Ending Learning Systems
Advances in Artificial Intelligence – IBERAMIA 2012AbstractThe Machine Learning community have been introduced to NELL (Never-Ending Language Learning), a system able to learn from web and to use its knowledge to keep learning infinitely. The idea of continuously learning from the web brings concerns ...
Clinical Trial Active Learning
BCB '23: Proceedings of the 14th ACM International Conference on Bioinformatics, Computational Biology, and Health InformaticsThis paper presents a novel approach to active learning that takes into account the non-independent and identically distributed (non-i.i.d.) structure of a clinical trial setting. There exists two types of clinical trials: retrospective and ...
Comments