Abstract
PDF (216 K)
E-mail Article
Add to my Quick Links
Cited By in Scopus (5)
Purchase PDF (394 K)
doi:10.1016/S0890-5401(03)00135-4 
Copyright © 2003 Elsevier Inc. All rights reserved.
Uniform-distribution attribute noise learnability
Nader H. Bshouty
, a, Jeffrey C. Jackson, b and Christino Tamon
, , c
Received 13 March 2002;
Abstract
We study the problem of PAC-learning Boolean functions with random attribute noise under the uniform distribution. We define a noisy distance measure for function classes and show that if this measure is small for a class
and an attribute noise distribution D then
is not learnable with respect to the uniform distribution in the presence of noise generated according to D. The noisy distance measure is then characterized in terms of Fourier properties of the function class. We use this characterization to show that the class of all parity functions is not learnable for any but very concentrated noise distributions D. On the other hand, we show that if
is learnable with respect to uniform using a standard Fourier-based learning technique, then
is learnable with time and sample complexity also determined by the noisy distance. In fact, we show that this style algorithm is nearly the best possible for learning in the presence of attribute noise. As an application of our results, we show how to extend such an algorithm for learning AC0 so that it handles certain types of attribute noise with relatively little impact on the running time.
Author Keywords: Computational learning theory; Learning with noise; Fourier analysis
References
[1]. A. Blum, C. Burch, J. Langford, On learning monotone Boolean functions, in: Proc. 39th Ann. IEEE Symp. on Foundations of Computer Science, 1998, pp. 408–415
[2]. I. Benjamini, G. Kalai and O. Schramm, Noise sensitivity of Boolean functions and applications to percolation. Inst. Hautes Études Sci. Publ. Math. 90 (1999), pp. 5–43.
[3]. S. Decatur, R. Gennaro, On Learning from noisy and incomplete examples, in: Proc. 8th Ann. ACM Conf. on Computational Learning Theory, 1995, pp. 353–360
[4]. S. Goldman and S. Sloan, Can PAC learning algorithms tolerate random attribute noise?. Algorithmica 14 1 (1995), pp. 70–84. View Record in Scopus | Cited By in Scopus (9)
[5]. J. Håstad, Computational Limitations for Small-Depth Circuits. , The MIT Press, Cambridge, MA (1987).
[6]. J. Kahn, G. Kalai, N. Linial, The influence of variables on Boolean functions, in: Proc. 29th Ann. Symp. on Foundations of Computer Science, 1988, pp. 68–80
[7]. N. Linial, Y. Mansour and N. Nisan, Constant depth circuits, AC0 circuits, and learnability. J. ACM 40 3 (1993), pp. 607–620. View Record in Scopus | Cited By in Scopus (110)
[8]. G. Shackelford, D. Volper, Learning k-DNF with noise in the attributes, in: Proc. 1988 Workshop on Computational Learning Theory, 1988, pp. 97–103
[9]. H.U. Simon, General bounds on the number of examples needed for learning probabilistic concepts, in: Proc. 6th Ann. ACM Workshop on Computational Learning Theory, 1993, pp. 402–411
[10]. K. Yang, On learning correlated Boolean functions using statistical queries, in: Proc. 12th Int. Conf. Algorithm Learning Theory, 2001, pp. 59–76
