ScienceDirect® Home Skip Main Navigation Links
You have guest access to ScienceDirect. Find out more.
 
Home
Browse
My Settings
Alerts
Help
 Quick Search
 Search tips (Opens new window)
    Clear all fields    
Journal of Computer and System Sciences
Volume 66, Issue 3, May 2003, Pages 496-514
 
Font Size: Decrease Font Size  Increase Font Size
 Abstract - selected
Article
Purchase PDF (211 K)

 
 
 
Related Articles in ScienceDirect
View More Related Articles
 
View Record in Scopus
 
doi:10.1016/S0022-0000(03)00038-2    How to Cite or Link Using DOI (Opens New Window)
Copyright © 2003 Elsevier Science (USA). All rights reserved.

On the difficulty of approximately maximizing agreements*1

Shai Ben-DavidE-mail The Corresponding Author, a, Nadav EironE-mail The Corresponding Author, b and Philip M. LongCorresponding Author Contact Information, E-mail The Corresponding Author, c

a Department of Computer Science, Technion, Haifa 32000, Israel b IBM Almaden Research Center, 650 Harry Road, San Jose, CA 95120, USA c Genome Institute of Singapore, 1 Science Park Road, The Capricorn, #05-01, Singapore 117604, Singapore

Received 29 November 2000; 
revised 25 October 2002. 
Available online 6 May 2003.

Purchase the full-text article



References and further reading may be available for this article. To view references and further reading you must purchase this article.

Abstract

We address the computational complexity of learning in the agnostic framework. For a variety of common concept classes we prove that, unless P=NP, there is no polynomial time approximation scheme for finding a member in the class that approximately maximizes the agreement with a given training sample. In particular our results apply to the classes of monomials, axis-aligned hyper-rectangles, closed balls and monotone monomials. For each of these classes, we prove the NP-hardness of approximating maximal agreement to within some fixed constant (independent of the sample size and of the dimensionality of the sample space). For the class of half-spaces, we prove that, for any var epsilon>0, it is NP-hard to approximately maximize agreements to within a factor of (418/415−var epsilon), improving on the best previously known constant for this problem, and using a simpler proof. An interesting feature of our proofs is that, for each of the classes we discuss, we find patterns of training examples that, while being hard for approximating agreement within that concept class, allow efficient agreement maximization within other concept classes. These results bring up a new aspect of the model selection problem—they imply that the choice of hypothesis class for agnostic learning from among those considered in this paper can drastically effect the computational complexity of the learning process.

Author Keywords: Machine learning; Computational learning theory; Neural networks; Inapproximability; Hardness; Half-spaces; Axis-aligned hyper-rectangles; Balls; Monomials

Article Outline

1. Introduction
2. Preliminaries
3. Main results
4. Monomials and axis-aligned hyper-rectangles
4.1. Gadget construction for monomials
4.2. Analyzing the reduction
4.3. A hardness result for monotone monomials
5. Balls and half-spaces
5.1. Gadget construction for balls and half-spaces
5.2. Analysis of the balls reduction
5.3. Analysis of the half-space reduction
6. Computational aspects of model selection
7. Conclusion
Acknowledgements
References


 
Home
Browse
My Settings
Alerts
Help
Elsevier.com (Opens new window)
About ScienceDirect  |  Contact Us  |  Information for Advertisers  |  Terms & Conditions  |  Privacy Policy
Copyright © 2008 Elsevier B.V. All rights reserved. ScienceDirect® is a registered trademark of Elsevier B.V.