Bilevel Model Selection for Support Vector Machines

Published in CRM Proceedings and Lecture Notes, 2008

Recommended citation: G. Kunapuli, K. P. Bennett, J. Hu and J.-S. Pang. Bilevel Model Selection for Support Vector Machines CRM Proceedings and Lecture Notes. Volume 45 (2008), pp. 129-158. American Mathematical Society. Pierre Hansen and Panos Pardolos, Editors.

The successful application of Support Vector Machines (SVMs), kernel methods and other statistical machine learning methods requires selection of model parameters based on estimates of the generalization error. This paper presents a novel approach to systematic model selection through bilevel optimization. We show how modelling tasks for widely used machine learning methods can be formulated as bilevel optimization problems and describe how the approach can address a broad range of tasks — among which are parameter, feature and kernel selection. In addition, we also discuss the challenges in implementing these approaches and enumerate opportunities for future work in this emerging research area.