Tuning of the hyperparameters for L2-loss SVMs with the RBF kernel by the maximum-margin principle and the jackknife technique

Abstract

The hyperparameters for support vector machines (SVMs) with L2 soft margins and the radial basis function (RBF) kernel include the parameters for the RBF kernel and the L2-soft-margin parameter C. In this paper, the parameters for the RBF kernel are determined through maximization of a margin-based criterion. This criterion is approximately optimized through solving two easier subproblems: one is related to margin maximization in the input space and the other is related to the determination of the extent of sample spread in the feature space. After that, the L2-soft-margin parameter C is obtained by an analytic formula in terms of a jackknife estimate of the perturbation in the eigenvalues of the kernel matrix. In comparison with SVM model selection based on differentiable bounds, such as radius/margin bounds, experimental results on a number of open data sets show that the proposed approach is efficient and accurate.