Numerical optimization for the compatibility constant of the lasso

Authors

Kei Hirose

Categories

Abstract

Compatibility condition and compatibility constant have been commonly used to evaluate the prediction error of the lasso when the number of variables exceeds the number of observations. However, the computation of the compatibility constant is generally difficult because it is a complicated nonlinear optimization problem. In this study, we present a numerical approach to compute the compatibility constant when the zero/nonzero pattern of true regression coefficients is given. We show that the optimization problem reduces to a quadratic program (QP) once the signs of the nonzero coefficients are specified. In this case, the compatibility constant can be obtained by solving QPs for all possible sign combinations. We also formulate a mixed-integer quadratic programming (MIQP) approach that can be applied when the number of true nonzero coefficients is moderately large. We investigate the finite-sample behavior of the compatibility constant for simulated data under a wide variety of parameter settings and compare the mean squared error with its theoretical error bound based on the compatibility constant. The behavior of the compatibility constant in finite samples is also investigated through a real data analysis.