Distributionally robust binary classifier under Wasserstein distance

Abstract

The robustification of statistical models has been a popular topic for decades. Statistical robustification and robust optimization are the two main approaches in the literature, where the former stabilizes the model output by removing the outlier points while the latter concerns more the outlier points in making the conservative decisions. This thesis develops a novel robust optimization perspective to robustify a class of binary classifiers. Our model considers the worst-case distribution within a pre-determined uncertainty ball that centers at the given benchmark distribution with the radius calculated as per the Wasserstein distance. We derive the tractable formulation for the general problem. When focusing on the support vector machine (SVM), the general problem boils down to an easy-to-solve second- order cone programming problem. The robustified SVM is then applied to synthetic data with and without contamination, and our simulation studies show that our robustified SVM model can outperform the classical SVM and the extreme empirical loss SVM models under many circumstances.