Sparse and robust geometric twin support vector machine via asymmetric RoBoSS loss function

This paper proposes a novel asymmetric, robust, bounded, sparse and smooth (aR) loss function for l1-norm penalized geometric twin support vector machine (aRSGTSVM) to handle classification and regression tasks. The l1-norm penalty can achieve the feature selection. The proposed aR loss function can not only effectively mitigate the impact of label noise, but also significantly enhance the stability to resampling noise, i.e., the zero-mean feature noise around the boundary hyperplanes. Furthermore, a statistical analysis of the robustness of aRSGTSVM was also conducted using the influence function. Since aRSGTSVM involves nonconvex and nonsmooth optimization, we develop a fast and stable proximal gradient descent based solving algorithm. Compared with related state-of-the-art methods, experimental results demonstrate the superiority of the proposed aRSGTSVM on both synthetic and UCI datasets. Furthermore, we apply aRSGTSVM to index tracking tasks, where results for tracking the different indices in the China stock market show that it can achieve satisfactory performance.

The main contributions are summarized as follows:

1. We propose a new asymmetric and robust (aR) loss function. Especially, compared with RoBoSS loss function, the proposed aR loss function can not only effectively mitigate the impact of label noise, but also enhance the stability to the zero-mean noise near the boundary hyperplanes (so-called resampling noise).

2. We employ influence function to demonstrate the robustness of aR loss function. Although there is extensive literature on enhancing SVM robustness by constructing non-convex functions, few studies theoretically elucidate the robustness of the constructed functions. From a statistical perspective, we further proved that the influence function of aR loss is bounded, thereby providing a theoretical guarantee of its robustness.

3. Based on the aR loss and l1-norm penalty, the aRSGTSVM models are proposed for robust classification and regression learning problems (In the following content, we refer to the regression model as aRSGTSVR to avoid confusion). In addition to robustness, the proposed aRSGTSVM and aRSGTSVR can also reduce the influence of noise features.

4. To optimize the nonconvex and nonsmooth aRSGTSVM and aRSGTSVR optimization problems, we design an efficient algorithm based on the proximal gradient descent algorithm, termed as iPiano algorithm. The implemented algorithm is demonstrated to be fast and stable, and is applicable to high-dimensional learning problems.

5. A lot of numerical studies on synthetic and benchmark datasets demonstrate that the proposed aRSGTSVM and aRSGTSVR are robust against label noise and resampling noise, while maintaining excellent feature selection capability in high-dimensional settings.

6. To further validate the generalization capability, we apply the model into index tracking task. The results show that aRSGTSVR delivers consistently superior performance across different underlying indices.

The proposed aR loss function is defined as:

LaR (u) = λ(1 − (au + 1) exp(−au)), u > 0,

LaR (u) = τλ(1 − (au2 + 1) exp(−au2)), u ≤ 0,

where the shape parameter a > 0, the margin parameter λ > 0, and the parameter τ ∈ [0, 1] control the asymmetry of the loss function, thereby enhancing the model's robustness to resampling noise.

The aR loss function has the following properties:

Property 1. The aR loss function is C1-smooth.

Property 2. The aR loss function is bounded. For u > 0, LaR(u) is strictly monotonically increasing on (0, +∞). For u < 0, LaR(u) is strictly monotonically decreasing on (−∞, 0). The function reaches its global minimum at u = 0, and LaR(0) = 0. The limits at infinity are: lim u→+∞ LaR(u) = λ, lim u→−∞ LaR(u) = τλ. Since 0 < τ ≤ 1, we have τλ ≤ λ. Combined with monotonicity on both intervals, all function values satisfy 0 ≤ LaR(u) ≤ λ, ∀u ∈ R.

Property 3. The aR loss is robust to label noise (outliers), which can be theoretically guaranteed by influence function. The influence function satisfies IF(u) ≤ max(λa/e, 3τλ√(6a)/(2e(3/2))) < ∞, ∀u ∈ R. Therefore, according to the influence function theory, for the SVM model designed based on the aR loss, even if the loss u caused by label noise is extremely large, their impact on the model is still limited.

The aRSGTSVM model for classification is formulated as:

min (c1/2)(∥X+ w+ + e+ b+∥2 + ∥X− w− + e− b−∥2) + λ(∥w+∥1 + b+ + ∥w−∥1 + b−) + c2(Σ i∈N+ LaR((ξ+)i) + Σ i∈N− LaR((ξ−)i))

s.t. X+ w+ + e+ b+ + X+ w− + e+ b− ≥ e+ − ξ+, X− w− + e− b− + X− w+ + e− b+ ≤ ξ− − e−, ξ+ ≥ 0, ξ− ≥ 0.

The aRSGTSVR model for regression is formulated similarly, using X+ = (X+, Y + ε1 e+), X− = (X−, Y − ε2 e−), w+ = (wT+, η+), w− = (wT−, η−), where ε1, ε2 > 0 are tuning parameters.

The optimization is performed using the iPiano algorithm, which can solve optimization problems of the form min h(x) = f(x) + g(x), where g is convex (possibly nonsmooth), f is C1-smooth (possibly nonconvex). The algorithm iterates: w̃n+1 = (I + αn ∂g)−1(w̃n − αn ∇f(w̃n) + βn(w̃n − w̃n−1)), with step size parameters β = 0.5, α = (1−β)/L, where L is the Lipschitz constant of ∇f. The convergence is guaranteed by Theorem 4.8 in the iPiano framework established by Ochs et al., under the condition that the objective satisfies the Kurdyka-Lojasiewicz property and appropriate step size parameters are adopted.

Numerical experiments were conducted on synthetic datasets, UCI datasets, and real-world stock datasets. For classification, the proposed aRSGTSVM was compared with 1-SVM, TPMSVM, Pin-TSVM, rhingeSVM, and RoBoSS-SVM. For regression, aRSGTSVR was compared with SVR, LASSO, Elastic Net, TSVR, and Res-TSVR. The results consistently demonstrate that aRSGTSVM achieves the highest or tied-for-highest accuracy in nearly all experimental settings, including high-dimensional scenarios where p > n, while maintaining stable performance across repeated trials. As label noise increases, aRSGTSVM consistently outperforms competing methods. The ablation experiments confirm that the asymmetric RoBoSS loss function is effective in scenarios with label noise and high-dimensional features. The CPU time experiments show that aRSGTSVM maintains an extremely low running time for large sample sizes, though it is sensitive to feature dimensionality, incurring high computational costs in high-dimensional scenarios.

For regression, aRSGTSVR consistently achieves the lowest RMSE values across all experimental scenarios, including low-dimensional, boundary high-dimensional, and classical high-dimensional cases, while maintaining stable performance with relatively small standard deviations. In the index tracking task, the proposed aRSGTSVR achieves the best performance across all six index tracking datasets (bz50, cy200, hs300, xf100, ys50, zz500), demonstrating strong generalization capability.

The conclusion states that the core contribution of this study lies in that the designed asymmetric loss naturally balances model robustness and sparsity, which provides a reliable and superior alternative to conventional symmetric loss functions for practical scenarios suffering from label noise, high-dimensional features, and resampling instability. The paper also notes that cross-validation can only detect overfitting rather than alleviate it, and suggests exploring optimal information criteria (e.g., AIC, BIC) as a valuable direction for future research, as well as distributed or parallel algorithms to further improve computational efficiency.

Improvements to AI Systems:

1. Robust Loss Function Integration: Replace standard symmetric loss functions (e.g., hinge, squared error) in existing SVM, neural network, or gradient boosting frameworks with the proposed asymmetric, bounded, smooth aR loss. This enables AI systems to automatically down-weight outliers and label noise without manual threshold tuning, improving generalization on noisy real-world data.

2. Theoretical Robustness Guarantee: Incorporate the influence function bound (IF(u) ≤ max(λa/e, 3τλ√(6a)/(2e(3/2)))) into model selection and validation pipelines. An AI system can now certify the maximum impact of any single noisy sample on its decision boundary, enabling safer deployment in high-stakes domains (e.g., medical diagnosis, fraud detection) where worst-case robustness is critical.

3. Asymmetric Noise Handling: Use the τ parameter to control asymmetry in loss, allowing AI systems to treat positive vs. negative errors differently. This is particularly useful for imbalanced classification (e.g., rare disease detection) or regression with skewed targets (e.g., financial risk), where over- and under-prediction have different costs.

4. Sparse Feature Selection via l1-Norm: Integrate the l1-norm penalty into the model to automatically discard irrelevant features in high-dimensional settings (p >> n). The improved system can handle genomic, text, or sensor data with thousands of features while maintaining interpretability and reducing overfitting.

5. Fast Nonconvex Optimization: Adopt the iPiano proximal gradient algorithm for any nonconvex, nonsmooth objective. This allows AI systems to train robust models with bounded loss functions efficiently, even when the loss is not convex—previously a major computational bottleneck. The system can now scale to large datasets (e.g., millions of samples) with stable convergence.

6. Unified Classification and Regression Framework: Use the same aR loss and optimization backbone for both tasks, enabling a single AI system to switch between classification and regression without redesign. This reduces engineering overhead and allows transfer learning between tasks (e.g., pretraining on regression for stock index tracking, then fine-tuning for classification of market regimes).

7. Resampling Noise Stability: Leverage the loss's boundedness and smoothness near the boundary (u=0) to make AI systems robust to small perturbations in training data (e.g., duplicate samples, minor measurement errors). This improves reproducibility across repeated experiments and cross-validation folds.

8. Index Tracking and Portfolio Optimization: Apply the aRSGTSVR model directly to financial AI systems for index tracking, achieving lower tracking error than LASSO/Elastic Net. The improved system can dynamically select a sparse set of stocks to replicate an index while being robust to market noise, reducing transaction costs and improving risk-adjusted returns.

What the Improved AI System Can Do:

• Train robust classifiers and regressors on noisy, high-dimensional data (e.g., medical records with mislabels, financial time series with outliers) with provable worst-case performance bounds.

• Automatically select relevant features and ignore noise features, producing interpretable models even when p >> n.

• Handle asymmetric error costs (e.g., false negatives cost more than false positives) without custom loss engineering.

• Achieve stable performance across repeated resampling (bootstrap, cross-validation) and under label corruption rates up to 30% or more.

• Solve nonconvex optimization problems (e.g., robust deep learning with bounded activations) faster than generic subgradient methods, with convergence guarantees.

• Deploy in real-time financial systems for sparse index tracking with lower tracking error and lower turnover than existing sparse regression methods.

• Provide a single, unified API for both classification and regression tasks, simplifying model maintenance and enabling rapid prototyping across domains.