Casewise and Cellwise Robust Tensor-on-Tensor Regression

Tensor-on-tensor (TOT) regression is an important tool for analyzing tensor data, aiming to predict response tensors from predictor tensors, and this paper introduces ROTOT, a novel robust method that can handle both casewise and cellwise outliers simultaneously while also accommodating missing values.

The gist

ROTOT is a robust Tensor-on-Tensor (TOT) regression method designed to handle both casewise and cellwise outliers in response tensors, along with missing values in both response and predictor tensors, by using a single loss function that reduces the influence of these contaminants.

Methodology Overview

The ROTOT method addresses contamination in the predictors through Robust Multilinear Principal Component Analysis (ROMPCA), which yields imputations for cellwise outliers and casewise weights. For missing values in response tensors, a tensor Mn is used to indicate observed cells, and m is the total number of observed cells. The core of ROTOT estimates the slope array B along with the intercept B0 by minimizing a complex objective function (Equation 6).

Robustness through Loss Functions

To simultaneously address both types of outliers, ROTOT employs a weighted M-estimation approach using two functions, ρ1 and ρ2, in the loss function. The objective function (6) is defined as:

L = σˆ2 / 2m X N n=1 mn w x nρ2 (1 / σˆ2 vuut 1 mn Q1X···QM q1···qM mn,q1…qM σˆ2 1,q1…qM ρ1) + λ∥B∥F2.

The functions ρ-functions are chosen to be the hyperbolic tangent (tanh), which is described as a valid ρ-function. Specifically, ρ1 limits the influence of cellwise outliers in response tensors by tempering their contribution via boundedness, while ρ2 mitigates casewise outliers with a large deviation.

Iteratively Reweighted Least Squares (IRLS) Algorithm

The minimization of the ROTOT objective function is performed via an iteratively reweighted least squares algorithm. This involves solving a system of first-order conditions (Equations 11, 12, and 13) in each iteration step. The update for the core tensors Ul and Vm is obtained by solving linear systems involving matrices T(−l)U and T(−m)V, which are constructed from the current estimates. The weight tensor Wn is defined as Wn = Wxn ⊙ Wcasen ⊙ Wcelln ⊙ Mn, where the entries of Wcelln reflect cellwise weights based on standardized residuals (Equation 18).

Outlier Detection and Visualization

ROTOT provides numerical and graphical diagnostics. First, ROTOT residual tensors Rn are computed, followed by M-scales to yield standardized residual tensors Ren. A residual cellmap visualizes the cellwise outlyingness of entries in the data, flagging cells whose absolute value exceeds a threshold ccell = qχ210.998 = 3.09 as cellwise outliers. Second, an outlier map is proposed based on ROTOT and ROMPCA outputs, displaying the residual distance (∥Ren∥F) versus a score distance (SDn). This map visualizes both casewise and cellwise outliers in the response, as well as good and bad leverage points in the predictor.

Performance Evaluation

The performance of ROTOT is evaluated through extensive simulations using Monte Carlo studies. The Relative Prediction Error (RPE) is used as a measure of performance on an uncontaminated validation set. Simulations show that ROTOT consistently outperforms classical TOT regression, OnlyCase-TOT, and OnlyCell-TOT across various contamination scenarios involving cellwise outliers, casewise outliers, and missing values in both response and predictor tensors. The method demonstrates favorable performance even when the predictors are contaminated.

Real Data Application

ROTOT is applied to the Labeled Faces in the Wild (LFW) dataset to predict facial attributes from images. The method is compared against classical TOT regression, and its performance, measured by robMSE (trimmed Mean Squared Error), consistently shows a lower median robMSE than TOT across all folds and contamination levels. Furthermore, analysis of residual cellmaps confirms the presence of outlying attributes in the response tensor for specific cases.

Software Availability

The R code that reproduces the example is available at https://wis.kuleuven.be/statdatascience/robust/software, and data availability information is provided via a DOI link for LFW. The supplementary materials contain additional material and R code for the proposed method and an example script.

References

Alqallaf et al. (2009), Ballard & Kolda (2025), Beale & Little (1975), Bi et al. (2021), Centofanti et al. (2026), Dian et al.

As a fastidious and diligent AI researcher, I have analyzed the provided scientific paper, Robust Tensor-on-Tensor Regression (ROTOT), and identified several high-impact areas where its methodology could be directly leveraged to improve existing AI systems.

Here are the specific improvements and what the resulting AI system can achieve:

---

) 1. Enhanced Robustness in Tensor Data Modeling

The core improvement is moving from standard Tensor-on-Tensor (TOT) regression to the proposed ROTOT method, which simultaneously handles both casewise outliers (outliers across observations) and cellwise outliers (anomalous individual cells within tensors).

2. Improved AI System Capabilities:

A system utilizing ROTOT can perform highly reliable prediction or inference on complex, multiway data structures where noise and errors are inevitable. This is critical in domains like:

• Predicting complex biological processes from multi-modal imaging (e.g., predicting disease progression from time-series/spatial tensor data).

• Modeling high-dimensional user behavior (e.g., predicting next purchase sequences or user preferences based on multiple interaction tensors).

• Analyzing large, sparse datasets where individual feature interactions (cells) and overall observation characteristics (cases) can be anomalous.

3. Robust Handling of Missing Data

ROTOT explicitly accommodates missing values in both the response and predictor tensors through imputation schemes derived from ROMPCA and the iterative reweighted least squares (IRLS) framework.

4. Improved AI System Capabilities:

• Predictive accuracy on incomplete datasets is significantly enhanced, as it does not require discarding observations with missing data or relying on simple mean/mode imputation.

• The system can generate coherent predictions even when input data is partially corrupted by missing entries, leading to more stable and less biased outputs.

5. Outlier Detection and Diagnosis

The paper proposes two sophisticated diagnostic tools: a residual cellmap (visualizing localized cellwise anomalies) and an outlier map based on the combination of ROTOT residuals and ROMPCA scores (visualizing both response outliers and predictor leverage points).

6. Improved AI System Capabilities:

• Automated anomaly detection within the predictive pipeline. The system can not only make predictions but also provide a confidence score indicating whether a specific prediction is influenced by an outlier in the input data (e.g., This prediction is highly sensitive to cell 'X' in predictor tensor 'Y').

• Diagnostic feedback loop: By analyzing the outlier map, developers can pinpoint whether performance degradation stems from poor model fitting (bad leverage points) or corrupted input features (outlying predictors).

7. Robust Feature Extraction via ROMPCA

The use of Robust Multilinear Principal Component Analysis (ROMPCA) to handle predictor outliers ensures that the underlying latent structure captured by the core tensors is less biased by contaminated input data.

8. Improved AI System Capabilities:

• More reliable feature extraction from noisy or incomplete sensor/image data. The resulting imputed tensors provide a cleaner representation of the true underlying manifold, improving downstream tasks like classification or clustering that depend on these extracted features.

9. Optimized Learning Algorithm (IRLS)

The Iteratively Reweighted Least Squares (IRLS) algorithm, coupled with robust loss functions based on the hyperbolic tangent, allows the model to dynamically adjust its weighting scheme based on the estimated robustness parameters and residuals.

10. Improved AI System Capabilities:

• Adaptive learning rate/weighting mechanism. The system learns how much to trust each data point or cell in real-time during training/inference, leading to a more adaptive and self-correcting learning process compared to fixed weight methods (like standard ridge regression).

11. Optimized Hyperparameter Tuning

The methodology for selecting regularization parameters (like the penalty term weighting factor λ and the rank R for CP decomposition) is based on cross-validation using robust metrics like the median Relative Prediction Error (robMSE), specifically targeting robustness against both cellwise and casewise contamination.

12. Improved AI System Capabilities:

• Automated pipeline tuning for deployment. The system can automatically select optimal regularization strengths that maximize performance under expected real-world contamination scenarios, reducing the need for manual hyperparameter search in production environments.