Published Paper Details:

Distribution shift, where the test-time data distribution differs from that of the training data, is a critical challenge in real-world machine learning deployments. This paper presents an empirical study of model brittleness under naturalistic distribution shifts in tabular datasets, a domain that remains understudied despite tabular data being central to applications in finance, healthcare, and public policy. We assess the stability of various machine learning models, including logistic regression, gradient-boosted decision trees, and multilayer perceptrons, in response to deployment-driven shifts that alter feature distributions while maintaining the semantics of the prediction task. In addition to these naturalistic shifts, we implement controlled feature-level perturbations as stress tests. Across five classification datasets, all models exhibit significant performance degradation under distribution shift, with vanilla neural networks generally showing larger robustness gaps than tree-based methods, although these gaps can be substantially reduced by standardization and regularization. The study provides an empirical foundation for understanding robustness trade-offs across model classes in structured data settings and highlights open challenges for robust tabular machine learning