pipette
ESEspañol

A Unifying View of Distributionally Robust Optimization for Probability Smoothing via Maximum Entropy

Kota Kurihara, Yuya Yamakawa, Yoichi Izunaga

Preprint

In the authors' words

The probability smoothing problem seeks estimators that assign positive probability to all categories, including those unobserved in a finite sample. In our previous work, we established that the estimator derived from distributionally robust optimization (DRO) with a -norm ambiguity set (-DRO) satisfies the axioms of Positivity, Symmetry, and Order Preservation, while relaxing the rigid Ratio Preservation axiom characterizing Laplace smoothing. In this paper, we observe that Symmetry itself imposes a strong constraint: it forces all unobserved categories to receive identical probability regardless of their relation to observed categories. Motivated by this observation, we study DRO with an optimal transport ambiguity set (OT-DRO), which incorporates inter-category dissimilarities through a cost matrix and thereby enables structure-respecting smoothing. We derive a tractable dual reformulation of OT-DRO as an empirical cross-entropy minimization with a Lipschitz-type penalty on the log-probability vector, and we identify an explicit threshold on the robustness radius at which the optimal estimator reduces to the uniform distribution, paralleling the threshold established for -DRO. We then show, through Sion's minimax theorem, that DRO for probability smoothing over any compact convex ambiguity set that contains a strictly positive distribution is equivalent to entropy maximization over that set. This unifying perspective explains the parallel threshold structure in -DRO and OT-DRO.

Main resultThe abstract does not state a limitation.

Appeared: Tuesday, October 20. arXiv. Preprint, not yet peer-reviewed.

Authors' comment: 17 pages, 4 figures