AI Insight
This paper addresses the computational complexity of solving constrained convex-concave min-max optimization problems using stochastic gradient methods. The authors improve the oracle complexity for reducing the gradient mapping norm below a threshold epsilon from O(epsilon^-4) to O(epsilon^-2), matching the efficiency achieved in unconstrained optimization. They extend this result to scenarios without bounded variance assumptions by employing the Blum-Gladyshev condition.
Why it matters
This work has significant implications for machine learning applications involving adversarial training, robust optimization, and game theory, where constrained min-max problems frequently arise. The improved complexity bound means more efficient algorithms that require substantially fewer gradient computations to reach a desired accuracy level, potentially reducing training time and computational costs in practical applications.
Understand the Science
⚠️ Preprint – Noch nicht peer-reviewed
Dieser Artikel wurde noch nicht von unabhängigen Experten begutachtet. Die Ergebnisse sind vorläufig und sollten mit Vorsicht interpretiert werden.
Abstract: We study the stochastic first-order oracle complexity for constrained or regularized convex-concave min-max optimization and stochastic monotone variational inequalities. We focus on the case when suboptimality is measured in terms of the gradient mapping, also known as, forward-backward or natural residual, an optimality notion that generalizes the gradient norm for unconstrained problems. In this setting, under standard unbiased oracle access with now-standard variance assumptions, the best-known complexity for making the norm of the gradient mapping less than $varepsilon$ is $widetilde{O}(varepsilon^{-4})$, compared to the near-optimal $widetilde{O}(varepsilon^{-2})$ that is established in the unconstrained case. We bridge this gap to improve the gradient mapping complexity for constrained convex-concave min-max problems to $widetilde{O}(varepsilon^{-2})$. We then extend to prove the same complexity for problems without the bounded variance, by using the Blum-Gladyshev assumption.
Source: How to Make the Gradient Mapping Small for Constrained Stochastic Min-Max Problems and Beyond