-
Duality of Hoffman constants
Authors:
Javier F. Pena,
Juan C. Vera,
Luis F. Zuluaga
Abstract:
Suppose $A\in \mathbb{R}^{m\times n}$ and consider the following canonical systems of inequalities defined by $A$: $$ \begin{array}{l} Ax=b\\ x \ge 0 \end{array} \qquad \text{ and }\qquad A^T y - c \le 0. $$ We establish some novel duality relationships between the Hoffman constants for the above constraint systems of linear inequalities provided some suitable Slater condition holds. The crux of o…
▽ More
Suppose $A\in \mathbb{R}^{m\times n}$ and consider the following canonical systems of inequalities defined by $A$: $$ \begin{array}{l} Ax=b\\ x \ge 0 \end{array} \qquad \text{ and }\qquad A^T y - c \le 0. $$ We establish some novel duality relationships between the Hoffman constants for the above constraint systems of linear inequalities provided some suitable Slater condition holds. The crux of our approach is a Hoffman duality inequality for polyhedral systems of constraints. The latter in turn yields an interesting duality identity between the Hoffman constants of the following box-constrained systems of inequalities: $$ \begin{array}{l} Ax=b\\ \ell \le x \le u \end{array}\qquad \text{ and }\qquad \ell \le A^T y - c \le u $$ for $\ell, u\in \mathbb{R}^n$ with $\ell < u.$
△ Less
Submitted 15 December, 2023;
originally announced December 2023.
-
Equivalence and invariance of the chi and Hoffman constants of a matrix
Authors:
Javier F. Pena,
Juan C. Vera,
Luis F. Zuluaga
Abstract:
We show that the following two condition measures of a full column rank matrix $A \in \mathbb{R}^{m\times n}$ are identical: the chi constant and a signed Hoffman constant. This identity is naturally suggested by the evident invariance of the chi constant under sign changes of the rows of $A$. We also show that similar equivalence and invariance properties extend to variants of the chi and Hoffman…
▽ More
We show that the following two condition measures of a full column rank matrix $A \in \mathbb{R}^{m\times n}$ are identical: the chi constant and a signed Hoffman constant. This identity is naturally suggested by the evident invariance of the chi constant under sign changes of the rows of $A$. We also show that similar equivalence and invariance properties extend to variants of the chi and Hoffman constants that depend only on the linear subspace $A(\mathbb{R}^n):=\{Ax: x\in\mathbb{R}^n\} \subseteq \mathbb{R}^m$. Finally, we show similar identities between the chi constants and signed versions of Renegar's and Grassmannian condition measures.
△ Less
Submitted 18 May, 2020; v1 submitted 15 May, 2019;
originally announced May 2019.
-
The condition number of a function relative to a set
Authors:
David H. Gutman,
Javier F. Pena
Abstract:
The condition number of a differentiable convex function, namely the ratio of its smoothness to strong convexity constants, is closely tied to fundamental properties of the function. In particular, the condition number of a quadratic convex function is the square of the aspect ratio of a canonical ellipsoid associated to the function. Furthermore, the condition number of a function bounds the line…
▽ More
The condition number of a differentiable convex function, namely the ratio of its smoothness to strong convexity constants, is closely tied to fundamental properties of the function. In particular, the condition number of a quadratic convex function is the square of the aspect ratio of a canonical ellipsoid associated to the function. Furthermore, the condition number of a function bounds the linear rate of convergence of the gradient descent algorithm for unconstrained convex minimization.
We propose a condition number of a differentiable convex function relative to a reference convex set and distance function pair. This relative condition number is defined as the ratio of a relative smoothness to a relative strong convexity constants. We show that the relative condition number extends the main properties of the traditional condition number both in terms of its geometric insight and in terms of its role in characterizing the linear convergence of first-order methods for constrained convex minimization.
When the reference set $X$ is a convex cone or a polyhedron and the function $f$ is of the form $f = g\circ A$, we provide characterizations of and bounds on the condition number of $f$ relative to $X$ in terms of the usual condition number of $g$ and a suitable condition number of the pair $(A,X)$.
△ Less
Submitted 18 April, 2020; v1 submitted 24 January, 2019;
originally announced January 2019.
-
Perturbed Fenchel duality and first-order methods
Authors:
David H. Gutman,
Javier F. Peña
Abstract:
We show that the iterates generated by a generic first-order meta-algorithm satisfy a canonical perturbed Fenchel duality inequality. The latter in turn readily yields a unified derivation of the best known convergence rates for various popular first-order algorithms including the conditional gradient method as well as the main kinds of Bregman proximal methods: subgradient, gradient, fast gradien…
▽ More
We show that the iterates generated by a generic first-order meta-algorithm satisfy a canonical perturbed Fenchel duality inequality. The latter in turn readily yields a unified derivation of the best known convergence rates for various popular first-order algorithms including the conditional gradient method as well as the main kinds of Bregman proximal methods: subgradient, gradient, fast gradient, and universal gradient methods.
△ Less
Submitted 3 December, 2021; v1 submitted 25 December, 2018;
originally announced December 2018.
-
The condition of a function relative to a polytope
Authors:
David H. Gutman,
Javier F. Pena
Abstract:
The condition number of a smooth convex function, namely the ratio of its smoothness to strong convexity constants, is closely tied to fundamental properties of the function. In particular, the condition number of a quadratic convex function is precisely the square of the diameter-to-width ratio of a canonical ellipsoid associated to the function. Furthermore, the condition number of a function bo…
▽ More
The condition number of a smooth convex function, namely the ratio of its smoothness to strong convexity constants, is closely tied to fundamental properties of the function. In particular, the condition number of a quadratic convex function is precisely the square of the diameter-to-width ratio of a canonical ellipsoid associated to the function. Furthermore, the condition number of a function bounds the linear rate of convergence of the gradient descent algorithm for unconstrained minimization.
We propose a condition number of a smooth convex function relative to a reference polytope. This relative condition number is defined as the ratio of a relative smooth constant to a relative strong convexity constant of the function, where both constants are relative to the reference polytope. The relative condition number extends the main properties of the traditional condition number. In particular, we show that the condition number of a quadratic convex function relative to a polytope is precisely the square of the diameter-to-facial-distance ratio of a scaled polytope for a canonical scaling induced by the function. Furthermore, we illustrate how the relative condition number of a function bounds the linear rate of convergence of first-order methods for minimization of the function over the polytope.
△ Less
Submitted 1 February, 2018;
originally announced February 2018.
-
Convergence rates of proximal gradient methods via the convex conjugate
Authors:
David H. Gutman,
Javier F. Pena
Abstract:
We give a novel proof of the $O(1/k)$ and $O(1/k^2)$ convergence rates of the proximal gradient and accelerated proximal gradient methods for composite convex minimization. The crux of the new proof is an upper bound constructed via the convex conjugate of the objective function.
We give a novel proof of the $O(1/k)$ and $O(1/k^2)$ convergence rates of the proximal gradient and accelerated proximal gradient methods for composite convex minimization. The crux of the new proof is an upper bound constructed via the convex conjugate of the objective function.
△ Less
Submitted 8 January, 2018; v1 submitted 8 January, 2018;
originally announced January 2018.
