2.1 Relationships between convexity and contraction

Definition 1 (Strong Convexity). A twice differentiable function is α-strongly convex with α > 0 if its Hessian matrix ∇² f(x) satisfies the matrix inequality

As its name suggests, a function that is strongly convex is convex in the usual sense, while the converse is not always true. From a dynamic systems perspective, strong convexity provides exponential convergence of gradient flows:

Proposition 1 (Exponential Convergence of Gradient Systems for Strongly Convex Functions). If a twice differentiable function is α-strongly convex, then its gradient system (1) converges to the unique global minimum of f exponentially with rate α.

Toward proving this proposition, we will consider stability analysis through the application of nonlinear contraction theory.

Definition 2 (Contraction Metric [15]). A system is said to be contracting at rate α > 0 with respect to a symmetric positive definite metric , if for all and all , (2) where is the system Jacobian and . The system is said to be semi-contracting with respect to M when (2) holds with α = 0.

Given an α-contracting system and an arbitrary pair of initial conditions x₁(0) and x₂(0), the solutions x₁(t) and x₂(t) converge to one another exponentially (3) where denotes the geodesic distance on the Riemannian manifold . This property can be shown by considering the evolution of differential displacements δx, which describe the evolution of nearby trajectories and coincide with the notion of virtual displacements in Lagrangian mechanics. More precisely, letting x(t;x₀, t₀) denote the solution of from initial condition x(t₀) = x₀, differential displacements evolve according to Property (3) follows from the evolution of the squared length of these differential displacements [15], which verifies (4) Furthermore, if a system is α-contracting in a metric M that satisfies M(x) ≽ β I uniformly for some constant β > 0, then any two solutions verify

Example 2. Consider an α-strongly convex function f and its associated gradient descent system (1). Since f is strongly convex, it has a unique global minimum x*, which is a equilibrium point of (1). It can be verified that the gradient descent dynamics of f are contracting in the identity metric M = I with rate α. Since geodesic distances are just Euclidean distances in this metric, (3) immediately implies that thus proving Proposition 1.

From this example, it is clear that strongly convex functions are a special case of ones whose gradient systems are contracting. The following proposition shows that one does not lose the convergence properties to a global optimum on this more general class of functions.

Proposition 2 (Exponential Convergence of Contracting Gradient Systems). Consider again gradient descent as in Eq (1). The system converges exponentially to a unique global minimum if it is contracting in some metric.

Proof. Because (1) is autonomous and contracting, it converges exponentially to a unique equilibrium x* [15]. Furthermore, this equilibrium must be a global minimum since f can only decrease along trajectories, with for x ≠ x*.

The above result, which emphasizes contraction rather than convexity as a sufficient condition to converge to a global minimum, can be extended to the semi-contracting case as follows.

Proposition 3 (Asymptotic Convergence of Semi-Contracting Gradient Systems). Consider a twice differentiable function , a symmetric positive definite metric , and the associated gradient system (5) Assume that dynamics (5) is semi-contracting in some metric, and furthermore that one trajectory of the system is known to be bounded. Then, (a) f has at least one stationary point, (b) any local minimum of f is a global minimum, (c) all global minima of f are path-connected, and (d) all trajectories asymptotically converge to a global minimum of f.

Proof. (a) By assumption, there exists some initial condition x₀ such that x(t; x₀) remains bounded. This, in turn, implies that the ω-limit set ω[x(t; x₀)] is non-empty, compact, forward invariant, and that Let x* denote an element of ω[x(t; x₀)]. Since (5) is a gradient system, Theorem 15.0.3 of [28] guarantees that x* must be an equilibrium point of (5). This proves that f has at least one stationary point.

Let us now show that ω[x(t; x₀)] consists only of the single point x*, by contradiction. Let and be distinct elements in ω[x(t; x₀)]. Further let the geodesic distance between and . Then, the geodesic balls and are disjoint. Further, since the system is semi-contracting these geodesic balls are forward invariant. Yet, since a limit point, x(t;x₀) arrives within at some point, and never leaves. Likewise, since is a limit point, x(t;x₀) arrives within at some point, and never leaves. Thus, we have a contradiction, and the limit set must consist of a single point.

(b) and (c): Consider now two equilibrium points of (5), and , and a smooth path γ(s) such that and . Since the gradient dynamics are semi-contracting, for each s the solution x(t;γ(s)) remains bounded. Thus, by the same reasoning as above, each x(t;γ(s)) converges to some equilibrium x*(s) as t → + ∞. Since ∇f(x*(s)) = 0 for each s, and x*(s) smoothly connects and , it follows that . That is, all solutions converge to the same value for f.

(d): That all solutions of (1) asymptotically converge to a global minimum of f follows from that fact that f decreases along all solutions, and all solutions converge to the same value for f.

Remark 1. In the case that a contraction metric needs to be found numerically, note that the conditions (2) for certifying contraction or semi-contraction are convex criteria. Thus, in many instances, the process of finding a metric numerically to verify contraction may be accomplished via convex optimization approaches, such as those based on sums-of-squares programming [29].

2.2 Relationship between geodesic convexity and contraction

Geodesic convexity [12] generalizes conventional notions of convexity to the case where the domain of a function is equipped with a Riemannian metric. A special case occurs in geometric programming (GP) [30]. In GP, a non-convex problem over positive variables can be transformed into a convex problem by a change of variables . Alternately GP can be formulated over the positive reals viewed as a Riemannian manifold by measuring differential length elements ds in a relative sense (6) Geodesically-convex optimization generalizes this transformation strategy to a broader class of problems [13]. However, beyond special cases (see, e.g., [31]), generative procedures remain lacking to formulate g-convex optimization problems or recognize g-convexity.

To introduce g-convexity more formally, consider a function and a positive definite metric . We note that geodesic convexity of f is not an intrinsic property of the function itself, but rather is a property of f defined on the Riemannian manifold .

Definition 3 (g-Strong Convexity [32]). A twice differentiable function is said to be geodesically α-strongly convex (with α > 0) in a symmetric positive definite metric M if its Riemannian Hessian matrix H(x) satisfies: (7) The elements of the Riemannian Hessian are given as [32] (8) where provide the elements of the conventional (Euclidean) Hessian and denotes the Christoffel symbols of the second kind with M^ij(x) = (M(x)⁻¹)_ij. The function f is g-convex when (7) holds with α = 0.

The Riemannian Hessian generalizes the notion of the Hessian from a Euclidean context and captures the curvature of f along geodesics. Likewise, the natural gradient generalizes the notion of a Euclidean gradient to the Riemannian context in the following sense.

Definition 4 (Natural Gradient [33]). Consider equipped with a Riemannian metric M. The natural gradient of a differentiable function is the direction of steepest ascent on the manifold and is given in coordinates by M(x)⁻¹∇f(x).

Remark 2. When M(x) is the Hessian of some twice differentiable strictly convex scalar function ψ(x), natural gradient descent coincides with the continuous-time limit of mirror descent [34, Sec. 2.3] with potential ψ(x).

Remark 3. From a differential geometric viewpoint, the first covariant derivative of f is a covector field given in coordinates by ∇f(x), while the natural gradient is a vector field given in coordinates by M(x)⁻¹∇f(x) [33]. In a Euclidean context, where M(x) is identity, this distinction between covariant (covector) and contravariant (vector) representations of the gradient is immaterial.

Similarly, the Riemannian Hessian H represents in coordinates the second covariant derivative of f.

When M is the identity metric, geodesic α-strong convexity naturally coincides with the definition of α-strong convexity in Definition 1. The natural gradient can be used to directly mirror Proposition 1 within the Riemannian context.

Theorem 1 (Equivalence between g-Strong Convexity and Contraction of Natural Gradient). Consider a twice differentiable function , a symmetric positive definite metric , and the natural gradient system [33] (9) Then, f is α-strongly g-convex in the metric M for each t if and only if (9) is contracting with rate α in the metric M. More specifically, the Riemannian Hessian verifies (10) where .

Appendix 1 provides a self-contained proof using conventional tensor analysis methods [35], whose relationship with contraction conditions have been noted previously [36, 37]. The same relationships drive coordinate-free versions of the result in [38].

Remark 4. Theorem 1 can also be viewed as a special case of contraction analysis for complex Hamilton-Jacobi dynamics [37]. A reorganization of (9) as may be recognized as the generalized momentum being the negative covariant gradient within a Hamiltonian mechanics context.

Remark 5. While Theorem 1 applies to α-strong convexity, the link between the Riemannian Hessian and the contraction condition (2) also provides immediate equivalence between g-convexity of a function and semi-contraction of its natural gradient dynamics.

Remark 6. Eq (10) provides an alternate way to compute the geodesic Hessian H, and, as expected, leaves it invariant when the metric M is scaled by a strictly positive constant. Because of the structure of the natural gradient dynamics, scaling M is akin to scaling time and implies inversely scaling the contraction rate α, consistently with (7).

By contrast, note that given a fixed dynamics H, the contraction metric analyzing it can always be arbitrarily scaled while leaving the contraction rate unchanged.

Similar to in Section 2.1 where convexity corresponded to contraction of gradient in the identity metric, we likewise see that Thm. 1 imposes g-convexity via a particular choice of contraction metric for the natural gradient dynamics. Mirroring Prop. 2, removing this restriction on the contraction metric leads to significant additional flexibility for guaranteeing convergence to a globally optimal point.

Proposition 4 (Exponential Convergence of Contracting Natural Gradient Systems). Consider again natural gradient descent as in Eq (9). The system converges exponentially to a unique global minimum if it is contracting in some metric.

Proof. The proof follows immediately from the same logic as the proof of Proposition 2.

Remark 7. Note that contraction also provides robustness. Consider perturbed dynamics with uniformly. If the dynamics are contracting with rate λ, then all trajectories contract to a geodesic ball of radius R/λ [15]. This observation implies favorable properties for algorithms where an exact gradient may be difficult or intractable to compute, with approximation methods used in their place.

Theorem 2 (Semi-Contraction for Natural Gradient). Consider a twice differentiable function , a symmetric positive definite metric , and the associated natural gradient system (11) Assume that dynamics (11) is semi-contracting in some metric, and furthermore that one trajectory of the system is known to be bounded. Then, (a) f has at least one stationary point, (b) any local minimum of f is a global minimum, (c) all global minima of f are path-connected, and (d) all trajectories asymptotically converge to a global minimum of f.

Proof. The proof follows the exact same line of logic as the proof to Prop. 3. The result of Theorem 15.0.3 of [28], which guarantees that any ω-limit point of gradient descent (5) is an equilibrium point, generalizes immediately to the case of natural gradient descent (11).

Remark 8. The topology of global optimizers satisfying this semi-contraction condition is the same as those observed when training over-parameterized networks [2, 3]. However, empirical loss functions in these networks often also experience multiple saddle points [39, 40]. The attractor sets associated with strict saddles have measure zero [41, 42] under discrete gradient descent with sufficiently small stepsize (i.e., with adequately close approximation to the continuous time case), while the dimensionality of the attractor sets can be further reduced via smoothed versions of the gradient [43].

While the presence of strict saddles precludes the ability of a gradient system to be globally semi-contracting, any of the results given here can be generalized to forward invariant contraction or semi-contraction regions [15]. In principle, saddles could then be treated by excluding their measure zero attractor sets from suitably chosen contraction or semi-contraction regions.

The topology of equilibria in semi-contracting gradient systems immediately implies the following result.

Corollary 1. Consider an autonomous, semi-contracting natural gradient system. If the linearization at some equilibrium point is strictly stable, then all system trajectories tend to this global minimizer.

More generally, if some equilibrium is locally asymptotically stable, all trajectories tend to this global minimizer.

Proof. We prove the second part, the first then follows directly from Lyapunov’s linearization method. Existence of an equilibrium implies existence of a bounded trajectory. Furthermore, by definition, there exists a ball around the equilibrium point x* such that all trajectories initiated in that ball tend to x*. If there was another equilibrium, the path connecting it to x* would intersect that ball, which is a contradiction since the path is itself composed of equilibria via Thm. 2.

Remark 9. Strict stability of a natural gradient system at an equilibrium point can of course be established simply by ensuring that all eigenvalues of its Jacobian at this point are strictly in the left-half complex plane. This condition is equivalent to requiring that the Hessian of the objective function is positive definite at x*.

Indeed, given the natural gradient dynamics (11) with h(x) = −M(x)⁻¹∇f(x), the Jacobian at any equilibrium x* is Applying a similarity transformation with the symmetric square root of M(x*) yields All eigenvalues of the symmetric matrix above are real, and they are all strictly negative if and only if the Hessian ∇² f(x*) is positive definite.

Note that this condition is equivalent to the geodesic Hessian at x* being positive definite in any metric, as the Euclidean Hessian is numerically equal to the geodesic Hessian in any metric in this case, due to all terms multiplying the Christoffel symbols in (8) being zero.

Corollary 2. Consider an autonomous semi-contracting natural gradient system, and assume that the system has more than one equilibrium. Then, at any equilibrium, both the Jacobian matrix of the dynamics and the Hessian of the objective have at least one zero eigenvalue.

Proof. Consider an equilibrium x*, and an equilibrium path connecting it to some other equilibrium. The unit tangent vector at x* along this path is an eigenvector of the Jacobian with eigenvalue zero. Given the algebraic relation between the Jacobian and the objective Hessian pointed out in Remark 9, this shows in turn that the objective Hessian has a zero eigenvalue.

2.3 Examples

Let us illustrate Theorem 1 using the classical nonconvex Rosenbrock function: (12) This function has a unique global optimum at x* = [1, 1]^⊤, which is located along a long, shallow, parabolic-shaped valley.

Example 3 Consider the Rosenbrock function (12) and the metric [32] The metric M(x) satisfies and , and thus M(x) ≻ 0. Note that M(x) is not the Hessian of f(x). The natural gradient dynamics follows It can be verified algebraically that which shows that natural gradient descent is contracting with rate α = 2. This implies that the natural gradient dynamics satisfy where x* = [1, 1]^⊤. Equivalently, the Rosenbrock function is geodesically α-strongly convex with α = 2.

The Rosenbrock metric M(x) can be viewed as following from a differential change of variables where M = Θ^⊤ Θ yields δ x^⊤ M δ x = δ z^⊤ δ z. This differential change of variables is integrable, so that g-convexity of the Rosenbrock can be shown using the explicit nonlinear coordinate change and z₂ = x₁ − 1 that provides .

Example 4. Mirror descent provides another example of a metric corresponding to an explicit state transformation, with Newton’s method as a special case.

Consider a twice differentiable scalar objective function f(x), and a smooth strictly convex scalar function ψ(x). Denoting by H_f(x) = ∇² f(x) and H_ψ = ∇² ψ the Hessians of these functions, continuous-time mirror descent of f(x) under potential ψ(x) corresponds to natural gradient in the Hessian metric H_ψ [34, Sec. 2.3] (13) Consider the explicit change of variables z = ∇ψ(x), which can be written in differential form as δ z = H_ψ δ x. The dynamics (13) can be viewed in the mirror space as and therefore Letting , this yields Thus, continuous mirror descent (13) is contracting with rate λ > 0 in the metric if (14)

In the particular case when f is α-strongly convex and the potential function is chosen as ψ(x) = f(x), Eq (13) simply corresponds to Newton’s method, and (14) verifies that Newton’s method is contracting with rate 1 in the squared Hessian metric [44].

Note that the well-known result that the transformation z = ∇ϕ(x) is one-to-one (given the strict convexity of ψ) can also be shown by constructing, for a given z, the system (15) which is autonomous and contracting in the identity metric and thus must reach a unique equilibrium point.

The following proposition provides further insight into the case when the contraction metric is related to an explicit change of variables more generally.

Proposition 5 (Relationship between gradient and natural gradient under a diffeomorphic change of variables). Consider a diffeomorphic change of variables z = g(x), and the associated metric M(x) = Θ(x)^⊤ Θ(x), with . For any twice differentiable function , natural gradient descent in x is equivalent to gradient descent in z Proof. In the z coordinates we have

Proposition 6. Consider a metric M(x) and suppose there exists a diffeomorphic change of variables z = g(x) such that M(x) = Θ(x)^⊤ Θ(x), with . Then, the associated Riemannian curvature tensor with components R_ikℓm must be identically zero.

Proof. Note that since δz = Θ(x)δx, it follows that δz^⊤ δz = δx^⊤ M(x)δx and thus the Riemannian metric tensor expressed in the z coordinates is the identity. Since the components of the Riemannian metric tensor are constant in these transformed coordinates, it follows that the components of the Riemannian curvature tensor are identically zero [32]. Transformation laws for tensors ensure that the components of the curvature tensor remain zero under arbitrary coordinate change, thus R_ikℓm = 0.

The general freedom to consider differential changes of coordinates δz = Θ(x)δx where Θ is non-integrable provides additional flexibility and generality to both contraction analysis and g-convexity, as illustrated by the following examples.

Example 5. Consider the non-convex function which has a global minimum at x = 0. Contours of the function are shown in Fig 1. Gradient descent can be shown to be contracting at rate λ = 2 in the metric Fig 1 shows two solutions and plots their geodesic distance. The decay is, as expected, at a rate faster than the exponentially decreasing upper bound as derived from (3). The curvature tensor for this metric has some non-zero components, such as From Proposition 6, this shows that this metric cannot be derived from an explicit change of coordinates.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Contracting gradient descent corresponding to Example 5.

https://doi.org/10.1371/journal.pone.0236661.g001

Example 6. Consider the function and natural gradient descent with a given natural metric Θ(z)^⊤ Θ(z), where This natural gradient dynamics is verified semi-contracting in the metric Similar to Example 5, this metric has non-zero Riemannian curvature, and thus cannot be derived from a change of coordinates. Fig 2 shows the contours of f and two solutions of natural gradient descent. Fig 3 shows that the the geodesic distance between these two solutions is non-increasing. Since the system is only semi-contracting, the distance between solutions does not tend toward zero. It can be verified that f is a sum of squares and thus f(z) ≥ 0, and that f(z) = 0 when . Both initial conditions asymptotically lead to this path connected set of global optima.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Semi-contracting natural gradient descent for Example 6.

https://doi.org/10.1371/journal.pone.0236661.g002

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Semi-contracting natural gradient descent for Example 6.

https://doi.org/10.1371/journal.pone.0236661.g003

Example 7. Geodesically-convex optimization can also be used to carry out manifold-constrained optimization in an unconstrained fashion via recasting problems over a Riemannian manifold directly [14, 31]. Taking an intrinsic view of the manifold, coordinate free results are available [38], however, for the purposes of computation, we assume a global coordinate chart here. Consider for instance optimization over the set of n × n positive definite matrices, and specifically the problem of finding the Karcher mean of m matrices [13], which minimizes the objective function where denotes the Frobenius norm of a matrix A. The function f(X) is m-strongly convex [13] on in the metric that measures symmetric differential displacements as (16) Naturally, the requirement that δX be symmetric makes it an element of the tangent space to the manifold of symmetric positive definite matrices.

This metric generalizes the GP case (6), and coincides with the second-order terms in the Taylor series of the log barrier −logdet(X) [45]. The gradient of f(x) can be written and accordingly the natural gradient can be shown to satisfy From Theorem 1, any trajectory with arbitrary initial condition will remain within under the natural gradient descent dynamics since (intuitively) the Riemannian metric (16) makes any element on the boundary of the positive definite cone an infinite distance away from any one in the interior, and contraction of the natural gradient dynamics ensures that geodesic distances decrease exponentially.

Example 8. An approximation to the Riemannian distance of two positive definite (PD) matrices on the PD cone is given by the Bregman LogDet divergence on (17) The metric is convex in its first argument, and can be shown to be geodesically convex in the second. We illustrate the connection with contraction to show this property. Note that so that the natural gradient descent dynamics are simply with differential dynamics where the differential displacement δ X must be symmetric. Considering the rate of change in length of these differential displacements and defining the differential change of variables , one has and (18) for all δ Z ≠ 0. Hence, considering only the second argument to LogDet divergence, its Riemannian Hessian is positive definite, thus proving g-convexity via Thm. 1.

2.4 Non-autonomous systems and virtual systems

In our optimization context, the fact that contraction analysis is directly applicable to non-autonomous systems can be exploited in a variety of ways. As we shall detail later, a key aspect is that it allows feedback combinations or hierarchies of contracting modules to be exploited to address more elaborate optimization problems or architectures. Also, it makes the construction of virtual systems [46] possible to potentially extend results beyond natural-gradient descent.

Remark 10. The natural gradient M⁻¹(x)∇_x f(x, t) represents the direction of steepest ascent on the manifold at any given time. With this in mind, Remark 7 on robustness enables convergence analysis for natural gradient descent within time-varying optimization contexts [25]. Let x*(t) denote the optimum of a time-varying α-strongly g-convex function. If , then the natural gradient will track with accuracy R/α after exponential transient.

Remark 11. Consider a contracting natural gradient system of the form (9). In the autonomous case, equations governing the differential displacement follow (19) which has a similar structure to the time evolution of h(x) (20) Thus, for natural gradient descent of an α-strong g-convex function f(x), the same algebra leading to (4) also gives so that the Krasovskii-like function can be viewed as an exponentially converging Lyapunov function, with global minimum V = 0 at the unique minimum of f(x). Of course, (19) remains valid for non-autonomous systems as well, while (20) does not.

The use of virtual contracting systems [46–48] allows guaranteed exponential convergence to a unique minimum to be extended to classes of dynamics which are not pure natural gradient. For instance, it is common in optimization to adjust the learning rate as the descent progresses. Consider a natural gradient descent with the function f(x) α-strongly g-convex in metric M(x), and define the new system (21) where the smooth scalar function p(x, t) modulates the learning rate [33] and is uniformly positive definite, Let us show that this system tends exponentially to the minimum x* of f(x).

Consider the auxiliary, virtual system, (22) For this system, p(x(t), t) is an external, uniformly positive definite function of time, and thus so that the contraction of (9) with rate α implies the contraction of (22) with rate αp_min. Since both x(t) and x* are particular solutions of (22), this implies in turn that x(t) tends to x* with rate αp_min.

Note that since we only assumed that p(x, t) is uniformly positive definite, in general the actual system (21) is not contracting with respect to the metric M(x).

Remark 12. The learning rate may also be selected to improve the numerical properties of the algorithm in a discrete time implementation. For example, p(x, t) could vary as the inverse of the condition number of ∇² f(x) to improve numeric conditioning without impact on stability guarantees.

2.5 Contraction +

Corollary 1 above points to a more general class of results where contraction or semi-contraction properties are combined with other information, such as a stable local linearization or a decreasing cost, to provide global results.

3.1 Primal dual dynamics in natural adaptive control

This section illustrates the presence of natural primal-dual dynamics embedded in the application of natural adaptive control laws. Consider a system given by (26) with configuration , control , and unknown parameters . The regressor and symmetric matrix may depend nonlinearly on the state and its derivatives. We assume that the matrix J remains positive definite for all and that it is linear in a. As a result, there exists a regressor function W such that (27) (28) and a regressor function Q such that for any

Consider a desired trajectory x_d(t) and the associated sliding variable [27, 63] (29) where . With this sliding variable, we define a reference for the order n − 1 derivative of the state.

Choosing the control law where provides the closed-loop dynamics (30)

Inspired by the elegant modification of the Slotine and Li adaptive robot controller introduced by Lee et al. [64, 65], we consider the Lyapunov-like function where denotes the Bregman divergence of a function f assumed convex on and given by Note that the LogDet divergence from (17) follows this form for f(X) = −logdet(X). Here, we consider the case when is open and f is chosen as a convex barrier function on , such that the Hessian metric H = ∇² f(x) endows with a barrier Hessian manifold structure [65, 66]. Note that if f is a second-order function , the Bregman divergence is simply , with H⁻¹ equal to the constant matrix P⁻¹ similar to the standard adaptive algorithm [63].

A quick calculation shows that the derivative of the Bregman divergence is simply , so that the adaptation law (31) yields Considering a virtual system with W and Q as externally provided functions of time, the dynamics (30) and (31) are equivalent to natural primal-dual over the function in the decoupled metric M_s = J and . Overall, this construction enables the results in natural adaptive robot control [64, 65] to be extended to the broader class (26).

Remark 21. Note that a similar construction could be applied to provide natural adaptation within recent applications of nonlinear adaptive control [67, Thm. 2] based on control contraction metrics [48, 50].

3.2 Natural primal dual

Continuous-time convex primal-dual optimization is analyzed from a nonlinear contraction perspective in [25], building on a earlier result of [19]. As we now show, Theorem 1 yields a natural extension to geodesic primal-dual optimization, where convexity in terms of primal and dual variables is replaced by g-convexity, thus broadening the above results to state-dependent metrics.

Theorem 5. Consider a scalar function ℒ(x, λ, t), with ℒ g-strongly convex over x and g-strongly concave over λ in metrics M_x(x) and M_λ(λ) respectively. Then, the geodesic primal-dual dynamics (25) is globally contracting, in metric (32)

Proof. Letting z = [x^⊤, λ^⊤]^⊤ and denote the overall system dynamics, the system’s Jacobian can be written (33) so that, using Theorem 1,

Proposition 10. Consider the primal dual dynamics (25) for a scalar cost function ℒ(x, λ), with ℒ g-strongly convex over x and g-concave (not necessarily strongly so) over λ in metrics M_x(x) and M_λ(λ) respectively. Suppose also that one solution of (25) is known to be bounded. Then, for any initial condition, the geodesic primal-dual dynamics (25) converge to an equilibrium x*, λ*. Moreover, x* is independent of initial conditions.

Proof. The proof is given as a corollary to Theorem 6 in the next section.

Remark 22. The above proposition highlights that contraction of the PD dynamics (e.g., as developed in [25]) is not necessary to guarantee convergence to a unique primal solution. Note however, that the above results only guarantee asymptotic convergence toward the unique primal equilibrium, as opposed to exponential convergence [25] when contraction can be shown for the PD dynamics as a whole.

This observation is reminiscent of results in adaptive control wherein the error dynamics of a certainty-equivalent controller may be asymptotically stable despite the fact that an associated adaptation law may not converge to the actual unknown parameters [63, 68], with adaptation occurring on a “need-to-know” basis in that sense. Conceptually, this principle can apply to more general contexts involving concurrent control and learning, when effective control is the main goal (e.g., in reinforcement learning).

4.1 Sum of g-convex

If two functions f₁(x, t) and f₂(x, t) are g-convex in the same metric for each t, then their sum f₁(x, t) + f₂(x, t) is g-convex in the same metric.

Example 9. Consider a function f₁(x₁, y₁, t) g-convex for each t in a block diagonal metric and a function f₂(x₂, y₂, t) g-convex for each t in a block diagonal metric . Then, the function: is g-convex in metric for each t.

4.2 Skew-symmetric feedback coupling

Assume that a scalar function f₁(x₁, x₂) is α₁-strongly g-convex in x₁ in a metric M₁(x₁) for each fixed x₂, and similarly that a scalar function f₂(x₁, x₂) is α₂-strongly g-convex in a metric M₂(x₂) for each fixed x₁. If f₁ and f₂ satisfy the scaled skew-symmetry property (34) where k is some strictly positive constant, then the natural gradient dynamics (35) is contracting with rate min(α₁, α₂) in metric M(x₁, x₂) = BlkDiag(M₁(x₁), k M₂(x₂)). Since the overall system is both contracting and autonomous, it tends to a unique equilibrium [15] which satisfies the Nash-like conditions

Note that the result can be broadened to cases where the scaled skew-symmetry property is not exactly satisfied, by using the small-gain extension in [19]. Taking again the machine learning context as a potential example, such two-player game dynamics can occur in certain types of adversarial training.

The result extends to a game with an arbitrary number of players. Consider n functions such that each f_i is α_i-strongly g-convex over x_i in a metric M_i(x_i). If the functions satisfy the skew-symmetry conditions for each j > i, then the suitable generalizations of (35) result in a coupled system that is contracting with rate min(α₁, …, α_n) in the metric The overall system converges to a unique Nash-like equilibrium satisfying and a similar relation for each other player.

Likewise, the result can be extended to the case when the natural gradient dynamics for each individual player may only be semi-contracting.

Theorem 6. Consider the two player case (35), wherein (a) f₁ is α₁-strongly g-convex with α₁ > 0 in a uniformly positive definite metric M₁(x₁) for each x₂ (b) the Riemannian Hessian H₂(x₁, x₂) of f₂(x₁, x₂) in x₂ is only positive semi-definite for each x₁ in a uniformly positive definite metric M₂(x₂) and (c) the skew-symmetry property (34) holds. Assume that one trajectory of (35) is known to be bounded. Then, every trajectory of (35) converges to a Nash equilibrium , . Moreover, does not depend on initial conditions (i.e., every Nash has the same strategy for player 1).

Proof. It can be shown that virtual displacements evolve such that which implies, by Barbalat’s lemma, Via the same argument as follows from (20), it follows that x₁(x₁(t), x₂(t)) → 0 as t → ∞. So, for each initial condition, x₁(t) must converge to some equilibrium of the x₁ dynamics. Furthermore, since any δx₁ → 0, must be unique and independent of initial conditions. Let us now turn to the behavior of the x₂ dynamics. Given an arbitrary initial condition (x_1,0, x_2,0) for (35), let L⁺ denote its ω-limit set. Any point in (x₁, x₂) ∈ L⁺ must satisfy . Since the dynamics are autonomous, L⁺ is composed of trajectories of the system (36) (37) Moreover, L⁺ must be closed and bounded. Since (37) is a natural gradient system of a g-convex function, Thm. 2 ensures that any trajectory of (37) must converge to an equilibrium point that is a global minimizer for . Considering any initial condition of (37) that begins in L⁺, we denote as the resulting equilibrium point. However, since (35) is semi-contracting, any geodesic ball around is forward invariant for (35), which implies that . Thus, as t → ∞. Note again that, while depends on initial conditions, does not.

Corollary 3. Consider any two equilibrium points and for a system that satisfies the condition of Theorem 6. Then, the geodesic between these points is comprised of extremal Nash equilibrium points, all of which have the same cost.

Further, if one equilibrium of (37) is locally asymptotically stable, then it is necessarily globally attractive, and thus all trajectories of (37) converge to this unique equilibrium regardless of initial conditions.

Proof. Proof of the first part follows immediately from applying Corollary 3.1 of [12] to the function . Proof of the second part follows immediately from the application of Corollary 1 herein.

Corollary 4. Proposition 10 is true.

Proof. Consider Thm. 6 with and .

4.3 Hierarchical natural gradient

Consider a function f₁(x₁) α₁-strongly g-convex in a metric M₁(x₁), and a function f₂(x₁, x₂) α₂-strongly g-convex in a metric M₂(x₂) for each given x₁. Then, the hierarchical natural gradient dynamics is contracting with rate min(α₁, α₂) in metric M(x₁, x₂) = BlkDiag(M₁(x₁), M₂(x₂)), under the mild assumption that the coupling Jacobian is bounded [15]. Since the overall system is both contracting and autonomous, it tends to a unique equilibrium [15] at rate min(α₁, α₂), and thus to the unique solution of

By recursion, this structure can be chained an arbitrary number of times, or applied to any cascade or directed acyclic graph of natural gradient dynamics. Such hierarchical optimization may play a role, for instance, in backpropagation of natural gradients in machine learning, with all descents occurring concurrently rather than in sequence.

Remark 23. In large-scale optimization settings such as those appearing commonly in machine learning, natural gradient with a fully-dense metric can become intractable. In specific cases, such as natural gradient descent based on Fisher information [33], computationally effective approximations have been derived [69, 70]. In addition, the combination of simple (e.g., diagonal) metrics through hierarchical structures lends an opportunity to recover significant complexity at broad scale − see, e.g., the hierarchical combination of scalar metrics to learn hierarchical representations of symbolic data in [71, 72]. Such simpler metrics are also well motivated in the context of positive or monotone systems [73, 74]. In special cases of a dense Hessian metric M(x) = ∇² ψ(x) from a potential ψ(x), note that continuous mirror descent (see also Proposition 5 and Example 4) provides an alternate method to compute continuous natural gradient. These methods can avoid the need to invert the metric in cases where there is an explicit inverse exists for the change of variables z = ∇ψ(x), or when (15) can be run at a fast time scale to invert the gradient map through dynamics.