2.1 Rooted phylogenetic networks and their layout

Let X be a set of taxa. A rooted phylogenetic network on X is a directed graph with node set V, edge set E × , root node , and taxon labeling λ, such that [26]:

1. The graph N is a directed acyclic graph (DAG),
2. There is exactly one root node with in-degree 0, namely ρ (and thus, in particular, N is connected),
3. The labeling is a bijection between X and the set of leaves (nodes of out-degree 0),
4. There are no “through nodes” that have both in-degree 1 and out-degree 1, and
5. All leaves should have in-degree .

A node v is called a tree node if it has in-degree , and a reticulate node otherwise. An edge is called a tree edge or reticulate edge if its target node w is a tree node or reticulate node, respectively.

We call a network bifurcating if all nodes v have out-degree , and bicombining if all nodes v have in-degree .

If there are no reticulate edges, then N is a tree, and thus planar, it can be drawn in the plane without edge crossings. In the presence of reticulate edges, N may be non-planar, and any drawing of N may necessarily involve crossings.

Viewing a rooted phylogenetic network as a generalization of the widely-used concept of a rooted phylogenetic tree, our goal when drawing a non-planar rooted phylogenetic network is to do so in such a way that tree edges never cross each other, while reticulate edges may cross any edges. To enhance visual clarity, we aim to reduce the number and extent of crossings in the drawing.

[11] investigates the problem of minimizing the number of crossings in bifurcating and bicombining rooted phylogenetic networks. They assume that the network N is tree-based (i.e., possesses a spanning tree T, which is assumed to be provided as part of the input), and study theoretical properties of different ways to draw the network in a “transfer view” (as defined below).

Here, we consider the more general case in which networks are not necessarily bifurcating or bicombining, and reticulations can be displayed either in a combining view, a transfer view, or using a mixture of both. Rather than explicitly trying to minimize the number of crossings, we instead aim to minimize the total reticulate displacement of reticulate edges in a left-to-right drawing of the network.

2.2 Combine view

There are two distinct ways to interpret reticulate nodes in a rooted phylogenetic network. In a combining view, a reticulate node represents a combining event, such as hybridization-by-speciation, where all incoming edges are treated equally and rendered in a similar fashion (see Fig 2B). In contrast, a transfer view represents a transfer event, such as horizontal gene transfer. In this case, one incoming edge –the transfer-acceptor edge– represents the main lineage and is drawn like a regular tree edge, while all other incoming edges represent transferred material and are drawn as reticulate edges (see Fig 2D).

While the visualization of a rooted phylogenetic network can easily accommodate the simultaneous occurrence of both combining and transfer nodes, when drawing cladograms, two different strategies appear to lead to better results depending on whether the network mainly contains combining or transfer nodes.

In this section, we focus on the combining view. A later section will describe how to adapt the algorithm to support the transfer view. We consider drawing rooted networks in a left-to-right layout—from the root on the left to the leaves on the right—and how to optimize such drawings.

We begin with the case of a cladogram, where edge lengths are ignored, and later extend the approach to phylograms, which incorporate edge lengths. Finally, we discuss how to adapt the algorithm to obtain a circular layout of the network. All algorithms described here use the following concept to calculate y-coordinates.

Backbone tree. For any reticulate node v in N, we define the lowest stable ancestor LSA(v) to be the last node on all paths from the root ρ to v. Consider the following operation on N: for each reticulate node v, set , delete all edges entering v, and create a new edge from u to v. Note that application of this operation to all reticulate nodes gives rise to a rooted backbone tree B whose root is ρ and whose leaf set contains the leaf set of N, possibly along with other unlabeled leaves. Also note that the tree may contain through nodes. We call B the backbone tree of N.

Combining cladogram algorithm. To compute a left-to-right layout of a rooted phylogenetic network N, we proceed in two main steps.

First, we perform a post-order traversal of the backbone tree B. For each leaf u encountered, we assign , where u is the i-th leaf visited. For each internal node v, we assign y(v) either as the average of the y-coordinates of its children or, optionally, as the average of all its descendant leaves.

Second, we recursively compute the depth d(v) of each node v in the network N, defined as the maximum number of edges in any directed path from the root to v. We then set for all nodes.

Once coordinates have been assigned, we construct the left-to-right rooted cladogram by drawing each edge as a two-segment path: from (x(v), y(v)) to (x(v), y(w)), then to (x(w), y(w)). Tree edges are rendered as right-angled paths, while reticulate edges may be drawn using quadratic curves, for example.

This algorithm yields an early layout, in which branching nodes are positioned as close to the root as possible (see Fig 2B). To produce a late layout, where branching nodes appear nearer to the leaves (see Fig 2D), a post-processing step can be applied that pushes nodes toward the leaves, keeping source nodes before target nodes.

2.3 Layout optimization

We define the reticulate displacement of a left-to-right drawing as

where is the set of reticulate edges (but excluding transfer-acceptor edges, as introduced later). This depends solely on the y-coordinates assigned to nodes during the post-order traversal of the backbone tree B, which in turn depend on the order in which the children of each node v are visited.

Among all orderings of B, we propose to use one that minimizes the reticulate displacement. We refer to this approach as the displacement optimization (DO) algorithm.

NP-completeness. The decision version of the reticulate displacement minimization problem is NP-complete. The input consists of a rooted tree and a threshold k, and the question is whether there exists an assignment of child orders (i.e., a permutation of the children at each node) such that the resulting layout score is at most k. This problem lies in NP, since a candidate embedding can be described by a polynomial-size set of permutations and evaluated in linear time. Furthermore, the optimization version of the problem is NP-hard, implying that the decision version is NP-complete.

An instance of MinLA consists of an undirected graph with vertices, and the goal is to find a bijective assignment that minimizes the total edge cost

To reduce this to our layout optimization problem, we extend G to a rooted phylogenetic network N as follows: We introduce a root node and create an edge from to every original node . The original edges are considered reticulate edges. The new edges makeup the backbone tree B.

In this construction, the total reticulate displacement of the reticulate edges corresponds exactly to the MinLA cost function applied to the layout determined by the order of children below . Thus, any solution to our layout optimization problem for this network instance provides a solution to the MinLA instance, and vice versa. Since MinLA is NP-hard [19], it follows that minimizing total reticulate displacement in our problem is also NP-hard.

Heuristic optimization. To address this optimization in practice, we perform a pre-order traversal of B. For each node v that is the lowest stable ancestor (LSA) of some reticulate node, that is, the last node that lies on all paths from the root to the reticulation, we seek a permutation of its children in B that minimizes the total reticulate displacement. If the number of children is at most eight, we exhaustively consider all permutations. Otherwise, we perform a heuristic search by iteratively swapping pairs of children, using simulated annealing [20] to escape local minima (parameters: start temperature = 1000, end temperature = 0.01, 1000 iterations per temperature step, cooling rate = 0.95).

2.4 Transfer view

Let N be a rooted phylogenetic network on X. In the absence of additional information, we assume by default that every reticulate node is a combining node, to be displayed in the combining view.

A reticulate node v is called a transfer node if exactly one of its incoming edges is designated as the transfer-acceptor edge, and the remaining incoming edges are called transfer edges. (For example, in the extended Newick format, the transfer-acceptor edge is indicated using ##, rather than #, for the corresponding incoming edge.)

Transfer cladogram algorithm. To account for transfer nodes and their edges, we modify the construction of the backbone tree B as follows: Tree nodes and combining nodes are handled as in the standard approach. For each transfer node v, we retain only its designated transfer-acceptor edge and remove all other incoming edges. This adjustment ensures that any drawing of B avoids edge crossings among tree edges and transfer-acceptor edges.

We then compute the x and y coordinates as described for the combing view in Sect 2.2, with one modification. To encourage the source and target nodes of a transfer edge to share the same x-coordinate, we process each such edge as follows: if d(v) < d(w), we set and update the map d accordingly for all affected nodes. This process is repeated multiple times to account for cases where transfer edges mutually influence each other. Finally, we set for all nodes v in N.

2.5 Phylograms

So far, we have discussed drawing a cladogram, in which all leaves are placed on the right side of the drawing, all with the same horizontal coordinate 0, and any edge lengths, if given, are ignored.

To compute a phylogram of N, we assume that every tree edge or transfer-acceptor edge is assigned a non-negative weight . The algorithm described here can be applied to both the combined view and the transfer view [26].

Combine and transfer phylogram algorithm. We first use the approach described in Sect 2.2 to compute all y-coordinates of the nodes, followed by the layout optimization strategy from Sect 2.3.

To compute the x-coordinates, we assign the root the coordinate and then perform a breadth-first traversal as follows:

Let v be a node for which all parent nodes in N have already been processed. There are two cases to consider:

• Tree or transfer node: Then v has an incoming edge that is either a tree edge or a transfer-acceptor edge and we set .
• Combining reticulation node: Then v is a combining node with incoming reticulate edges , …, , and we set  +  δ, where δ is a small, fixed value that is used to ensure that reticulate edge always go from left-to-right.

2.6 Circular layout

A circular cladogram or phylogram can be derived from a left-to-right layout by converting y-coordinates, which range from 0 to n–1, into angles ranging from to , and by computing radii instead of x-coordinates, see Fig 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Circular layout.

(A) PhyloSketch circular combining view, network from [Fig 2, 22]. (B) PhyloSektch circular transfer view, network from [Fig S12E, 25].

https://doi.org/10.1371/journal.pcbi.1013805.g003

Optimization of the circular drawing can be achieved by optimizing the underlying left-to-right layout, as described above, using a slightly modified cost function: For each reticulate edge , the cost is defined as

where H is the number of leaves in B, plus 1. This accounts for the fact that in a circular layout, the shortest path between two nodes may go around the circle in either direction.

2.7 Equal spacing of leaves

The cladogram and phylogram layout algorithms presented above suffer from the deficiency that leaves may not appear equally spaced, due to the presence of unlabeled leaves in the backbone tree B of the network N.

To address this issue, once an optimal layout has been determined, we perform the following post-processing step: We repeat the post-order traversal of the backbone tree B to recompute y-coordinates. This time, we assign integer values to the labeled leaves and assign intermediate fractional, equally spaced coordinates to the unlabeled leaves. Some existing approaches fail to address this problem, as is evident in Fig 2C and in Fig 4C.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Cladogram, phylogram and IcyTree visualization.

A rooted phylogenetic network obtained by applying PhyloFusion [9] to 11 NADH dehydrogenase-associated gene trees of water lilies [27], is shown here as (A) a combining cladogram and (B) a combining phylogram, computed using the described algorithms. Note that the vertical spacing of leaves is uniform. In contrast, (C) an “ancestral recombination graph” visualization computed by IcyTree [15] exhibits gaps in the vertical spacing.

https://doi.org/10.1371/journal.pcbi.1013805.g004

3.1 PhyloSketch App

The PhyloSketch App is an interactive tool for working with phylogenetic trees and rooted networks. Trees and networks can be imported and exported using the extended Newick format. By default, reticulate nodes are interpreted as combining nodes; however, users can interactively declare transfer-acceptor edges, thereby reinterpreting the corresponding reticulate nodes as transfer nodes.

PhyloSketch also supports the interactive construction and editing of trees and networks. When a user draws a path using a mouse or touch screen, the software attempts to interpret the path as an edge, connecting its start point to a nearby node or an interior point of an existing edge, or its end point to another nearby node or interior point, if possible. A proposed path is accepted as a new edge only if it will not introduce a directed cycle into the network.

The program additionally provides a range of operations for modifying the topology of the tree or network, as well as tools for styling and labeling nodes and edges.

In Fig 5, we show that layout computations in PhyloSketch complete in acceptable time, although they are slower than in Dendroscope. However, the new algorithm produces layouts with substantially lower displacement values than the algorithm implemented in Dendroscope.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Comparison with Dendroscope.

Using randomly generated rooted phylogenetic networks with taxa and h = 0.2 × n reticulations (10 replicates each), we compared the performance of the new displacement optimization (DO) algorithm with that implemented in Dendroscope. (A) Wall-clock time (in seconds) on a MacBook Pro (M4 processor) to compute and optimize a combined rectangular cladogram. (B) Total reticulate displacement for both methods, normalized by the total height of the drawing.

https://doi.org/10.1371/journal.pcbi.1013805.g005

3.2 Network capture

The PhyloSketch App includes a workflow that assists users in capturing a rooted phylogenetic tree or network directly from an image (see Fig 2A and 2C). To capture a phylogenetic tree or network, the user loads an image into the program and then specifies the location of the root. The workflow then proceeds as follows:

1. The Tesseract library [28] is used to detect labels in the image.
2. A skeletonization algorithm [29] reduces all lines to single-pixel width.
3. All nodes (i.e., branching points and endpoints of lines) are detected using a pixel mask.
4. A flood-fill algorithm is applied to identify all paths connecting pairs of nodes.
5. Paths that lie entirely within the bounding box of a detected label are removed.
6. A root node is created at the chosen root location, splitting any nearby path if necessary.
7. The layout orientation, such as left-to-right, top-to-bottom, or radial, is determined by comparing the indicated root location with the positions of detected nodes.
8. Detected labels are associated with nearby nodes, taking the root location into account.
9. In order of increasing distance from nodes added so far, each remaining path is considered a potential new edge, and an attempt is made to incorporate it, as described in the previous section.

This approach assumes that the tree or network is drawn in dark colors on a white background, that edges are rendered as reasonably thin lines, and that nodes are small. Also, the presence of labels overlapping the the nodes or edges of the phylogeny, or additional lines indicating annotations, etc, will have a negative impact on performance. In practice, complex or poor-quality images will require preprocessing in third-party image software.

The capture of phylogeny from images will usually require editing by the user especially when reticulation edges cross other edges. Several post-processing steps are available to clean-up the captured phylogeny: “through nodes” (of in-degree and out-indegree 1) can be removed; nodes of in-degree two and out-degree two can be replaced by crossing edges; edge directions can be reversed; and sets of nodes can be merged into a single node while maintaining labels and connectivity.