eulerAPE: Drawing Area-Proportional 3-Venn Diagrams Using Ellipses

Venn diagrams with three curves are used extensively in various medical and scientific disciplines to visualize relationships between data sets and facilitate data analysis. The area of the regions formed by the overlapping curves is often directly proportional to the cardinality of the depicted set relation or any other related quantitative data. Drawing these diagrams manually is difficult and current automatic drawing methods do not always produce appropriate diagrams. Most methods depict the data sets as circles, as they perceptually pop out as complete distinct objects due to their smoothness and regularity. However, circles cannot draw accurate diagrams for most 3-set data and so the generated diagrams often have misleading region areas. Other methods use polygons to draw accurate diagrams. However, polygons are non-smooth and non-symmetric, so the curves are not easily distinguishable and the diagrams are difficult to comprehend. Ellipses are more flexible than circles and are similarly smooth, but none of the current automatic drawing methods use ellipses. We present eulerAPE as the first method and software that uses ellipses for automatically drawing accurate area-proportional Venn diagrams for 3-set data. We describe the drawing method adopted by eulerAPE and we discuss our evaluation of the effectiveness of eulerAPE and ellipses for drawing random 3-set data. We compare eulerAPE and various other methods that are currently available and we discuss differences between their generated diagrams in terms of accuracy and ease of understanding for real world data.


Introduction
Data is routinely generated and analysed. For instance, relationships between groups of genes are studied to understand biological processes, improve health care, find cures to illnesses, and solve problems in agriculture. To aid analysis, Venn diagrams are often used. Each data set is represented by a closed curve and each set relation is represented by one of the spatial relationships between the curves. Both the curves and their spatial relationships are often easily visible, as closed curves are processed preattentively and pop out as complete distinct objects [1], particularly when the curves are smooth and have good continuation [2]. Closed curves also aid set analysis due to the perceptual grouping principles of common regions [3] and closure [4].
A Venn diagram with n curves is referred to as an n-Venn diagram and its regions depict all of the 2 n different combinations of the curve overlaps. A Venn diagram can also be areaproportional, such that the area of each region in the diagram is directly proportional to quantitative data corresponding to the depicted set relation [5]. Size is processed preattentively [6] and is easily noticeable due to its pop-out effect [7]. Thus, a Venn diagram can easily depict the data set relations as well as their cardinality or other associated quantitative data. Small multiples of such diagrams can also facilitate the analysis of a collection of data sets for different attributes (e.g., Venn diagrams depicting overlapping disease symptoms for different countries [8]; [9,10]).
An informal study identified various area-proportional Venn diagrams in the world's most cited journals (e.g., Nature) [29]. Almost all of these diagrams have two or three curves and are drawn using circles. Most of those with three circles are misleading and depict the required data inappropriately, like Figure 1A-C. For instance: in Figure 1A, the region with value 45 is bigger than those with value 53 and 54; in Figure 1B, the region with value 25 is bigger than that with value 36; in Figure 1C, the region with value 3 is much smaller than that with value 4. In some cases, the generated diagrams do not depict all the required overlaps between the curves, as demonstrated in Section 4.3 and Section 4.4.
Such area-proportional Venn diagrams cannot be drawn analytically using a specific curve shape and so numerical methods or heuristics are required [30]. Circles can draw Venn diagrams with region areas that are proportional to any data with two sets [5], but not three [30] due to their limited degrees of freedom (i.e., a centre and a radius). Polygons can draw accurate areaproportional Venn diagrams for any data with three sets [30], but as shown in Figure 1D-F, their non-smooth and nonsymmetric curves are not easily distinguishable and impede comprehension [31,32]. Despite these problems, current drawing methods use either circles or polygons.
Ellipses have more degrees of freedom (i.e., a centre, two semiaxes, an angle of rotation) than circles and are similarly smooth. So diagrams drawn with ellipses are more likely to be accurate with respective to the required quantitative data and easy to comprehend due to their distinguishable curves. This is illustrated in Figure 2 where the diagrams accurately depict the quantities indicated by the numeric labels of the respective diagram in Figure 1. The diagrams in Figure 2 were drawn using our novel drawing method and software, eulerAPE, which is the first to use ellipses.
The benefits of ellipses was noted earlier (in 2004 in the first paper on area-proportional Venn diagrams [5] and later [29,30]), but ellipses were never adopted due to difficulties in calculating the area of the regions of overlapping ellipses and in adjusting the various properties of the ellipses [5]. Thus, eulerAPE is the first to Comparing the cell-type of differentially regulated genes after an anti-cancer drug treatment [11]. The method used to draw the diagram has not been noted. This diagram is a reprint of Figure 3B in [11], previously published under a CC BY license. (B) Summarizing prognostic indicators of severe malaria [12]. The method used to draw the diagram has not been noted. This diagram is a reprint of Figure 3 in [12] (with the N value and the percentages in parenthesis removed), previously published under a CC BY license. (C) Analysing differences and similarities between identified chicken egg white proteome in three different studies [13]. Drawn using Venn Diagram Plotter [http://omics.pnl.gov/software/VennDiagramPlotter.php]. This diagram is a reprint of Figure 2A in [13], previously published under a CC BY license. (D) Analysing overlaps between gene libraries [14]. Drawn using DrawVenn [5]. This diagram is a reprint of Figure 4B in [14] under a CC BY license, with permission from John Wiley and Sons, original copyright 2009. (E) Studying transcriptome variation of different tissue types of the male field cricket, namely testis, accessory gland and the rest of the body [15]. Drawn using Convex Venn-3 [51]. This diagram is a reprint of Figure 1 (left) in [15] under a CC BY license, with permission from Nathan Bailey, original copyright 2013. (F) Summarizing genes affecting Top1-induced DNA damage identified in three different studies [16]. Drawn using DrawEuler [55]. This diagram is a reprint of Figure 3A in [16] (with added numeric labels indicating the quantitative data that according to the article each region in the diagram should represent), previously published under a CC BY license. doi:10.1371/journal.pone.0101717.g001 eulerAPE: Drawing Area-Proportional 3-Venn Diagrams Using Ellipses address this well-known, difficult problem. It is also novel, in that it is neither an extension nor an adaptation of any other previous drawing method. The current applications of eulerAPE are extensive and diverse. Diagrams generated by eulerAPE have appeared in numerous journal articles from diverse disciplines (e.g., [33][34][35]). A New York Times Science article (http://www. nytimes.com/interactive/2013/04/16/science/disease-overlap-inelderly.html) cites eulerAPE, and the Pacific Northwest National Laboratory recommends it on their Venn diagram drawing software webpage (http://omics.pnl.gov/software/VennDiagram Plotter.php). This work is also the first to assess the effectiveness of ellipses in drawing accurate diagrams for the required set relations and associated quantitative data. We discuss the availability of eulerAPE in Section 3.5.
Our contributions include: (1) an optimization method to automatically draw accurate diagrams with ellipses comprised of (1a) a novel cost function to direct the optimization process (Section 3.2), (1b) a method to generate a rational starting (Section 3.3), and (1c) a mechanism to adjust the properties of the ellipses in search for a good solution (Section 3.4); (2) evaluation of (2a) the effectiveness of eulerAPE in drawing an accurate diagram when one is known to exist for the given 3-set data (Section 4.1), (2b) the effectiveness of eulerAPE and ellipses in drawing an accurate diagram for random 3-set data for which an accurate diagram drawn with ellipses might not exist and the comparison of these diagrams with those generated by a variant of eulerAPE that restricts the ellipses to circles (Section 4.2), (2c) the effectiveness of eulerAPE and venneuler [29] (the latest proposed circle-based method) in generating accurate Venn diagrams for 3-set data (Section 4.3), and (2d) the quality of the diagrams generated by eulerAPE and various other drawing methods that use circles or polygons in depicting real world medical data (Section 4.4).
All the experiments mentioned in this article were run on an Intel Core i7-3770 CPU @3.4GHz with 8GB RAM, 64-bit Microsoft Windows 7 Professional SP1 and Java Platform 1.7.0_10.
The latest proposed method, venneuler, is different from most others as it uses a statistical model for fitting an area-proportional diagram to the required quantitative data. The model is comprised of a normalized loss function stress (defined as the sum of squared residuals divided by the total sum of squares) and a mechanism to minimize the function. Compared to 3 Circle Venn [36] and a method VennMaster [47,48] that draws the diagrams with convex regular polygons, venneuler is the most effective with respect to both accuracy and generation time. However, venneuler still generates inaccurate and misleading diagrams for most quantitative data due to the limitations of circles.
An accurate area-proportional 2-Venn diagram can be drawn for any quantitative data using two circles a and b. This is fully constrained, as given a and b have radius r a and r b respectively and distance d ab between the centre of a and b, only one overlapping region area exists. When a circle c is added, the overlapping region of the 2-Venn diagram is split up into two and new regions are introduced. So a 3-Venn diagram is made up of three 2-Venn diagrams (i.e., 2-Venn ab , 2-Venn ac , 2-Venn bc ). Thus, an areaproportional 3-Venn diagram for the quantitative data indicated by the numeric labels in Figure 4A can be constructed by first drawing the accurate 2-Venn diagrams in Figure 4B (the numeric values in the regions indicate their current area), whereby two copies of circle c are drawn to ensure that both 2-Venn ac and 2-Venn bc are accurate. Then, rotate the left copy of c anticlockwise about the centre of a and the right copy of c clockwise about the centre of b. Only one circle c is required and thus, the two copies of c must be rotated until they overlap completely ( Figure 4C). At this point, the overlapping regions in a 3-Venn diagram are automatically formed ( Figure 4D). However, the resulting region areas ( Figure 4E) are unlikely to be the same as those required (i.e., the numeric labels in Figure 4A) and yet, no changes can be made to improve the accuracy of a region area without making others less accurate.

eulerAPE Method and Software
Our drawing method eulerAPE is based on the simple hillclimbing optimization technique to draw an area-proportional diagram with ellipses in a time that is relatively fast and that  Each of the quantities in the provided data, for which a diagram should be drawn, is first scaled by a factor of (100 / smallest quantity in the data), so the same diagram is generated for quantitative data that is different but proportional. The scaled quantity corresponding to a region is then the required area of the region in the diagram to be generated. Later, the search for a solution that satisfies our diagram goodness measure (Section 3.1) commences, so that a diagram with region areas that are directly proportional to the scaled quantitative data is generated. A cost function directs the optimization process to a good solution (Section 3.2), in that, starting with a rational diagram for the required region areas (Section 3.3), the properties of the ellipses are adjusted based on the cost of the modified diagram (Section 3.4). The software is online and free to use (Section 3.5).

The Diagram Goodness Measure
To verify whether the region areas of an area-proportional diagram are accurately and directly proportional to the required quantitative data, eulerAPE uses the following measure: If N v is the set of quantities for which a diagram had to be drawn, N d is an area-proportional diagram generated for v, N R is the set of labels describing the required set of regions interior to the curves of the diagram, N v (r) [R z is the quantity assigned to r[R that should be depicted by the area of r in d, and then the error in each r [ R is defined as In eulerAPE, e = 10 26 , which value is consistent with that of other methods when defining a value for zero in their implementation (e.g., venneuler [29]). The value of regionError and diagError is always in [0,1].
Rather than using the absolute region area, eulerAPE's measure considers the proportion of each region area to the area of the total diagram. An inaccuracy in one region could make other accurate regions or curves in the diagram seem erroneous, as regions and curves might be compared for their area to be estimated. This can be avoided by considering the area of the region with respect to the total area of the diagram. A similar measure to the one used by eulerAPE was considered by a previous drawing method [36].

The Cost Function
To obtain a good, accurate diagram with respect to the required quantitative data (as defined in Section 3.1), our optimization algorithm minimizes a cost function that takes into account the accuracy of the diagram as well as paths that could lead to a local minimum. In an informal experimentation, we observed that the cost function of most of the current methods, such as venneuler's stress [29] and Chow and Rodgers's 'idealistic' function [36], often drive the optimization to a local minimum, as the overall error of the diagram is reduced at the expense of diminishing a region to a point where it is close to non-existent and its actual-to-required area ratio is close to zero. In such cases, no further changes can be carried out otherwise the diagram would no longer depict all of the required regions. Following our observation, we devised the following novel cost function: If then the cost of d is defined as Thus, the cost of a diagram is the mean of the cost of all the regions in that diagram. The sum could have been used since this work focuses on 3-Venn diagrams. However, we used mean so this function could be used in other future algorithms for diagrams with any number of curves and overlaps.
A diagram is generated for the required region areas scaled (i.e., those in v9) and so, it is adequate to consider the absolute area of the regions. A similar approach was adopted by previous techniques, such as VennMaster [47,48]. If relative region areas are considered as in our goodness measure, the search could be restricted, particularly at the initial stages of the optimization when a good but non-refined solution is adequate.
The denominator A(r) for r[R in our function prevents the optimization from taking paths that reduce the overall error of the diagram at the expense of diminishing the actual-to-required area ratio of a region, leading to a local minimum. With our function, a region with a very small actual-to-required area ratio will have a very large cost and thus prevent the optimization from taking such paths.
Though our cost function in Equation (4) is non-dimensionless, it is still adequate as the provided quantitative data is scaled before a diagram is generated for this data. However, we still considered ways how to make Equation (4) dimensionless. The denominator A(r) for r[R could be squared as in or the numerator could be the absolute difference between the required and actual region area not squared as in However, these two dimensionless functions will have a different effect from that intended by our non-dimensionless cost function, as the cost of a region would be much smaller than that in Equation (4) when the actual region area is greater than one and much greater than that in Equation (4) when the actual region area is less than one. This means that large errors would be less easily identified and the actual region area would have a greater impact on the cost than that intended to identify regions with a very small actual-to-required area ratio. The cost of a region in Equation (6) is the same as that in Equation (5) except that it is not squared. Thus, if large errors in a region result in a smaller cost in Equation (5) than in Equation (4), then in Equation (6) the cost would be even smaller, making it more difficult to identify inappropriate paths. So, we expect Equation (5) and Equation (6) to be less effective than Equation (4), and for Equation (6) to perform worse than Equation (5).
To choose the most effective cost function for eulerAPE, we conducted an experimental comparison of the following cost functions: F1, venneuler's stress, using the source code of venneuler version 1.1-0, but with v9 (r) and A(r) for the required and the actual area of a region r[R; F2, the first component of Chow and Rodgers's 'idealistic' function, which is related to our regionError; F3, the sum of the squared relative error of the regions; F4, the maximum of the relative error of the regions; F5, the sum of the relative error of the regions; F6, our nondimensionless cost function Equation (4); F7, our dimensionless cost function Equation (5); F8, our dimensionless cost function Equation (6).
The cost function of the optimization algorithm in Section 3.4 was replaced by one of F1-F8 and used to generate diagrams (with the rerun option disabled) for two libraries of 10,000 random 3-set data items each: L1 with quantitative data for which an accurate 3-Venn diagram with ellipses is known to exist; L2 with quantitative data for which it is not clear whether an accurate 3-Venn diagram with ellipses can be drawn. The details and results of this evaluation are available in Micallef's PhD thesis [59] (Chapter 5 and Appendix A).
This evaluation and experimental comparison indicated that our non-dimensionless cost function F6 is the most effective in: N Generating good diagrams for quantitative data for which a good diagram is known to exist; N Converging to diagrams that have a low diagError even when a good diagram cannot be drawn with respect to the given data; N Identifying and avoiding paths that lead the optimization to a local minimum when the overall error of the diagram is reduced at the expense of diminishing the area of a region to a point where it is close to non-existent and its actual-to-required region area ratio is close to zero; eulerAPE: Drawing Area-Proportional 3-Venn Diagrams Using Ellipses N Taking the least amount of the time and iterations to generate a diagram, particularly for data for which a good diagram is known to exist; N Generating a large majority of the diagrams (97.3%, N = 20000) within a time (1 second) that ensures that the users' train of thought is maintained, and generating nearly all the diagrams (99.6%, N = 20000) within a time (10 seconds) that ensures that the users' attention is maintained.
The effectiveness of F6 over the other cost functions with respect to the generation of good diagrams, the diagError of the non-good diagrams, generation time and number of iterations was highly evident for the diagrams generated for the data in L1, but less evident for those of L2. The results for L1 could be more important than those for L2, as an accurate diagram with ellipses exists for all of the 10,000 data items in L1. There is 3-set data for which an accurate diagram cannot be drawn with convex curves [30] (and thus ellipses) and it is unknown how much of this data is in L2. This evaluation also demonstrated that all of the cost functions (F1-F5), except for those we devised (F6-F8), often direct the optimization to a local minimum as the actual-to-required area ratio of a region is reduced to a value close to zero. Thus the cost function should heavily weight regions whose area is very small with respect to that required, as done in F6-F8. However, as expected, our dimensionless cost functions F7 and F8 were not as effective as our non-dimensionless F6.
Following the results of this evaluation, eulerAPE uses our cost function F6, as given in Equation (4).

The Starting Diagram
The optimization process has to commence with a solution. This is often an arbitrary or an invariant solution. Both types of starting diagrams were considered for eulerAPE. The arbitrary starting diagram was a Venn diagram comprised of three ellipses whose properties were assigned random values. The invariant starting diagram was a Venn diagram with 3-fold rotational symmetry, comprised of three equally-sized circles and regions that were similar in size, except for those in only one curve that were around three times as much as the rest. As expected, both types of starting diagrams led to poor results in terms of generation time and diagram quality, as such starting diagrams do not take into account the data that the good solution must satisfy and are more likely to direct the optimization to a local minimum [60,61]. We wanted eulerAPE to be deterministic and so an arbitrary starting diagram was particularly inappropriate, as different diagrams would be generated for the same data every time the optimization is run (as in e.g., VennMaster [47,48]).
A rational starting diagram that is adapted to the required quantitative data is more effective, as it reduces convergence time and the likelihood of reaching a local minimum. Such a starting diagram is used by for instance venneuler [29]. The starting diagram used and generated by eulerAPE is drawn using three ellipses with equal semi-axes, so the ellipses are depicted as circles. An angle of rotation of 0, p/3 and 2p/3 is assigned to the three respective ellipses to ensure that the entire space of possible angles of rotation is considered during the optimization. Assigning an appropriate centre for the ellipses is difficult and yet important as this determines the accuracy of the region areas. The two largest required ellipses, e 1 and e 2 , in the diagram are chosen and an areaproportional 2-Venn diagram that accurately depicts the data corresponding to e 1 and e 2 and their overlap is drawn using Chow and Ruskey's bisection method [5]. This increases the likelihood that the overall starting diagram is close to that required, as an accurate area-proportional Venn diagram can be drawn with circles for any data with two sets [5] and the 2-Venn diagram with e 1 and e 2 covers a large portion of the starting diagram.
Changes to the ellipses during the optimization affect the area of the region in exactly the three ellipses. So a starting diagram that minimizes the error of this region seems helpful. To achieve this, the centre for the third ellipse e 3 is obtained by applying the bisection method in an interval along a line L, as shown in Figure 5. L is the bisector of the angle, y, between two lines, T 1 and T 2 , that are respectively tangents to e 1 and e 2 at i 1 (i.e., the upper intersection point of e 1 and e 2 ). The interval along L is (u, l) where u is a point that lies above another point l on L, such that, as shown by the faded blue circles in Figure 5, the centre of e 3 must be between u and l (but not equal to u or l) for e 3 to intersect each of e 1 and e 2 twice and form the seven regions interior to the curves of a 3-Venn diagram. As illustrated by the faded blue circles in Figure 5, the endpoints of the interval as well as any value that is not in the interval will generate a diagram that is not a Venn diagram.
Out of the starting diagrams generated for 10,000 random 3-set data items for which an accurate Venn diagram with ellipses is known to exist, 63% had diagError #0.05 (i.e., 5%). The value of diagError is always in [0,1] and so, this result indicates that the generated starting diagrams are relatively close to the required solution. Also, the time to generate the diagrams is 10 times less than the 0.1 second limit for an instantaneous response [62,63] with a mean of 8 milliseconds for the 10,000 diagrams.

The Optimization Algorithm
Our simple hill-climbing algorithm commences with a rational starting diagram and systematically adjusts the properties of its ellipses to minimize our cost function, until a good diagram with respect to the given quantitative data is obtained. Though simple and a local search, it rarely encounters a local minimum and if it does, our algorithm is capable of handling such cases and obtain a good solution whenever an accurate area-proportional 3-Venn diagram drawn with ellipses is known to exist for the given data (as shown in Section 4.1).
Our optimization algorithm is characterized by the following three parameters that determine how at every iteration, each ellipse e is modified in search for other possible solutions: N pc, the number of pixels by which one or both coordinates of the centre of e are modified to explore eight new centres for e-these are shown in Figure 6A, where the black ellipse and black point are e and its centre prior to any change, and the grey points are the eight new centres for e; N pab, the scaling percentage by which one or both semi-axes of e are modified to explore eight new semi-axes for e-these are shown in Figure 6B, where the solid black ellipse is e prior to any change and the dashed coloured ellipses are e with the eight new semi-axes; N ph, the number of radians by which the angle of rotation of e is modified to explore two new angles of rotation for e-these are shown in Figure 6C, where the solid black ellipse is e prior to any change and the dashed coloured ellipses are e with the two new angles of rotation.
Changes that lead to a reduced cost of the diagram are accepted. At the start, pc = 10 pixels, pab = 5% and ph = 2p/3. These values were chosen after the diagram generation for different 3-set data was observed for different pc, pab and ph values. If, at the end of an iteration, a property of the ellipses is not changed, the value of the respective parameter is reduced linearly (halved). This means that major changes only occur at the start when the search space is explored for a good yet non-refined solution. As the values of pc, pab and ph are reduced further, minor changes to the diagram are explored, so that the diagram is refined to the required good solution. This cooling schedule, similar to that used in the global optimization method of simulated annealing, reduces the likelihood of converging to a local minimum and the time required to converge to the good solution. We halve the parameters as we observed that faster cooling rates restrict diagram refinement, while slower cooling rates are computationally expensive. This is the algorithm for the diagram generation process given a set of quantities, v: Algorithm eulerAPE (v) Input: v is a set of seven quantities each corresponding to a region interior to the curves of a 3-Venn diagram Output: an area-proportional 3-Venn diagram drawn with ellipses for v and whether the diagram is accurate 1: d r a rational starting diagram with respect to v 2: if d is a good diagram for v by Equation (

44: loop
Step 38 is reached when a local minimum is encountered. To handle such cases, eulerAPE has a rerun option which when enabled, it reruns the optimization using starting values for pc, pab and ph that are 20% larger than those used in the previous run. eulerAPE then terminates either when a good diagram is found or when 10 reruns are completed and a good diagram is yet not found. In the latter case, the diagram with the lowest diagError out of the 11 generated diagrams is returned. We opted for a 20% increase in the parameter values following our information experimentation of different parameter values when we observed cases that could benefit from such an increase and extended exploration of the search space.

Availability and How to Use
The software executable and the Java source code are freely available under the GNU General Public License version 3 at www.eulerdiagrams.org/eulerAPE. The latest release of eulerAPE should be downloaded from the webpage and opened by clicking on the downloaded jar file. Three steps are then required to use eulerAPE: 1. enter the quantities to be depicted by the regions of the diagram-these quantities should be typed in manually, generated randomly or loaded from a file; if the cost of d by Equation (4)  Further details, example how to load the required quantitative data from a file or how to save the diagram, are available on eulerAPE's webpage. The latest release, v3.0.0, has been fully tested on Windows and Mac OS X, is locale-independent, supports command-line execution (details on eulerAPE's webpage), and exports diagrams in png and svg format as well as in textual format with details about the properties of the ellipses in the diagram.

Effectiveness of eulerAPE and Ellipses
To evaluate the effectiveness of ellipses in drawing accurate area-proportional 3-Venn diagram for given data, we first evaluated the effectiveness of eulerAPE in drawing good diagrams for drawable 3-set data, that is data for which a good diagram is known to exist (Section 4.1). Being able to handle such data means that eulerAPE can avoid and handle local minima and if eulerAPE cannot draw an accurate, good diagram for any  . Time and total number of iterations to generate good diagrams for data in L1. The log 10 (time in seconds) and log 10 (total number of iterations) taken to generate good diagrams for 9939 of the 10,000 data items in L1 during the first run (labelled as 'Run 1') and for 61 of the 10,000 data items in L1 during any of the one to a maximum of 10 reruns (labelled as 'Reruns'). doi:10.1371/journal.pone.0101717.g008 eulerAPE: Drawing Area-Proportional 3-Venn Diagrams Using Ellipses random data in our second evaluation (Section 4.2), then it is highly likely that a good diagram drawn with ellipses does not exist for that data. In this way, we were able to identify characteristics of 3-set data that are drawable with ellipses (Section 4.2). In our second evaluation, we also generated diagrams for the same random data using a variant of eulerAPE that restricts the ellipses to circles to identify whether in these cases an accurate diagram could be drawn with circles (Section 4.2). We then compared these results to the diagrams generated by the latest circle-based method, venneuler [29], for the same random data (Section 4.3). Finally, we compared the accuracy and the curve aesthetics of the diagrams generated by eulerAPE and various other drawing methods using circles or polygons for real world data in a medical application area (Section 4.4).
The error of the diagrams generated by eulerAPE and other drawing methods was measured by diagError in Equation (2) whose value is in [0,1]. Good diagrams are those that satisfy our diagram goodness measure in Equation (3) and are thus diagrams that depict all the required regions and have diagError #10 26 . In our experiments, the number of iterations and the time taken to generate the diagrams were also recorded.
This evaluation focuses on 3-set data that associates a quantity greater than zero to each of the seven regions interior to the curves of a 3-Venn diagram. Diagrams with region areas that are zero percent of the total area of the diagram can still be drawn with eulerAPE, but further evaluation is required in the future.
In this section, L1 and L2 refer to two libraries each with 10,000 sets of seven numbers greater than zero. The numbers of a set in L1 are the region areas of a 3-Venn diagram generated after random values are assigned to the properties of three overlapping ellipses. The numbers of a set in L2 are randomly obtained from a uniform distribution in the interval [1,10000]. The data in these two libraries is different from that used in the evaluation of different cost functions in Section 3.2.

For Drawable Data
Diagrams were generated with ellipses by eulerAPE for the 10,000 drawable data items in L1. The rerun option of the optimization algorithm (Section 3.4) was enabled to verify whether eulerAPE still draws a good diagram if a local minimum is reached in the first run.
By the first run, good diagrams were generated for 9939 of the 10,000 data items (i.e., 99.4%). Despite generating a non-good diagram for the remaining 61 data items (i.e., 0.6%), the diagError of these diagrams was relatively low (median 1.06610 24 , mean 2.38610 23 , minimum 1.02610 26 , maximum 3.09610 22 ) and 54 of them (i.e., 88.5%) had diagError #0.01. Good diagrams were generated for all of these 61 data items after the optimization algorithm was rerun. For the majority (38/61, i.e., 62.3%), a good  (Figure 7; number of reruns, median 1 and mean 2.1). Thus, with 99% confidence, these results indicate that for 99.2% to 99.6% of drawable 3-set data, eulerAPE draws a good diagram during the first run, and for 99.9% to 100.0% of the same type of 3-set data, eulerAPE draws a good diagram after one to 10 reruns.
When the optimization algorithm is rerun, more time and total number of iterations are required to generate a good diagram (Figure 8). Even so, the generation of the 10,000 good diagrams had an overall median and mean time of respectively 0.4 seconds and 2.5 seconds, and an overall median and mean number of iterations of respectively 32 and 273. Also, for 97.7% of the 10,000 data items, a good diagram was generated within 1 second (98.1% and 34.4% of respectively the 9939 good diagrams generated during first run and the 61 good diagrams generated during a rerun), and for 99.7% of the 10,000 data items a good diagram was generated within 10 seconds (99.9% and 62.3% of respectively the 9939 good diagrams generated during first run and the 61 good diagrams generated during a rerun). So, with 99% confidence, these results indicate that for 97.4% to 98.0% of drawable 3-set data, eulerAPE draws a good diagram within 1 second, and for 99.6% to 99.8% of the same type of 3-set data, eulerAPE draws a good diagram within 10 seconds. These results are important as a response time of 1 second ensures the users' train of thought is uninterrupted and a response time of 10 seconds ensures the users' attention is retained [62,63]. Figure 9A and Figure 9B illustrate (i) the good diagram obtained from (ii) the starting diagram generated for the data item in L1 that was equal to the set of region areas of (iii) a randomly generated diagram. These examples illustrate that whenever possible eulerAPE draws circle-like curves (e.g., the semi-axes of ellipses a, b and c in Figure 9Ai differ by 6.0%, 5.4% and 8% respectively). In other cases, elongated ellipses are required to accurately draw the desired region areas (e.g., in Figure 9Bi, the required area for the regions located in only one of the curves is large compared to that of other regions), but the curves are still highly symmetric and distinguishable in shape from the regions, thus facilitating diagram comprehension [32]. Also, the curves in the diagrams are often evenly distributed, thus increasing the likelihood that the curves are easily distinguishable.
The majority of the non-good diagrams generated during the first run had a low diagError and just needed further refinement. Figure 10Ai is an example of such a diagram generated during the first run with diagError = 6.51610 24 . Figure 10Aii is the good diagram that was generated after one rerun for the data in L1 obtained from the diagram in Figure 10Aiii. The area of region bc is 0.003% of the area of the total diagram and yet eulerAPE was still capable of accurately computing the region areas and generate a good diagram. Figure 10Bii is an example of a good diagram that was generated after the first rerun for the data in L1 obtained from the diagram in Figure 10Biii. As shown in Figure 10Bi, during the first run, the optimization was trapped in a local minimum as ellipse b approached the edge of ellipse c (making region c seem like it was made up of two regions). By rerunning the optimization algorithm, different paths were explored and Figure 10Bii was generated.
The results of this evaluation indicate the effectiveness of eulerAPE in drawing good diagrams for drawable data. So, if eulerAPE cannot draw a good 3-Venn diagram for a set of seven quantities greater than zero, each corresponding to a region in the diagram, then it is highly likely that a good 3-Venn diagram drawn with ellipses does not exist for that quantitative data.

For Random Data
Since eulerAPE can generate good diagrams for drawable data (Section 4.1), we used eulerAPE to evaluate the effectiveness of ellipses in drawing accurate area-proportional 3-Venn diagrams for any random 3-set data with values greater than zero. Diagrams for the 10,000 data items in L2 were generated using eulerAPE. The data in L2 is made up of random values and thus, it is unknown whether an accurate diagram drawn with ellipses exists for this data (i.e., whether the data is drawable). The rerun option of the optimization algorithm (Section 3.4) was enabled to ensure that a good diagram is drawn for all drawable data. Diagrams with circles are preferred and the most effective [32], so we also evaluated whether an accurate, good diagram can be drawn using a variant of eulerAPE that restricts the ellipses to circles for any of the 10,000 data items in L2.
Good diagrams drawn with ellipses were generated for 8607 of the 10,000 data items in L2 (i.e., 86.1%)-8372 after the first run (i.e., 97.3% of the 8607) and 235 after one to a maximum of 10 reruns (i.e., 2.7% of 8607). More than half of the 235 good diagrams (56.2%) were generated during the first rerun and only one was generated after 10 reruns, as the diagError of the nongood diagrams generated for these data items during the first run was relatively low (diagError in [1.51610 26 , 3.28610 22 ] with median 1.89610 23 and mean 3.77610 23 ).
None of the diagrams drawn with circles for the 10,000 data items in L2 were good, and the diagError of these diagrams was greater than that of the non-good diagrams drawn with ellipses (median, mean: 6.28610 22 , 6.73610 22 for circles; 1.65610 22 , 2.11610 22 for ellipses). With a 99% confidence, these results indicate that for 85.2% to 86.9% of random 3-set data, a good diagram can be drawn (using eulerAPE) with ellipses, and for 0.0% to 0.1% of random 3-set data, a good diagram can be drawn (using eulerAPE) with circles. There are 3-set data for which an area-proportional 3-Venn diagram cannot be drawn accurately using convex curves [30] and so, drawing good diagrams with ellipses for a large majority of the 10,000 random data items in L2 indicates great potential for using curves that are regular and smooth as circles, but more general and with more degrees of freedom like ellipses.
The time and number of iterations that were required for the generation of the good diagrams using ellipses were similar to those of our evaluation in Section 4.1 (this evaluation: medians 0.4 seconds and 35 iterations, means 1.9 seconds and 201 iterations, N = 8607). Non-good diagrams with ellipses required more time and iterations as the optimization algorithm was rerun a maximum of 10 times (medians, 4.0 seconds, 586 iterations; means, 25.9 seconds, 4417 iterations). Similarly, the diagrams drawn with circles required more time and iterations (medians, 3.2 seconds, 500 iterations; means 3.4 seconds, 529 iterations), as none were good.
The majority of the 10,000 diagrams with ellipses were generated within 1 second (84.1%-8405/8607 good, 0/1393 non-good) and nearly all with ellipses within 10 seconds (96.9%-8569/8607 good, 1119/1393 non-good). So similar to Section 4.1, with 99% confidence, these results indicate that for 83.1% to 85.0% of random 3-set data, eulerAPE draws a diagram with ellipses within 1 second, and for 96.4% to 97.3% of the same type of 3-set data, eulerAPE draws a diagram with ellipses within 10 seconds. Out of the 10,000 diagrams with circles, none were generated within 1 second, but 99.6% (9959/10,000) were generated within 10 seconds.
This evaluation also revealed that data for which an areaproportional 3-Venn diagram can be drawn with ellipses often has larger areas for the regions in only one curve than those in only two curves, and an area for the region in only the three curves that is typically similar to those for the regions in only one curve.

Comparison with Circles and venneuler
Using a variant of eulerAPE, our evaluation in Section 4.2 indicates that it is highly unlikely that there is 3-set data for which a good diagram can be drawn with circles. To verify this finding, we used the latest circle-based method venneuler version 1.1-0 to generate diagrams with circles for the 10,000 data items in L2. This method is the first to take a statistical approach and differs from eulerAPE in various ways. For instance, venneuler uses a numerical approximation method to compute the region areas and a steepest descent method with an approximate gradient to minimize its loss function stress. The accuracy of venneuler's diagrams was then compared with that of the diagrams generated by eulerAPE with circles and ellipses in Section 4.2.
For eulerAPE, a good diagram is a 3-Venn diagram with diagError #10 26 (Equation (3)). For venneuler, a good diagram is one with stress #10 26 . Thus, to compare the accuracy of the diagrams generated by eulerAPE and venneuler, we computed: stress for the diagrams generated by eulerAPE using venneuler's version 1.1-0 source code, but eulerAPE's analytic method to compute the region areas; diagError for the diagrams generated by venneuler using eulerAPE's source code, but venneuler's numerical approximation to compute the region areas.
None of the diagrams generated by venneuler for the 10,000 data items in L2 had stress #10 26 or diagError #10 26 . Thus, none of the diagrams were good according to venneuler's and eulerAPE's diagram error measures. Also, only 64.5% (i.e., 6453/ 10,000) of the generated diagrams depicted all of the required regions. The other 35.5% (i.e., 3547/10,000) had one or more of the required regions missing. Figure 11Ai and 10Bi are examples of the diagrams generated by venneuler with missing regions. Both diagrams had a relatively low stress (5.69610 24 and 3.17610 23 respectively), close to that of a good diagram (i.e., stress #10 26 ). However, Figure 11Ai was missing region abc (despite that its required area was larger than that of regions ab and ac and similar to that of region bc) and Figure 11Bi was missing region ac (despite that its required area was similar to that of region b). Such diagrams are more misleading than ones with inaccurate region areas, as besides showing incorrect quantities, not all the required set relations are depicted. In contrast, the diagError for these diagrams was not so low (1.16610 22 and 2.07610 22 respectively). Some of venneuler's diagrams also had aesthetic features that could impede diagram comprehension [64]. For instance, Figure 11Bi has two regions representing only b. These problems are not evident in eulerAPE's diagrams ii and iii in Figure 11, as the diagram goodness measure and other checks during the optimization disallow the generation of such diagrams. When venneuler generated a diagram with all the required regions, the diagram was often misleading, as the region areas were inaccurate due to the limited degrees of freedom of circles. Figure 11Ci is an example of such a diagram. According to the data for which this diagram was generated, region a had to be 2.1 times larger than region ab and 1.7 times larger than region ac. However, region a was smaller than both regions ab and ac. The stress of the diagram was low (stress = 4.27610 23 , diagError = 2.30610 22 ), but greater than that of Figure 11Ai and Figure 10Bi, despite that the latter had missing regions and were thus more misleading.
The diagrams by eulerAPE with circles (ii in Figure 11) had inaccurate and misleading region areas like those of venneuler, but all depicted the required regions. All of eulerAPE's diagrams with ellipses (iii in Figure 11) had the required regions as well as stress # 10 26 and diagError #10 26 and were thus considered good by both venneuler's and eulerAPE's error measures.
As shown in Figure 12A (for stress) and Figure 12B (for diagError), the majority of venneuler's diagrams had a lower stress and diagError than those of eulerAPE's diagrams with circles (a lower stress for 8675/10,000 diagrams; a lower diagError for 6234/10,000 diagrams), but a greater stress and diagError than those of eulerAPE's diagrams with ellipses (a greater stress for 9730/10,000 diagrams; a greater diagError for 9660/10,000 diagrams).
The differences between venneuler's diagrams and eulerAPE's diagrams with ellipses were expected due to the limitations of circles in generating accurate diagrams for most data [30]. None of venneuler's diagrams were considered good by stress and diagError, but 8529 and 8607 of the 10,000 diagrams generated by eulerAPE with ellipses were considered good by respectively stress and diagError (the difference between the percentages of good diagrams by stress and diagError for eulerAPE's diagrams with ellipse is not statistically significant-using R's pro.test with Yates' continuity correction disabled, x2 (1) = 2.48, p = 0.12).
The differences between venneuler's and eulerAPE's diagrams with circles could be less expected. A Friedman rank sum test for non-normal distributions and repeated-measure data revealed a significant effect of drawing method on stress ( x2 (1) = 5402.3, p, 2.2610 216 ) and diagError ( x2 (1) = 609.1, p,2.2610 216 ). Post-hoc tests using Wilcoxon tests with Bonferroni correction showed significant differences between venneuler and eulerAPE with a large effect size on stress (W = 1763624, Z = 280.50, p, 2.2610 216 , r = 0.57) and a medium effect size on diagError (W = 14730686, Z = 235.58, p,2.2610 216 , r = 0.25). So according to these measures venneuler's diagrams were more accurate than those of eulerAPE. However, while all of eulerAPE's diagrams depicted the required regions, 35.5% of venneuler's diagrams had missing regions and yet 83.5% of these diagrams had a low stress (stress,10 22 ). So, eulerAPE's diagrams could still be more helpful than those of venneuler as all the required set relations are depicted. Also, out of the 10,000 diagrams drawn by eulerAPE with circles, zero (i.e., 0%) had diagError #10 26 (Section 4.3), but 28 (i.e., 0.3%) had stress #10 26 (the difference between these percentages is statistically significant-using R's pro.test with Yates' continuity correction disabled, x2 (1) = 28.04, p = 1.19610 27 ). Thus, with 99% confidence, these stress results indicate that a good diagram with stress #10 26 can be generated with circles for 0.2% to 0.5% of random 3-set data by eulerAPE and for 0.0% to 0.1% of the same type of data by venneuler. The diagError of these diagrams that were considered good by stress was still relatively low and close to diagError #10 26 .
This evaluation also revealed that if the required areas for the regions in only one curve are around twice as large as those for the regions in only two curves, and the area for the region in exactly the three curves is larger or as large as the areas for the regions in only one curve, then it is highly likely that a close to accurate areaproportional 3-Venn diagram drawn with circles exists.
With respect to the time taken to generate each diagram, venneuler was faster than eulerAPE. The median and mean generation time of venneuler were 0.6 seconds each, with a minimum of 0.4 seconds and a maximum of 1.0 second. The median and mean times for eulerAPE to generate a diagram with ellipses were 0.4 seconds and 5.3 seconds and with circles 3.2 seconds and 3.4 seconds. This could be due to the various differences between venneuler and eulerAPE (e.g., the method to compute the region areas; the way the optimization is run and terminated after a maximum of 200 iterations). Despite this, eulerAPE generates more accurate diagrams than venneuler and within a time that ensures users' attention is maintained (Section 4.2).

Comparison with Circles and Polygons, and Various Drawing Methods
Area-proportional 3-Venn diagrams are used extensively in various disciplines to facilitate data analysis, but often the diagrams are more misleading than helpful due to the limitations of the curve shapes used by current drawing methods. We investigated this further using real world medical data obtained from a BMC Medicine journal article [37]. Diagrams for this data were generated using most of the current drawing methods (Section 'Current Automatic Drawing Methods and Software'). These were then analysed and compared with the diagram generated by eulerAPE using ellipses.
The selected article discusses the results from a web-based survey that assessed whether US trainees in family and internal medicine are aware of the complications, screening methods and therapy for chronic kidney disease (CKD). This survey data was comprised of sets A, B and C: trainees who claimed that secondary hyperparathyroidism is a complication of CKD (set A); trainees who screened by stage 3 of CKD (set B); trainees who commenced therapy or referred the patient to a specialist when parathyroid hormone (PTH) reached a level of PTH.70 ng/ml (set C). Diagrams with respect to v were generated using eight circlebased drawing methods (from those listed and cited in Section 2.1), namely: C1, Stata's PVENN; C2, Venn Diagram Plotter; C3, 3 Circle Venn; C4, a module in PatternLab for proteomics; C5, BioVenn; C6, Vennerable circles; C7, venneuler; C8, Google Venn Charts. Other diagrams with respect to v were generated using six polygon-based drawing methods (from those listed and cited in Section 2.2), namely: P1, VennMaster, with regular, circle-like polygons; P2, Vennerable triangles, with triangles; P3, DrawVenn, with rectilinear polygons; P4, Vennerable squares, with rectangular polygons; P5, Convex Venn-3, with 4-sided and 5-sided convex polygons; P6, DrawEuler, with irregular, nonconvex polygons. All the diagrams are available in Figure 3, together with the diagram generated by E, eulerAPE with ellipses. The design of each diagram (e.g., labels, legend, colours, outlines, background) is precisely the same as that generated by the drawing method. Curve labels were only added to C2, P3 and P6 as no labels or legend are provided with the diagram. The numeric labels in eulerAPE's diagram were added manually to illustrate how the diagram in the article would have looked like if it was drawn with ellipses.
The diagError was devised to compute the error of only those diagrams that depict all the required set relations, as diagrams with missing regions are more misleading than those with inaccurate region areas and should not be accepted altogether. Thus, Figure 3 shows the diagError of only those diagrams with seven regions interior to their curves as required by the data. For the other diagrams, the missing regions are noted. To calculate diagError using Equation (2), the region areas of the diagrams were computed using eulerAPE's analytic method for those drawn with circles and standard geometry formulae for those drawn with polygons.
In Figure 3, we note that all the diagrams drawn with circles including D (the diagram in the article) have inaccurate region areas and are misleading. For instance, region B (1% in v) is much larger than region BC (3% in v), region C (11% in v) is larger than regions AB (10% in v) and ABC (15% in v), and region A (25% in v) in most diagrams is larger than region AC (29% in v). Similar problems are also evident in C7, the diagram generated by the latest method venneuler. C3, generated by the first circle-based drawing method 3 Circle Venn, also has region ABC (15% in v) similar in area to that of region AC (29% in v). The same is evident in D, as D was generated by the method of C3. However, C8 is the most misleading and inaccurate, as region BC is missing and regions B and AB (respectively 1% and 10% in v) are much larger than regions C, AC and ABC (respectively 11%, 29% and 15% in v). With respect to diagError, the most accurate are C4 and C7 (diagError = 0.03), followed by C2, C5, C6 (diagError = 0.04) and C1 (diagError = 0.05), and finally C3 and thus D (diagError = 0.14). Due to the regularity and good continuation of circles, the curves are often easily distinguishable and identifiable. In a few cases (e.g., C3 and D), it is difficult to comprehend in which curves the regions are located. However, this is often down to design as, for instance, different unrelated colours are used for regions located in the same curve (e.g., C2, C3, C6).
In contrast, most of the diagrams with polygons are either accurate with diagError #10 26 , as P3, P5, P6, or have region areas that are less misleading than those of diagrams with circles, as P2, P4. The latter is true as for instance, consistent with v, region B is always the smallest and region AC is always the largest. The only diagram with missing regions (regions C and BC) is P1 generated by the non-deterministic method VennMaster. Since the curves are depicted as regular, circle-like polygons, Venn-Master has the same limitations as others that use circles. Though the diagrams with polygons are more accurate than those with circles, the curves are non-smooth. So the curves are not easily identified [65] and are less likely to pop out as discrete and complete objects [2]. Curve identification is particularly difficult when, for instance: the curves met at bending points, as P5 and P6; the curves are partially concurrent, as P2, P3, P4; the curves are non-convex, as P6. Such features impede diagram comprehension [32,64], making these diagrams accurate but not usable. Thus, the preference for the less accurate diagrams with circles instead of polygons.
Using ellipses, diagram E has region areas that are accurately and directly proportional to the quantities in v (diagError # 10 26 ). It is also easy to comprehend, as the curves are regular and have good continuation like circles. So ellipses can be more effective than both circles and polygons. This was also demonstrated with other real world data in Section 'Introduction', where Figure 2 illustrates the accurate and easy to comprehend diagrams generated by eulerAPE with ellipses as alternatives to the respective misleading diagrams drawn with circles in Figure 1A-C and the incomprehensible diagrams drawn with polygons in Figure 1D-F. Being the only method that uses ellipses, the effectiveness of ellipses could be the primary reason why eulerAPE is being used in various areas and why its diagrams are appearing in various journal articles (discussed in Section 'Introduction'). The design of the diagrams adopted by eulerAPE is also different from that of other drawing methods, as eulerAPE uses a heterogeneous channel-based approach [2] whereby different feature types (i.e., outline, colour, texture) that are perceptually processed in parallel are used. In this way, none of the curve designs fuse perceptually at overlaps, and the curves and the regions are easily identified.
Another area-proportional 3-Venn diagram for the same data sets but for the management of anaemia rather than secondary hyperparathyroidism (so set C was based on the haemoglobin level rather than parathyroid hormone) was included in the article, as shown in Figure 13. As explained earlier, Figure 13Aii (replica of Figure 3-D) is misleading due to inaccuracies in the region areas. Figure 13Ai could be more misleading as besides inaccuracies in the region areas (e.g., region B with 3% is larger than region AC with 4%; region A with 36% is larger than region AB with 41%), seven regions are shown when according to the data (i.e.,  Figure 13Bi and ii, we can easily note that though most participants claimed that anaemia and secondary hyperparathyroidism were complications of CKD (set A), timely screening (set B) was more common with (i) anaemia than (ii) secondary hyperparathyroidism, while commencement of therapy or referral to a specialist (set C) was more often delayed when diagnosis was based on the (i) haemoglobin level than on the (ii) parathyroid hormone level. Thus, Figure 13B could have been more effective than Figure 13A in raising awareness of the need for trainees to be provided with further guidelines in managing CKD.

Conclusions
We have described eulerAPE, the first automated method for drawing area-proportional 3-Venn diagrams using ellipses. Previous methods used either circles or polygons. Circles are smooth and generate easy to comprehend diagrams, but are limited as they cannot draw accurate diagrams for most 3-set data. Polygons are flexible and generate accurate diagrams, but their non-smooth curves produce difficult to comprehend diagrams.
Our evaluation indicates that using ellipses and eulerAPE, accurate area-proportional 3-Venn diagrams can be drawn for a large majority of random 3-set data (86%, N = 10000), far more than is possible with the circles that are highly preferred over polygons. So curves that are smooth like circles but more general like ellipses should be considered whenever a diagram cannot be drawn accurately with circles. This finding opens a wider research eulerAPE: Drawing Area-Proportional 3-Venn Diagrams Using Ellipses question as to whether curves with different degrees of freedom such as circles, ellipses, ovals, n-ellipses, regular n-gons and irregular n-gons could be considered progressively from the most specific to the more general until a curve type that generates an accurate diagram for the required region areas is found [30].
The results of our evaluation also indicate great potential for using ellipses to draw area-proportional diagrams with more curves. However, first, further evaluation should be conducted to assess the effectiveness of ellipses and a method like eulerAPE in handling 3-set data that requires an area of zero for various regions in the diagram. Following this, characteristics of different types of quantitative 3-set data that can or cannot be depicted accurately with an area-proportional diagram drawn with ellipses should be determined, and analytic methods that identify whether a diagram can be drawn accurately for the given data should be formalized.
Apart from the shape of the curves, diagram design features (e.g., colours, labelling strategies) can also facilitate or impede understanding of the diagram and the depicted data. The effect of such features and the possible benefits of adding interaction should be investigated. Other features that could aid understanding for users with different abilities (e.g., spatial and numeracy abilities) should also be identified.
A number of the studies could be conducted to understand: how such diagrams are processed perceptually and cognitively; how region areas are perceived; the effect of the shape of the regions and curves on area judgement; what discrepancies in areas are not noticeable; whether perceptual scaling measures like those proposed for map symbols in cartography [66], but highly criticized by Tufte [67], aid or hinder area judgement in these diagrams. The findings of these studies will aid in the identification of diagrams whose region area errors are not human detectable. In this way, an inaccurate diagram drawn with ellipses for the required data could be considered accurate for human use and perception and so, it could be drawn with ellipses rather than other more complex curves with less desirable features, such as irregular and jagged polygons. A study should also determine whether numeric labels in the regions could conceal errors in region areas, thus allowing the use of smooth curves.
Following these studies, aesthetic criteria, metrics and cognitive measures as well as perceptual and design guidelines defining an effective, good diagram for human use that facilitates comprehension and reasoning should be formalized and prioritized. A variant of eulerAPE should then be devised to optimize such measures, such that a diagram that is the best compromise between region area accuracy and aesthetics is generated. Such a diagram should ideally have all the important aesthetic features and none of its region area inaccuracies should be noticeable to the human user. This would be particularly important for data for which an accurate diagram with specific aesthetic features cannot be drawn. With such a compromise, an inaccurate diagram with smooth curves whose errors are not human detectable could be generated.
It might also be interesting to assess the effectiveness of allowing users to select aspects of the diagram that they consider important and they would like to optimize. Such aspects could include aesthetic features, such as the shape of certain curves or the accuracy of the regions.