Fig 1.
Schematization of the components of an evolutionary systems subject to variations.
The genotype determines the characteristics of the phenotype (solution). The continuous interactions between the phenotype and the environment produce a behavior (represented by the grey rectangular area). The fitness indicates the efficacy of the behavior. As indicated by the black arrows, variations might affect the phenotype (i.e. the internal environment), and/or the environment (i.e. the external environment or the relation between the agent and the environment), and/or the fitness evaluation. Adapted from Figure 8.3 included in [3].
Fig 2.
Illustration of the fitness surface of two hypothetical candidate solutions in an environment that varies with respect to a single parameter (e.g. an environment where the temperature varies in the range [0, 30] oC).
Assuming that the temperature varies with a uniform distribution, the expected fitness of the two candidate solutions corresponds to the average height of the two curves, i.e. 0.55 and 0.46 for candidate solutions 1 and 2. In general, the higher the number of evaluation episodes, the better the precision of the estimation. The circles in the left and right pictures show the fitness measured during 5 and 10 evaluation episodes, respectively, in which the value of the temperature was set randomly. The offset between the average value of the fitness obtained during the evaluation episodes and the expected fitness amount to 0.036 and 0.021, in the case of the left and right picture respectively.
Fig 3.
The double-pole balancing problem.
Fig 4.
Left: A screenshot of the TORCS graphic render. Right: The CG-Track1 used in the experiments and the trajectory of the best evolved controller (see Section 5.2). The color indicates the speed in km/h during one lap of the circuit.
Fig 5.
Top view of the environments and of the robots.
The black lines and the gray circle represent the walls and the nest, respectively. The smaller circles represent the robots. The red and blue semicircles inside the robot indicate the red and blue LEDs located on the frontal and rear side of the robot.
Table 1.
Performance of the best solutions post-evaluated for 500 validation episodes for experiments in which the environmental conditions vary with different frequency across generations.
F = 1 / G, where G is the number of generations after which the environmental conditions vary. Experiments conducted with NEE = 20. Each number indicate the average result of 40 replications. Numbers in brackets indicate the standard deviations.
Table 2.
Performance of the best solutions post-evaluated for 500 validation episodes for experiments in which NEE is varied from 1 to 1000.
F is set to 1.0 (i.e. the ENV matrix is re-generated every generation). Each number indicate the average result of 40 replications. Numbers in brackets indicate the standard deviations.
Fig 6.
Performance of the best candidate solutions post-evaluated for 500 trials.
Experiments performed with the best combination of parameters for each algorithm: SSS (F = 0.1, NEE = 20); CMA-ES (F = 0.04, NEE = 20); xNES (F = 1.0; NEE = 100); sNES (F = 1.0, NEE = 100); NEAT (F = 0.02, NEE = 20). Boxes represent the inter-quartile range of the data and horizontal lines inside the boxes mark the median values. The whiskers extend to the most extreme data points within 1.5 times the inter-quartile range from the box. “+” indicate the outliers. Data obtained by running 40 replications.
Fig 7.
Performance of the best candidate solution so-far across generations for different methods.
The X-axis indicate evaluations. Data obtained in the experiments with the best combination of parameters (see caption of Fig 6). Each curve indicates the average results of 40 replications.
Fig 8.
Fraction of times in which the best solution was generated during one of ten consecutive phases of the evolutionary process.
Each phase corresponds to 32 * 105 evaluations for a total of 32 * 106 evaluations. Data computed on the experiments performed with the SSS, CMAES, xNES, sNES, and NEAT methods with the best combination of parameters (see Fig 6). Each histogram shows the fraction of runs, out of 40, in which the best solution was found during the corresponding phase.
Fig 9.
Performance of the best candidate solutions post-evaluated for 25 trials.
Each experiment has been replicated 20 times. Boxes represent the inter-quartile range of the data and horizontal lines inside the boxes mark the median values. The whiskers extend to the most extreme data points within 1.5 times the inter-quartile range from the box. “+” indicate the outliers.
Table 3.
Performance of the best controllers evolved with the 5 methods and of the best human-designed driver available in TORCS v1.3.7 called “Inferno” (i.e. the human programmed controller that achieves the best performance on the CG1-track circuit, see http://torcs.sourceforge.net/index.php?name=News&file=article&sid=100).
All controllers, including Inferno, have been tested and evaluated in the same conditions.
Fig 10.
Performance of the best solution post-evaluated for 20 episodes.
Boxes represent the inter-quartile range of the data and horizontal lines inside the boxes mark the median values. The whiskers extend to the most extreme data points within 1.5 times the inter-quartile range from the box. “+” indicate the outliers. Data have been obtained by running 30 replications.