Fig 1.
DBR structure consisting of TCO multilayers.
The center frequency was designed as 500 nm with the refractive indices of TiO2 and SiO2 of 2.88 and 1.45, respectively. (a) Design according to TMM. The thicknesses of the TiO2 and SiO2 layers were 43.4 nm and 85.6 nm, respectively. (b) Structure designed using reinforced learning with an irregular arrangement of TiO2 and SiO2 layers. (c) Approximation of the structure in (b) as a 1 x N vector composed of zeros and ones for application in reinforcement learning. Here, TiO2 corresponds to 1, whereas SiO2 is represented by 0. (a) TMM Design, (b) RL Design, (c) State and Action.
Fig 2.
Schematic diagrams of dueling architecture and double architecture.
(a) The dueling architecture separates and processes the state value function V(s) and the action value function A(s, a), thereby enabling an agent to separately learn the relative values when a specific action is taken in each state and at each value. The final Q-value is calculated by adding the two functions. (b) DQN mechanism. Overestimation of the Q-value is prevented because the two independent Q-networks separate the selection of the action and estimation of the Q-value. This method prevents the reinforcement-learning agent from taking overly optimistic actions and stabilizes the learning process. (a) Dueling architecture, (b) DQN mechanism.
Fig 3.
Experimental and designed reflectance of DBR mirror sized 516 nm (500 nm Bragg wavelength).
(a) Effect of the number of learning steps on the reflectance. The reflectance of D3QN (blue line) converges faster than those of DQN and Random Search. The maximum reflectance of D3QN was also the highest. (b) Designed reflectance of four pairs (516 nm) based on the TMM. The reflectance designed with D3QN (blue curve) is superior to that obtained with TMM (black curve). (a) D3QN vs DQN, (b) D3QN vs TMM (516nm).
Fig 4.
Designed reflectance of DBR mirrors with sizes of 645 and 387 nm (500 nm Bragg wavelength).
(a) Designed reflectance of 645 nm size corresponding to five pairs based on the TMM (black curve). The designed reflec-tance of D3QN (blue curve) outperforms that of the TMM. (b) Designed reflectance of three pairs (387 nm). The designed reflectance of D3QN (blue curve) is superior to that of the TMM. (a) D3QN vs TMM (645nm), (b) D3QN vs TMM (387nm).
Fig 5.
Designed reflectance for 500 nm Bragg wavelength.
(a) Effect of DBR mirror size (200—600 nm) on the reflectance. The 600 nm size mirror (red curve) has the best reflectance. (b) Effect of the number of pairs on the reflectance (two pairs (258 nm) to five pairs (645 nm). The reflectance of five pairs is the best.
Table 1.
Effects of mirror size (200—700 nm) and Bragg wavelength (400—900 nm) on the reflectance.
Table 2.
Comparative reflectance performance across different mirror sizes and Bragg wavelengths.
* the best results are highlighted.