3.2.1. Pre-processing & traffic feature analysis.

The raw traffic signal dataset undergoes several pre-processing steps to make it suitable for training the TD3P-ITC system. The time-based features, such as the hour and day are extracted from the time-stamp. For feature scaling, min-max normalization is applied to ensure uniformity across different metrics and adaptively applies z-score standardization for a zero mean and unit variance as . The feature extraction includes normalizing traffic volume with the queue length calculated based on traffic volume and average vehicle speed . Fig 2a-Fig 2d shows important parts of traffic management that utilize advanced analytics of traffic features. The correlation heatmap shows the relationships among key traffic parameters, including vehicle counts, average speed, and weather conditions. These features are important for constructing an optimum state space for signal management systems. The accident incidence heatmap examines accident rates by signal status and time. It demonstrates how changes in signal phase affect accident frequency. The combined subplots also provide a comprehensive view of traffic patterns. The average speed of vehicles in different weather conditions is shown, while the other chart displays the number of vehicles on the road at various times of day. These assessments work together to help plan real-time adaptive traffic signal control, thereby enhancing safety and improving traffic flow. The research emphasizes the necessity of dynamic signal adjustments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Traffic Environment Analysis a) Feature Correlation b) Accident Rate c) Average Speed Across Hours and d) Traffic Volume.

https://doi.org/10.1371/journal.pone.0339207.g002

3.2.2. State space model.

The state space gives the current condition of the traffic environment at time including the key features that define the flow of traffic and environment at each intersection point. The state space is composed of the following elements using Equations (2) and (3):

(2)

In contrast to the overall network state, Equation (2) describes the per-intersection state. Factors such as ambient variables, vehicle composition, average speed, predicted wait time, 12-dimensional feature vectors, and traffic volume are present at the intersection. All of these vectors, one for each of the N = 5 intersections in the network, add up to the global state. In this scenario, the state vector is 60 bytes, and the overall state dimension is 12N.

(3)

Here, the normalized traffic volume at is given as acts as a primary indicator of traffic congestion, and a high volume directly contributes to a higher negative reward, incentivizing the agent to clear traffic. The inferred queue length at based on vehicle speed and count, it is given as , one-hot encoded signal at is termed as and represents the one-hot encoded weather that impacts traffic flow by reducing speed during rainy days, and fog increases the distance, scaled temperature, and scaled humidity. Table 2 summarizes the key hyperparameters used in the experiments to train the proposed TD3P-ITC framework, including network learning rate, exploration rate, prioritized replay, reward weighting, and convergence.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. State Space feature specification.

https://doi.org/10.1371/journal.pone.0339207.t002

The chosen properties ensure that control relevance, observability, and learning stability are all equal in the process. Optimizing a signal timing requires fundamental congestion indicators such as speed, queue length, and traffic volume. The sensitivity to stop-go vibrations and the safety risk can be represented by stop events and speed variance, eliminating the need to model actual collisions. Signal-phase encoding maintains deterministic phase control while also providing context-aware phase control. To estimate the effects of external disturbances on capacity and accident probability, we use environmental factors and vehicle composition to capture the heterogeneous discharge behavior. Pedestrian movement, turning ratios, and lane movements are not part of this model because they are either not included in the provided dataset, are highly intersection-specific, or would significantly increase the state’s dimensionality without being uniformly observable. They can employ their state representation in continuous-control reinforcement learning thanks to their exclusion techniques, which make it compact, scalable, and sensor-realistic.

3.2.3. Action vector representation.

Action function represents the traffic signal control decisions based on current traffic signal phases, including red, green, and yellow, along with the duration of green lights across multiple intersections. It is represented as a vector containing the green and yellow durations for the North-South (NS) and East-West (EW) directions. The agent learns to choose the optimal signal phase or timing to minimize congestion and waiting times. The inclusion of signal phases and green light durations is formalized as in Equation (4). The time-varying action is determined by adjusting the duration of the continuous green phase at time t based on the current signal phase state.

(4)

Where signal phases and the signal phase for the North-South direction can be red, green, or yellow, and similarly for the East-West direction. as signal phase decision. The green durations are are given for North-South and East-West directions, where indicates the minimum green light duration allowed, and as the maximum green light duration allowed to prevent excessive delays at the opposite intersection, the action space manages the length of the currently active green stage for North-South or East-West movement without optimizing discrete phase transitions. To ensure traffic safety, the signal controller enforces yellow phases and all-red clearing intervals automatically during all phase transitions. Each green-to-red transition begins with a yellow gap and complete red clearing before the opposing way is activated. The green time values are set based on traffic engineering and network saturation to prevent dangerous switching or severe hunger. All experiments limit the agent’s continuous action outputs to this permissible range, then apply the signal timing controller. Yellow and all-red phase safety restrictions are set outside the learning loop and not violated by policy. These two represents the durations that mitigate congestion and maximize throughput while considering the traffic volume at each intersection. Also, the represents the estimation of the expected penalty for taking in . The agent selects the signal phase that yields the best trade-off between traffic flow optimization and safety. The proposed agent does not perform discrete phase selection, it selects only duration. External factors drive the establishment and imposition of signal phase sequencing to ensure compliance with safety and regulatory standards. Only activities that are constrained by the length of the running green stage are considered by the TD3 policy to be part of the continuous action space. There is no discretization or mapping of continuous actions to phase decisions, as the continuous outputs are fed directly into the timing control variables. This ensures policy stability and compatibility with continuous-control reinforcement learning.

However, the TD3 control strategy is unable to learn discrete phase selection; it can only learn temporal modifications of continuous values. To ensure safety and practicality, a predetermined operating sequence controls the signal phases, and the actor network generates continuous control signals that indicate whether the active green interval should be extended or shortened. To ensure full compatibility with continuous-action reinforcement learning, the phase terms in the action formulation refer to the current context of active control rather than the policy-optimized choice variables. The TD3P-ITC performs action updates to adjust the green timings dynamically if queue at lane 1 exceeds a threshold, increase signal green time for lane 1. If throughput for lane 2 is high, reduce to avoid congestion in other lanes

3.2.4. Reward function.

The reward function helps guide the DRL model in making decisions that optimize traffic flow, including minimizing queue lengths, maximizing throughput, and preventing accidents. This reward function ensures that the agent not only optimizes traffic flow but also prioritizes safety by discouraging accidents and reducing congestion at intersections in the traffic environment given in Equation (5).

(5)

Where as the inferred queue length , and as inferred stops based on vehicle speed and count, the accident reported at is given as . The queue length at intersection at time , the traffic volume at intersection at time , with as the maximum acceptable number of stops. In the stop penalty function,denotes the number of stopped vehicles at intersectionat time , and the term represents the average vehicle speed. Here represents the risk proxy, not the trained probabilistic classifier output. It is deterministically derived using normalized queue length, mean speed variation, traffic volume, weather condition indicators, and the current signal phase at time t. The accident risk threshold determines the level of risk tolerance for safety penalties, which is a non-learnable hyperparameter. It uses visible traffic and environmental factors, such as normalized wait length, traffic density, speed fluctuations, weather conditions, and signal phase conflicts, to produce a closed-form risk measure. These traits are linearly concatenated and passed through a sigmoid function to generate a normalized value in [0,1] representing short-term accident risk, not a learning probability model.

The incentive weights (α (Queue penalty weight), β (Stop penalty weight), γ (safety/accident weight)) were determined through empirical sensitivity analysis rather than heuristic estimates. After factoring in congestion reduction, stop minimization, and safety penalties, a coarse grid search of permissible ranges was fine-grained to train stability and convergence rate with respect to policy strength. The chosen weights steadily improved performance without causing oscillatory signals or safety breaches. After identification, the weights were held constant across all intersections, traffic demand levels, and weather to ensure fair comparison and demonstrate the policy’s generalizability. Training and evaluation were not weighted per intersection or scenario-reward.

Fig 3a-3c reports that the accident acts as a primary reward signal and safety indicator for the PER mechanism. A value of 1 triggers a large negative reward, ensuring that the agent quickly learns to avoid accident-routine scenarios. The signal status from agents’ outputs determines the next signal phase, as transitioning from red to green indicates control over the environment. In this simulation, the agent adjusts the signal phases at a traffic intersection based on weather conditions, accident reports, and traffic volume. The three plots show how the agent’s choices affect traffic over time. For example, traffic volume varies with weather conditions and signal phases, peaking during rush hours, and accidents exacerbate the issue. Second, the average speed follows a similar trend, slowing down when there are accidents or adverse weather conditions, such as rain or fog. Finally, the reward function indicates how effectively the agent achieved these objectives. Higher payouts mean better traffic management and fewer accidents.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Traffic Control Analysis Over Time Across a) Traffic Volume, b) Average Speed, and c) Reward Function.

https://doi.org/10.1371/journal.pone.0339207.g003

TD3 trains a deterministic policy in an off-policy manner, where agents learn to explore on-policy features in the learning environment, with dynamic updates driven by various learning signals. The TD3 framework uses double Q functions via mean-squared Bellman error minimization, which differs from other learning models in how it updates the dimension of each action function. Adding the noise to each dimension of the function validates the penalty rewarded for learning and ensures that the target actions are taken. The target actions follow stability techniques to provide a valid action range and corresponding reward update.

3.2.5. Clipped double Q-learning.

The traffic signal control model is implemented by combining clipped double Q-learning and TD3 to optimize the agent’s decision-making, minimizing bias in Q-value estimates and stabilizing learning across traffic states during intersection analysis. TD3 utilizes two Q-functions, Q1 and Q2, and selects and evaluated at each decision point. The target Q-value for a transition is given as a sequential transition tuple, with the initial state , the executed action , intermediate reward , as the discount factor and the subsequent state and as a binary indicator for episode termination. The Q-value target is computed as in Equation (6),

(6)

Target policy ensures smoother policy updates and prevents overfitting to Q-values, which might lead to instability. With the gradual update of the target network, the action decisions are made by the actor, for instance, the green duration for signals, which allows for stability and avoids overreacting to transient traffic fluctuations, and as the noisy target action. To prevent the policy from exploiting sharp, high peaks in the Q-function, it is expressed as in Equation (7),

(7)

The function smoothes the decision-making process, helping the agent avoid overreacting to transient traffic changes. Here is a Gaussian noise and is a clipping bound, where is a random noise that smoothes and helps average out Q-value errors over similar actions. The term is the target action policy based on traffic features like weather and crash type, evaluated at the state . From the actor network for the state followed by implies the lower and upper bounds for current signal phase durations and traffic risk policies, such as safety measures.

Fig 4 illustrates the complete process of the TD3P-ITC framework, which is utilized to manage traffic signals in a manner that adapts to changing conditions. The first step is to generate traffic data and a state space from sensor inputs, including vehicle counts, queue length, and wait time. The TD3 agent uses a policy to choose actions that modify the timing of signals and then implements those changes. The experience collection phase collects new states and rewards, then places them in the PER buffer. After that, the agent’s experience is prioritized based on the TD error, and the critic and actor networks are then updated during training. The framework continually improves its signal control policy, which enables it to work more effectively over time.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. System interaction sequence flow of TD3P-ITC.

https://doi.org/10.1371/journal.pone.0339207.g004

3.2.6. Target network and delayed updates.

The target networks provide stability by slowly maintaining the rewards of both actor and critic networks that generate consistent learning target stages, which prevents the moving target problem in the earlier RL models using the soft update mechanism . Along with delayed policy updates, enhancing this stability involves updating the actor network every 𝑑 = 2 step while critics update continuously, allowing Q-value estimates to converge near their true values before policy modification. This plays an important role in traffic scenarios where premature signal changes can cascade through the network. The soft update rule for the target network is given by Equation (8).

(8)

The policy network or parameters of the target policy update and the current policy network is given as . The term and are the parameters of the target-function networks Q1 and Q2, with and are the parameters of the current Q-function networks, with as the update factor that controls the traffic. A small means the update is gradual, ensuring stable learning. The analyzes the minimum Q-value from two Q-functions given in Equation (9).

(9)

The term is the safety reward for safe driving behavior, 𝛾 as the decay factor controlling the importance of future rewards, and 𝑑 as a flag indicating whether the traffic incident has concluded (1) or not (0). The parameter is the two Q-functions are used to evaluate traffic risks.

3.2.7. Loss function.

For each Q function, calculate the predicted Q-values and the observed target value variations, to represent the traffic risk estimation using as target value, then square the differences to get the error terms for both Q-functions. For calculating the loss function, the squared errors over a from the PER is given in Equations (10–12).

(10)

(11)

(12)

The term is the batch drawn from the PER learning, which includes traffic incident data such as vehicle counts, crash severity, and road conditions. For each Q function, the error term is analyzed using the Bellman formulation. The target value incorporates and the min of the next pair. The error terms are then squared and to penalize larger differences more heavily, and the loss for each Q-function is averaged. The final loss is minimized using gradient descent to make the TD3P-ITC decision, thereby improving the traffic signal control policy and estimating traffic risk through experience-based learning.

3.2.8. Actor network phase.

The actor network update enables the Q-function to converge closer to its true value before the policy is updated, leading to more stable learning. The represents the policy network parameters, as the learning rate for the policy network, and as the objective function for the policy network, each primary network policy and Q function is paired with a target network in Equation (13).

(13)

Each primary network is replicated in a target network and updated incrementally, providing stable prediction targets during model training. The actor network takes the and outputs the representation, which indicates the signal durations. The critic network provides future-state representation for real-time traffic control.

The input traffic environment listed in Fig.5 provides the state for multiple intersections with varying traffic densities. The critic network calculates the Q-values and through two separate estimations, then fed into the TD error to prioritize the experienced replay buffers in high-volume traffic. The actor network generates the optimal signal timing adjustments using current and pairs in association with target updates and and current policy updates . The use of PER enhances learning efficiency by focusing on critical traffic events, and the target network stabilizes updates, allowing for accurate traffic signal optimization in dynamic environments.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Real-time traffic signal optimization across multiple intersections.

https://doi.org/10.1371/journal.pone.0339207.g005

The system minimizes waiting times, reduces congestion, and improves throughput across multiple intersections in real-time. The TD3P-ITC training framework is defined below.

Algorithm: TD3P-ITC Framework Training Procedure

Data Required: state space

Output: Optimized policy , and Q functions

Initialize

1. ,

2. and targets

3. , priorities

4.

5. for E = 1 to max_episodes do

6. Init from traffic environment

7. for do

8.

9.  Update in environment

10.  update

11.  Calculate TD-error using

12.  Store

13.  if then

14.   Sample batch with PER probabilities

15.   for each in batch do

16.    calculate

17.    Compute target

18. end for

19. Update critics: and

20.

21.

22.

23.    Update actor:

24.    Soft update targets:

25. end if

26. end if

27. end for

28. end for

29. return

3.2.9. Prioritization experience replay.

The use of the PER technique ensures that the most critical experiences, which significantly impact traffic flow, are replayed and used to update the learning model. The priority of an experience is usually based on the absolute Temporal Difference error where is the TD-error and is a small positive constant ensuring all experiences have sampling probability. This PER samples experiences based on priority; higher-priority experiences are those that significantly affect traffic flow, and signal timings are used to update the model. The TD error is calculated as in Equation (14).

(14)

Where guides the adjustment of the signal timings to minimize delays and congestion while maximizing throughput. The and estimated by critic one and critic 2, respectively. The sampling probability for each experience is proportional to its priority given as As a hyperparameter that controls the overestimation bias towards higher-priority experiences, using these prioritized samples enables the model to update its Q-values, ensuring it focuses more on the most critical traffic control experiences. This helps refine the learning process, especially for actions that have a significant impact on traffic flow, such as managing high congestion or adjusting signal timings during heavy traffic.

Priority is determined through proportional prioritization to implement Prioritized Experience Replay (PER). The priority of each transition is calculated as where is the temporal-difference error of the critic networks, =10–6, prevents zero-priority, and =0.6 is the parameter used to interact with the degree of prioritization. The priorities are updated immediately after each learning update, using new calculated TD errors. The transitions are sampled based on to avoid the bias caused by the uneven sampling, the sample loss of the critic is corrected with importance-sampling weights , where is the size of a replay buffer. Practically, it is the exponent of the correction term that is linearly annealed between 0.4 and 1.0 during training, and the weights are normalized by to stabilize the optimization, which allows a reasonable balance between high-error transition exploration and an unbiased value estimate.

In Fig 6, PER accelerates learning by prioritizing critical traffic experiences with high temporal-difference errors, ensuring the system focuses on high-impact scenarios, such as emergency vehicle passages or congested areas, through probability-weighted sampling. This notable improvement in learning efficacy is achieved when employing PER within the TD3P-ITC framework. The TD3P-ITC with PER consistently outperforms the TD3 model without PER, as evidenced by the higher cumulative reward increase across training episodes. The shaded green region in Fig 6 illustrates the improvement achieved with PER, demonstrating its effectiveness in prioritizing experiences for enhanced learning. This study demonstrates the efficacy of PER-based reinforcement learning for controlling urban traffic signals. The cumulative reward for TD3P-ITC rises more quickly to higher levels, which means that policy optimization is more effective during training. Its practical significance lies in its ability to improve real-time traffic flow while ensuring safety, making it a strong candidate for future ITS applications.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Impact of PER on learning performance.

https://doi.org/10.1371/journal.pone.0339207.g006

In summary, the unique TD3P-ITC for real-time traffic signal optimization across multi-intersection networks provides an effective control of real-time traffic signal decisions. For stable learning in dynamic traffic situations, the system uses PER, which prioritizes essential traffic scenarios with high temporal-difference errors to efficiently use experience. The system employs dual normalization algorithms to handle a 12-feature state space, incorporating normalized traffic volume, inferred queue lengths, signal phases, and environmental conditions for context-aware decision-making across various traffic patterns. By balancing queue minimization, stop reduction, and accident prevention with weighted penalty terms (α, β, γ), a multi-objective reward function optimizes signal timing to achieve both safety and performance. Dynamic action-space controllers adjust the green signal phase based on queue thresholds and throughput measurements to regulate traffic flow within the modular architecture.

4.2.1. Average waiting time.

The plot shown in Fig 8a-Fig 8d displays the variation in waiting times for each model under different traffic volumes, with the box showing the interquartile range and the median line indicating the central tendency. The markers represent the range of waiting times, and the outliers are indicated. The adjusted waiting times suggest that the combination of traffic flow dynamics and model performance is crucial for optimizing real-time traffic signals. The average waiting time for five different traffic signal control models (SMART, FPPO, D3QN, M2SAC, and TD3P-ITC) under four different traffic conditions: Low Traffic (50 veh/h), Moderate Traffic (300 veh/h), High Traffic (600 veh/h), and Very High Traffic (998 veh/h). The box color indicates the traffic state, and each boxplot shows the average wait time for a specific situation. The data is simulated with noise to make it more realistic, and the traffic-volume scaling factors ensure that waiting times increase as traffic density rises. The graphs show the median, IQR, and outliers, making it easy to compare how well each model did in different situations. This picture illustrates how the TD3P-ITC model compares to other models in terms of reducing wait times, particularly when traffic conditions change.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Average Waiting Time for Different Traffic Signal Control Models Across Various Traffic Scenarios a) Low Traffic (50 Veh/h), b) Moderate (300 Veh/h), c) High Traffic (600 Veh/h), and d) Very High Traffic (998 veh/h).

https://doi.org/10.1371/journal.pone.0339207.g008

4.2.2. Throughput efficiency analysis.

The TD3P-ITC model achieves superior performance across various weather conditions, intersection types, and vehicle types, as shown in Table 4, due to its novel integration of TD3 with PER, which adapts traffic flow and safety. The 4.0% improvement over existing models M2SAC, D3QN, FPPO, and SMART is calculated based on throughput enhancement. Variant 1: Weather Conditions shows that TD3P-ITC consistently outperforms other models, with a 4.0% improvement in sunny weather and better handling of rain and fog. Variant 2: Intersection Types shows that TD3P-ITC performs best on highways and at transit hubs, where it can handle significantly more traffic. Finally, Variant 3: Vehicle sorts shows that TD3P-ITC performs well across cars, trucks, and buses, and outperforms the other versions across all vehicle types. The TD3P-ITC model adapts more effectively and performs better across varied traffic conditions, as evidenced by higher throughput and improved management of environmental and traffic complexity. By employing stability techniques, the framework reduces the likelihood of overestimation. These processes enable TD3P-ITC to make more informed and stable decisions about traffic signals, particularly when traffic is complex. The PER enhances learning efficiency by prioritizing experiences with a significant impact, such as heavy traffic or situations where accidents are more likely. This helps people learn more quickly and adapt more effectively to various traffic conditions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Throughput comparison of traffic signal control methods.

https://doi.org/10.1371/journal.pone.0339207.t004

4.2.3. Queue length minimization.

The illustrated Fig 9a-Fig 9e shows the lengths of car queues at five distinct types of intersections: Residential, Commercial, Highway, Mixed-Use, and Transport Hub. It utilizes multiple traffic signal control algorithms, including TD3P-ITC, SMART, FPPO, D3QN, and M2SAC.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 9. Queue Length Analysis on Various Traffic Intersections: a) Residential, b) Commercial, c) Highway, d) Mixed-Use, and e) Transport Hub.

https://doi.org/10.1371/journal.pone.0339207.g009

The results show that TD3P-ITC outperforms other models at reducing queues, especially in high-traffic areas such as highways and transport hubs. The Q-value update rule ensures the model continuously refines its decision-making, resulting in more efficient traffic flow. The changes in queue lengths over time demonstrate how each algorithm responds to varying traffic conditions. TD3P-ITC keeps queues that are substantially shorter and more stable. These results align with the objectives of real-time traffic signal control research, which seeks to enhance traffic flow and mitigate congestion. From this analysis, a 22% reduction in peak queue lengths at the Transport Hub intersection and a 25% reduction at the highway intersection are demonstrated, while managing high-traffic environments by optimizing traffic signal control and minimizing congestion.

4.2.4. Training model convergence analysis.

During training, the agent receives rewards based on how effectively it reduces congestion, optimizes traffic flow, and minimizes vehicle waiting time. After several iterations, called training episodes, the TD3P-ITC model converges to the optimal signal timings expressed as for lane . The model selects optimal timing settings that maximize throughput, minimize queue lengths, and reduce average waiting times in intersection scenarios, thereby effectively enhancing real-time traffic flow. During the inference shown in Fig. 10a and Fig. 10b, the trained policy network provides real-time traffic signal control decisions for each intersection, dynamically optimizing signal timing based on observed traffic conditions.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 10. Training Performance Analysis: a) TD3P-ITC Model and b) Convergence.

https://doi.org/10.1371/journal.pone.0339207.g010

4.2.5. Analysis with baseline models.

Table 4 presents the evaluation of traffic signal control algorithms, examining the impact of SMART [18], FPPO [19], D3QN [22], and M2SAC [26] on key evaluation metrics, including wait time, throughput, queue length, and accident rate. The differences between the traffic signal control algorithms. p < 0.001**. These markers help determine how reliable performance increases are by examining factors such as wait time, throughput, and accident rate. In traffic signal control, lower p-values (e.g., p < 0.001*) indicate a higher likelihood that the algorithm is responsible for the observed performance benefits (e.g., shorter wait times, reduced congestion) rather than random chance. This helps ensure that the algorithm operates effectively and reliably in real-life traffic management.

The traffic simulator reduced simulated accident incidences by 17.9% compared to baseline controllers under the same traffic demand and environmental conditions. Accident counts are recorded when a simulator’s safety model flags a collision or close contact. This measure is based on the empirical number of accident incidents per episode averaged over evaluation runs, not cumulative reward or punishment magnitude. Table 5 was statistically tested using two-tailed independent t-tests on the average per-episode metrics, with n = 50 evaluation episodes per method. The degrees of freedom for the waiting time, throughput, queue length, and accident rate were df = 98 in all paired comparisons. The related test values ranged from t = 3.12 to t = 6.47, and the effect size (Cohen d) ranged from 0.62 to 1.28, indicating medium to large practical significance. A Bonferroni correction was applied to a series of multiple pairwise comparisons of the four baselines and 4 performance metrics (K = 16), yielding an adjusted significance level of 0.0031.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Statistical significance analysis (Two-tailed t-test Results).

https://doi.org/10.1371/journal.pone.0339207.t005

4.2.6. Complexity analysis.

The computational complexity analysis provides that the TD3P-ITC system achieves better performance with space complexity and inference time complexity of , enabling real-time operation with faster decision-making. The state dimension indicates the number of features per intersection with queue length, vehicle counts, waiting times, and current signal phases and duration. The action dimension suggests the number of control signal parameters for phase timing adjustments. With five intersection points, 20 state features, and 256 hidden neurons with layers , the memory buffer remains manageable. Training complexity scales per as batch size for training termed as during each training time step with PER sampling efficiency of with memories range as experiences. Communication overhead ranges from the number of intersections in the traffic network as for coordination to for centralized approaches. The heatmap in Fig 11 shows the performance of five traffic signal control models across key metrics, including waiting reduction, travel time score, flow efficiency, performance score, and learning stability. TD3P-ITC achieves the best waiting reduction (42.8%) as a cumulative performance gain across benchmark and travel time scores (97), but it also exhibits the lowest learning stability (0.023). On the other hand, M2SAC reduces waiting time by 15.2%. However, it is 27.6% behind TD3P-ITC in terms of waiting reduction and 3% behind in travel time score, utilizing the clipped double Q learning stability technique. In terms of flow efficiency, the existing FPPO and D3QN perform better in training, while TD3P-ITC does a better job at reducing waiting time and improving performance scores. The traditional method, SMART, has the worst results, with a 22.0% decrease in waiting time and lower scores in other areas. TD3P-ITC outperforms SMART in terms of waiting reduction (20.8%) and travel time Score (13.5%), indicating its effectiveness in optimizing traffic signals in real time.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 11. Performance summary of research models.

https://doi.org/10.1371/journal.pone.0339207.g011

In addition to the asymptotic complexity study, an empirical timing analysis was conducted to assess real-time viability and to infer recent deployment. The taught TD3P-ITC policy decides at each intersection in 3–5 ms on conventional GPU processors and in less than 20 ms on a CPU-only edge traffic controller model, which is suitable for real-time operation with a normal signal update rate (>1s). The offline training takes 3.2 hours to converge over 5,000 episodes; however, the online operation remains unaffected. The computational delay of inference scales linearly with the intersection number; therefore, lightweight actor networks can execute inference in real time with fixed dimensionality. The proposed system can be deployed on standard traffic management equipment, with central or offline learning and low-latency inference at runtime.

The continuous-control Markov Decision Process for traffic signal control reduces vehicle waiting time and queue length, and ensures traffic safety under dynamic, uncertain traffic conditions. Actor-critic algorithms are excellent for this challenge because the model directly optimizes continuous green-phase timings, unlike discrete-action models, which pick a time phase from a timing schedule. The bias toward overestimation in the continuous action space, introduced by clipping and updating, and the delayed policy updates in double Q-learning influence the use of TD3. The instability of value estimates causes oscillatory signal behaviour and cascading intersection congestion, making such qualities useful in traffic signal regulation. It is used with PER to balance traffic with high-impact, low-frequency events, such as congestion peaks and high-risk accidents. Sample transitions based on temporal-difference error to highlight states that have the greatest impact on network-level performance, thereby improving sample efficiency and convergence stability.

The state representation covers instantaneous traffic conditions and contextual elements that affect signal operation. Congestion is measured by inferred queue length and normalized traffic volume, while signal-phase encoding ensures policy-awareness during the learning phase. Weather affects vehicle speed and accident risk. Normalising features ensures numerical stability during training and prevents value-function approximation bias towards large inputs. To balance efficiency and safety, the rewarding function is a weighted multi-objective signal. Stop-related penalties and wait length reduce congestion and improve traffic flow, and accidents will deter hazardous signal layouts. This formulation avoids hard-coded traffic algorithms and allows the agent to learn adaptive trade-offs as it operates in the system. To guarantee scale consistency across rewards and safety breach penalties compared to efficiency advantages, weighting coefficients are established. The suggested controller is tested in a tiny simulated traffic environment with varied traffic, weather, and vehicle compositions over a multi-intersection network. In the same conditions, the simulated environment enables controlled analysis of policy behaviour and benchmarking against current methodologies. This research examines learning dynamics and control stability at a small scale across a few crossings; however, it lacks structural limits that would prevent its application to larger networks. The methodology defines a clear relationship among control goals, the learning architecture, and the evaluation protocol, enabling performance gains to be attributed to the proposed TD3P-ITC design rather than to implicit assumptions or heuristic adjustments.