From traditional to deep learning-based methods

Personalized recommendations in power marketing have attracted much attention due to the rapid development of smart grids and big data technologies [12]. Traditional methods, particularly collaborative filtering, infer users’ interests from their historical behaviors [13] but face significant challenges such as data sparsity and the cold-start problem, which severely limit their recommendation accuracy and effectiveness when data is scarce [14–16]. These issues are exacerbated in power marketing, where user behavior encompasses complex factors like product interest, engagement, and demand fluctuations, going beyond simple purchase history [5]. To overcome these limitations, deep learning models have been widely adopted. Deep Neural Networks (DNNs) and Convolutional Neural Networks (CNNs) excel at automatically learning features from data, reducing the need for manual feature engineering and improving recommendation accuracy [17–19]. Specific to power marketing, models like Long Short-Term Memory (LSTM) have been used to predict user electricity demand by treating consumption as a time series.

Graph neural networks and reinforcement learning for recommendation

Graph Neural Networks (GCNs) represent a significant advancement for handling the relational structure inherent in recommendation data. By modeling users and products as nodes and their interactions as edges, GCNs can effectively capture complex, high-order relationships within the user-product graph, leading to more accurate and generalized recommendations, especially in scenarios with complex user behaviors and large-scale data [20,21]. Concurrently, Reinforcement Learning (RL), particularly Deep Reinforcement Learning (DRL) algorithms like the Deep Deterministic Policy Gradient (DDPG), has emerged as a powerful tool for creating dynamic and adaptive systems. DDPG allows recommendation strategies to be optimized in real-time based on user feedback (e.g., clicks, purchases), enabling the system to adapt to evolving user preferences and market conditions for long-term effectiveness [22,23].

Research gap and our contribution

While machine learning and deep learning methods have been applied to power marketing [6,7], existing approaches often grapple with persistent issues like cold start, data sparsity, and a lack of dynamic adaptability [8,9]. Although GCNs and DRL individually offer solutions to some of these challenges, there is a lack of a unified framework that synergistically combines their strengths for the power marketing domain. Our work addresses this gap by proposing a novel integrated model. We leverage Multi-Layer Perceptrons (MLPs) for robust user behavior prediction [24–26], Graph Convolutional Networks (GCNs) to overcome data sparsity and model complex relationships, and the Deep Deterministic Policy Gradient (DDPG) for dynamic, long-term strategy optimization, thereby providing a comprehensive solution tailored to the intricacies of power marketing.

Overview of the model

In this paper, we solve the problem of user behavior prediction and personalized recommendation in power marketing. The model consists of graph neural network (GCN), deep deterministic policy gradient (DDPG) and multi-layer perceptron (MLP). Through multi-level data processing and learning strategy optimization, it provides accurate personalized recommendation in power marketing [27–29]. In the design of the model, we take into account the multidimensional characteristics of data and the complexity of user behavior to improve the accuracy, real-time performance and adaptability of the recommendation system.

The model inputs are historical user behavior data, personal information, market transaction data and electricity consumption data. The model firstly performs data preprocessing. The data preprocessing step includes removing missing values, handling outliers and standardizing and normalizing data. Standardizing and normalizing data can ensure the quality and reliability of the input data. Specifically, for user behavior data, the model extracts time, location and behavior type as features. These features provide rich information for personalized recommendation. Then, we input the above data into the MLP model for behavior prediction. By inputting user’s historical behavior data into MLP model, we can get the user’s deep features and further predict the user’s possible electricity consumption needs. Afterwards, we input the above data into the GCN to process complex interactions between users and electricity services. The interaction between users and electricity services can be seen as an interaction between nodes in a graph, and the users and electricity services are nodes. Through graph convolution, GCN infers users’ possible interests and electricity consumption needs based on historical interaction information between users and electricity products to improve recommendation accuracy. The introduction of GCN not only captures correlations between user behaviors but also effectively mitigates data sparsity and cold start problems. For new users or products with limited interaction history, the GCN leverages the graph structure to infer preferences by propagating information from neighboring nodes with similar attributes or rich interaction patterns. This allows the model to generate meaningful representations even for entities with sparse or no direct interaction data. Multi-layer graph convolution further enriches these representations by capturing higher-order connectivity information between users and products.

Based on the above, we introduce Deep Deterministic Policy Gradient (DDPG) algorithm to optimize recommendation strategies based on deep reinforcement learning. The role of DDPG in this model is to adjust recommendation strategies in real time according to user’s real-time feedback (clicks and purchases). Through a trial-and-error process, DDPG learns how to make optimal recommendations according to different user needs and market scenarios. The core link of this part is to use reward mechanism and policy gradient update mechanism in reinforcement learning. This allows the model not only to make predictions based on user’s historical behavior, but also to optimize recommendations according to user feedback in real time, which optimizes the long-term effectiveness of personalized services.

The output of the model is the list of recommended electricity products or services based on users’ predicted behavior and current market environment. And the recommendations are continuously optimized by DDPG algorithm. Recommendations are not limited to recommend some electricity products, but also some customized electricity services according to users’ electricity usage in the future, for example, electricity usage in some specific time period and energy-saving plans. These recommendations based on users’ future behavior can further improve users’ experience and business benefit.

In summary, the model in this paper takes MLP, GCN and DDPG to build a multi-level and dynamically optimized recommendation model. It can predict accurately based on multi-dimensional users’ behavior data, and optimize the personalized recommendation strategy through reinforcement learning. Compared with traditional recommendation models, this model can solve the problem of cold start and data sparsity. And it can adapt to the changes of users’ behavior in real time, providing more efficient and accurate electricity marketing services.

Fig 1 shows the overall architecture of our model. In this diagram, we combine various types of user behaviors and behaviors of electricity products, e.g., electricity consumption, user interest, choice of electricity products, and purchase behavior, to construct users and items. Each type of behavior represents the granularity shown in the diagram, and representations of behaviors at the same granularity level are weighted and fused into a higher-order representation. Our model captures the relationships between users and electricity products more comprehensively and offers more accurate feature representations and recommendation strategies for personalized recommendation. By representing various types of behaviors of users, our model can capture users’ needs more accurately and electricity consumption patterns more comprehensively, which will optimize recommendation results.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. Integrated representation and model structure of multiple behavior types of users and power products.

https://doi.org/10.1371/journal.pone.0340851.g001

User behavior prediction module

The user behavior prediction module is one of the modules in our model. Its goal is to predict the user’s electricity consumption demand in the future from historical behavior data of the user. In order to accomplish this goal, we use MLP model to automatically extract deep-level features from historical user behavior data and then predict behavior. In this section, we will explain the architecture, principle and formula of this module.

The inputs of user behavior prediction module include historical user electricity consumption data, user socioeconomic characteristic data (age, gender, region), and other information (market price, etc.). After the inputs are imported, the data will be fed into MLP model for feature extraction and behavior prediction. The module’s workflow is as follows:

1. Data Input Layer: Converts historical user behavior data into feature vectors, which are then fed into the MLP model. Each feature vector contains information from multiple dimensions, such as the user’s past electricity usage history, geographic location, and socioeconomic background. 2. Hidden Layer: The MLP performs nonlinear transformations on the input data through multiple hidden layers, capturing the complex relationships within user behavior. The hidden layer uses activation functions (such as ReLU) to enhance the model’s expressiveness. 3. Output layer: Outputs a prediction of the user’s future electricity consumption behavior, typically using regression to predict electricity consumption or other related behaviors within a specific time period.

The following Fig 2 illustrates the architecture of this module, showing how input data is processed through multiple hidden layers and ultimately outputs a prediction result.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. MLP module architecture diagram.

https://doi.org/10.1371/journal.pone.0340851.g002

In this module, the MLP model works based on the structure of a feedforward neural network. After the input data passes through the weighted and activated functions of each layer, it gradually approaches the final output. Assume that the input layer data is , where represents the input features and is the dimension of the input features. The calculation for each layer can be expressed as:

(1)

Where, represents the output of the layer, is the weight matrix of the layer, is the bias term, is the activation function (typically ReLU or Sigmoid), and is the output of the previous layer. The choice of activation function determines how the data in each layer is nonlinearly transformed, which is very important for capturing complex user behavior characteristics.

At the output layer, the MLP model uses regression to predict the user’s future behavior. Assuming the prediction target is the user’s electricity consumption over a certain period of time, the prediction formula for the output layer is:

(2)

Where is the output of the last layer, and are the weights and bias terms of the output layer, respectively.

To train the model, we optimize the parameters by minimizing the loss function. The loss function typically uses the mean squared error (MSE), as follows:

(3)

Where is the number of training samples, is the actual electricity consumption for the th sample, and is the model’s predicted electricity consumption. Through the backpropagation algorithm, the gradient of the loss function is propagated back to each layer of the network, adjusting the weights and biases so that the model can gradually reduce the prediction error.

To ensure model convergence and accelerate training, we use stochastic gradient descent (SGD) or the Adam optimizer to update the model weights. The update rule for the Adam optimizer can be expressed as:

(4)

Where represents the weight after the th update, and are the momentum and variance estimates of the gradient, respectively, is the learning rate, and is a constant to prevent the denominator from being zero.

During training, within this broader analytical framework, we employed what might be characterized as a batch training approach, purportedly dividing the training data into multiple mini-batches to substantially reduce memory consumption and ostensibly accelerate the overall training process. Subsequently, what appears to represent the training results of each individual batch typically undergo a forward and backward pass, leading to what tends to be an updating of the model parameters using gradient descent. What seems to emerge from these findings is that through what seems to constitute multiple iterations, the model appears to tend to suggest an approach toward what seems to be an optimal solution, typically tending to mitigate prediction error.

Given the complexity of these theoretical relationships, in an effort to ostensibly improve the model’s generalization capabilities and prevent overfitting, we also tend to employ what appears to represent various regularization techniques. For example, L2 regularization appears to tend to suggest a control over model complexity by adding what appears to be a weight decay term to the loss function, thereby seemingly hindering the overfitting of the model to the training data.

What this pattern seems to suggest, therefore, and what appears particularly significant about these findings within these evolving conceptual parameters, is that the MLP model appears to effectively learn what appears to be user behavior characteristics from electricity consumption data, seemingly providing reasonably accurate user demand predictions for the personalized recommendation module.

Personalized recommendation module

The personalized recommendation module appears to represent what might be characterized as the core component of this model, seemingly designed to provide users with what appear to be personalized recommendations for electricity products or services, typically based on their historical behavior and real-time feedback. Within this broader analytical framework, what seems to be a key approach involves combining a graph neural network (GCN) and a deep deterministic policy gradient (DDPG) model, ostensibly to optimize the recommendation strategy and generally improve the accuracy of the recommendation system. What this combination of two models tends to indicate is that it appears to provide a mechanism to process what often represents complex user behavior data and the intricate relationships between users and electricity products. What appears particularly significant about these findings is that this approach seems to facilitate dynamically adjusting the recommendation strategy, thereby allowing the system to predominantly adapt to diverse user needs based on real-time feedback.

What the investigation appears to indicate is that the workflow of what seems to be the personalized recommendation module typically involves two main stages. First, the GCN model appears to suggest what seems to be the learning of the interaction graph between users and electricity products, largely to capture user preferences and product characteristics. What also appears significant in this context is that, second, DDPG apparently optimizes the recommendation strategy through deep reinforcement learning, what seems especially noteworthy in this analytical context, allowing for the dynamic adjustment of the recommendation list, typically based on user feedback (e.g., clicks, purchases, etc.), considering the nuanced nature of these findings.

Simply, the GCN model converts the interaction information between users and electricity products into a graph form, where nodes are electricity products and users, and edges are the interaction information between users and products (such as click, purchase, etc.). Finally, the GCN model acquires the relationship between users and electricity products through graph convolution operation, and recommends accordingly. At last, DDPG model adjusts its recommendation strategy according to the features output from above graph based on user’s real-time feedback information through the reinforcement learning method, to make the recommendation system adapt to the user’s current situation in real time.

The entire module workflow is as follows: User historical behavior data and electricity product data are first processed by the GCN model to generate embedded representations of users and electricity products. The DDPG model then optimizes these embedded representations and real-time feedback to output a personalized list of recommended electricity products or services. The model structure is shown in Fig 3.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Personalized recommendation module architecture.

https://doi.org/10.1371/journal.pone.0340851.g003

GCN processes the interaction graph between users and electricity products and captures high-order relationships within it through graph convolution operations. Suppose that the interaction data between users and electricity products can be represented as a graph , where is the set of nodes, consisting of users and electricity products, and is the set of edges, representing the interactions between users and electricity products. Each user node and product node has corresponding feature vectors and .

In GCN, node representations are updated via graph convolution operations. The node representation of the layer is obtained by weighted summing the node representation of the previous layer and the representations of the neighboring nodes:

(5)

Where is the neighbor set of node , is the normalization coefficient, and are the weights and biases of the th layer, respectively, and is the activation function (typically ReLU).

Through multi-layer graph convolution operations, GCN is able to capture the complex relationship between users and electricity products, generating embedded representations of the user and electricity products.

After generating these embedded representations, the DDPG model optimizes the recommendation strategy through deep reinforcement learning. DDPG is a reinforcement learning algorithm based on an actor-critic architecture, applicable to continuous action spaces. In this model, the recommended electricity products or services serve as actions, and user feedback serves as reward signals. DDPG continuously adjusts the recommendation strategy through an exploration and exploitation mechanism.

Let the current state be , which represents the embedded representation of the user’s behavioral characteristics and electricity products, the action space represents the recommended products or services, and the reward signal be . DDPG aims to maximize long-term rewards by optimizing the recommendation strategy by updating the policy network and value network. The DDPG update formula is:

(6)

(7)

Where is the output of the current policy network, is the state-action value function, and is the discount factor, representing the weight of future rewards. By maximizing rewards, DDPG can continuously optimize its recommendation strategy, enabling the system to adapt to changing user needs and market conditions.

Ultimately, the combination of GCN and DDPG provides the core capabilities for personalized recommendations. The GCN model captures the relationship between users and electricity products, while DDPG optimizes the recommendation strategy through real-time feedback. During model training, GCN and DDPG are jointly trained and optimized, and the recommendation strategy is gradually adjusted to achieve more accurate personalized recommendations.

(8)

Where, is the final recommendation list output by the model, representing the electricity products or services recommended to the user.

Within this broader analytical framework, what emerges during training is that the GCN first tends to suggest what appears to be a capture of what might be characterized as the inherent relationships between individual users and distinct electricity products. What appears to follow from this, therefore, is that DDPG seems to generally indicate its performance of reinforcement learning to appears to provide evidence that may support the optimization of what appears to represent an effective recommendation strategy. What this tends to indicate, within these evolving conceptual parameters, is that during training, the model predominantly appears to lend support to what may represent an adjustment of the weights of the graph convolutional layers and tends to point toward what appears to be the optimization of the policy network in DDPG, what seems to aim toward what appears to be the maximization of long-term rewards. What seems especially noteworthy in this analytical context, and what the investigation appears to indicate, is that in light of these methodological considerations, the Adam optimizer appears to be employed for what tends to suggest effective gradient updates, ostensibly enhancing both training speed and overall model performance.

Data preprocessing and analytical techniques

Experimental setup

To evaluate the performance of the proposed power marketing user behavior prediction and personalized recommendation model, we selected two public datasets for experiments: the PJM power market dataset and the power consumption dataset. These datasets not only provide user behavior data on power consumption but also contain rich market price and load information, facilitating in-depth analysis of user behavior and the effectiveness of personalized recommendations.

After data collection and import, data preprocessing was performed. This preprocessing step primarily includes data cleaning, missing value handling, outlier detection, and data normalization.

1. Missing Value and Outlier Handling: Some data may contain missing values, particularly in historical power consumption data. To minimize the impact of missing values on model training, we used interpolation methods (such as linear interpolation) to fill in missing values. For outliers, we used box plot analysis, setting upper and lower quartile thresholds to remove data that significantly deviated from the normal range.
2. Data Standardization and Normalization: Because the feature ranges in the input data vary widely, especially between electricity consumption and market prices, we standardized and normalized all numerical features. Specifically, we used standard deviation normalization (z-score normalization) to transform each feature into a distribution with a mean of 0 and a standard deviation of 1.
3. Feature Selection: During feature selection, we combined user behavioral characteristics (such as their electricity consumption history, usage time, and consumption preferences) with market characteristics (such as electricity market prices and load). We used correlation analysis and principal component analysis (PCA) to select the features that have the greatest impact on electricity consumption prediction and recommendations, and removed redundant features to improve model training efficiency.

All the experiments are conducted in a Python 3.x environment. The PyTorch and TensorFlow are adopted to implement and train the deep models. We implement the neural network using PyTorch(MLP, GCN) and implement the reinforcement learning using TensorFlow (DDPG). NVIDIA GPUs with CUDA are used to accelerate the training procedure.

Pandas is used to store and manage the data. The graph neural network is implemented based on DGL(Deep Graph Library), which is designed to deal with the big graph data and could efficiently process the user-electricity products interactions.

Rationality and reliability of the experiments are guaranteed by above setting. To mitigate the computational demands of the integrated model during our experiments, we employed several optimization techniques. These included graph sampling for the GCN to reduce neighborhood aggregation complexity, experience replay with prioritized sampling for the DDPG agent to improve training data efficiency, and gradient checkpointing to conserve GPU memory during the backward pass. The PJM electricity market data set and electricity consumption data set are used as experimental data set. Combined with the precise evaluation metrics, we could evaluate the effectiveness of our model in electricity marketing and provide adequate data support for the subsequent experimental results.

The evaluation metrics were selected to comprehensively reflect the model’s performance from both business and technical perspectives, tailored to the power marketing context. Precision measures the accuracy of our recommendations, crucial for avoiding customer fatigue from irrelevant suggestions. Recall assesses our ability to capture all potential products of interest to a user, which drives exploration and adoption of new tariffs or services. The AUC-ROC provides a robust, threshold-independent measure of the model’s overall ranking capability, important for handling the varying consumption thresholds of different users. While the F1-score (which we also report) balances precision and recall, we analyze them separately to provide clearer diagnostic insights for system optimization. For behavior prediction, MSE and MAE (both reported) quantify the error in consumption forecasting, directly impacting the cost of misallocated energy resources. NDCG was chosen to evaluate the ranking quality of the entire recommendation list, as the position of a relevant offer (e.g., a time-of-use plan) can significantly influence user uptake in a marketing context.

Experimental results

User behavior prediction experiment.

This experiment is to verify the effectiveness of MLP in the field of user behavior prediction. I use historical consumption data from PJM electricity market dataset and electricity consumption dataset to train the MLP to predict the future electricity consumption. Finally, we compared the MLP model with other traditional regression model, linear regression, decision tree regression, support vector regression (SVR), K-nearest neighbor regression (KNN), random forest regression, gradient boosting regression (GBR), XGBoost regression, LightGBM regression.

These comparison models represent different types of regression algorithms, including linear models, tree models, and ensemble-based models. This comparison clearly demonstrates the advantages of the MLP model in user behavior prediction, particularly in terms of the accuracy and generalization of electricity consumption predictions.

Table 1 shows the performance of different models in user behavior prediction. As can be seen, the MLP model outperforms other traditional regression models across all evaluation metrics, particularly in electricity consumption prediction, where it achieves higher accuracy.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Comparison of user behavior prediction experimental results.

https://doi.org/10.1371/journal.pone.0340851.t001

Table 1 shows the performance of different models in user behavior prediction. As can be seen, the MLP model outperforms other traditional regression models in all evaluation metrics, particularly in mean squared error (MSE) and R². Specifically, the MLP achieved a mean squared error (MSE) of 0.0322 and an R² value of 0.91, significantly lower than other models, particularly traditional linear regression (MSE of 0.0451 and R² of 0.84) and decision tree regression (MSE of 0.0398 and R² of 0.87). This demonstrates that the MLP better captures user electricity consumption behavior and provides more accurate predictions.

In terms of root mean squared error (RMSE), the MLP achieved a result of 0.179, which appeared lower than other models, particularly linear regression (RMSE of 0.213) and support vector regression (SVR) (RMSE of 0.208). What this particular pattern in the data appears to tend to suggest, from this particular interpretive perspective, is what seems to be the notably high accuracy of the MLP in the demanding context of electricity consumption forecasting. What also appears significant in this context is that the MLP also achieved a mean absolute error (MAE) of 0.141, ostensibly performing better than other models such as decision tree regression (MAE of 0.153) and support vector regression (MAE of 0.160). What this seems to generally indicate, what appears particularly significant about these findings, is what appears to be the MLP’s overall predictive accuracy.

Considering the nuanced nature of these findings, in terms of training time, the MLP model tends to exhibit a comparatively shorter training time (2.3 hours) than other deep learning models, while predominantly remaining within what appears to be an acceptable range when compared to other traditional regression models. For example, the training time for random forest regression is 3.0 hours, and for XGBoost regression is 2.7 hours. What these findings seem to point toward, what tends to emerge as theoretically important within this broader analytical framework, is what appears to represent the substantial computational efficiency of the MLP model, particularly when processing large-scale data.

To further verify the visualization of the experimental results, we present a comparison chart of the prediction results of the models listed in Fig 4. This chart provides a more intuitive view of the differences between the different models in predicting electricity consumption, particularly the advantages of the MLP model.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Regression models performance comparison (normalized metrics).

https://doi.org/10.1371/journal.pone.0340851.g004

According to above experimental process and results, we draw the conclusion that the MLP achieves the best performance in predicting user’s electricity consumption by extracting user consumption behavior from historical behavior sequence and offering accurate prediction, and the computational efficiency and training time performance of MLP is relatively good, so that it can be applied in data sets with large scale.

Personalized recommendation performance experiment.

The purpose of this experiment is to validate the effectiveness of the GCN model in personalized recommendation. As the electricity market and user behavior have become more complex, traditional recommendation methods (i.e., collaborative filtering and content-based recommendation) are hard to model the relationship between users and electricity products. Especially, when the data is sparse and there are cold start situations, the relationship between users and electricity products is difficult to reflect the user‘s preference and needs comprehensively. Hence, we attempt to use graph neural network (GCN) to apply user-product interaction data, and utilize the graph structure to reflect more user‘s preferences and needs.

In order to evaluate the effectiveness of GCN in personalized recommendation, we compared GCN with some classic recommendation algorithms, such as collaborative filtering (CF), content-based recommendation, and other popular recommendation algorithms. By comparing the performance of these methods on the same data set, we can better understand the advantages of GCN model in handling user-product interaction, especially in complex data environment.

Table 2 shows the comparison of different recommendation methods in personalized recommendation. Obviously, the GCN model is better than other traditional methods on all evaluation criteria, especially in precision and recall. GCN greatly enhances the recommendation system performance on the complex user-elec- tricity products interaction.

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Personalized recommendation performance comparison.

https://doi.org/10.1371/journal.pone.0340851.t002

In detail, the precision of GCN is 0.87, recall is 0.82, AUC is 0.91, F1-score is 0.84, and coverage is 0.92. While, the precision of collaborative filtering is 0.75, recall is 0.71, AUC is 0.85, F1-score is 0.72, and coverage is 0.80. Compared with content-based recommendation method, the precision is 0.78, recall is 0.74, AUC is 0.87, F1-score is 0.76, and coverage is 0.82. The GCN model can better reflect the complex relationship between users needs and electricity products.

The results in Table 2 show that the GCN model performs well across multiple evaluation metrics, including precision, recall, AUC, F1 score, and coverage. Especially in cold-start and data-sparse scenarios, GCN is able to effectively capture potential changes in user needs and provide more personalized recommendations. In contrast, traditional collaborative filtering and content-based recommendation methods, while performing well on some metrics, still struggle to match GCN in complex user behavior prediction and recommendation optimization.

To more intuitively demonstrate the recommendation performance of each model, Fig 5 shows a comparison of the AUC curves for different recommendation methods. As can be seen, the GCN model’s AUC curve consistently surpasses that of other models, demonstrating its superior classification capabilities across a wide range of user behaviors and market scenarios.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Personalized recommendation models performance comparison (sorted by AUC score).

https://doi.org/10.1371/journal.pone.0340851.g005

Through experimental results and visualizations, we validate the GCN model’s advantages in personalized recommendations, particularly when handling the complex interactions between users and power products. GCN effectively captures the underlying relationships between users and products, providing more accurate and personalized recommendations.

Reinforcement learning enhanced recommendation experiment.

What this particular inquiry appears to set out to explore is what might be characterized as the potential effectiveness of the Deep Deterministic Policy Gradient (DDPG) algorithm in what appears to be the optimization of personalized recommendation strategies. Within this broader analytical framework, conventional recommendation systems tend to predominantly rely on historical user data for what are essentially static recommendations. What seems especially noteworthy in this analytical context, however, is that as user needs appear to continually evolve, static recommendation strategies frequently seem to struggle to adapt effectively to what appear to be evolving changes in user preferences in real time, what appears to follow from this is what tends to represent decreased recommendation accuracy and, consequently, user satisfaction. What this pattern seems to suggest, therefore, is that this particular inquiry tends to introduce the DDPG algorithm, what appears to represent a methodological approach which appears to dynamically adjust the recommendation strategy through what seems to be reinforcement learning, what appears to represent a continuous optimization of recommendations based on real-time user feedback.

In the power marketing field, user needs are constantly changing and the environment is also dynamic. It is increasingly hard to satisfy the users‘ needs by using static recommendation methods. Since there is one feedback (click, purchase etc.) for each recommendation, reinforcement learning can adjust the system after each recommendation by learning from users‘ feedback, and enhance the long-term recommendation effectiveness. We attempt to prove that optimizing the recommendation strategy in each iteration by using DDPG can effectively improve the accuracy and real-time of personalized recommendation.

We compared the DDPG model with several recently proposed state-of-the-art recommendation methods, including Neural Collaborative Filtering (NCF), Matrix Factorization with Neural Network (MFNN), LightFM, BPR-MF (Bayesian Personalized Ranking), DeepFM, Wide & Deep, GRU4Rec, and Factorization Machines (FM). To comprehensively evaluate the performance of these recommendation methods, we selected multiple evaluation metrics, including precision, recall, AUC, F1-score, user satisfaction, NDCG (Normalized Discounted Cumulative Gain), and MAP (Mean Average Precision).

Table 3 shows the comparison results between the DDPG model and modern recommendation methods. As can be seen, the DDPG model demonstrates significant advantages across all evaluation metrics, particularly AUC, F1-score, and NDCG. DDPG optimizes recommendation strategies through reinforcement learning, significantly improving both recommendation accuracy and user satisfaction.

Download:

• PNG
  larger image
• TIFF
  original image

Table 3. Comparison of reinforcement learning optimization recommendation strategies.

https://doi.org/10.1371/journal.pone.0340851.t003

As shown in Table 3, DDPG has many advantages in all evaluation indicators, especially in AUC (0.94), F1-score (0.90), and NDCG (0.91). Compared with other modern recommendation algorithms, DDPG is still superior. For instance, NCF obtains AUC 0.91, and DeepFM gets AUC 0.92. Although the above models also get good performance, DDPG is still ahead of them in the accuracy of recommendation, especially in the long-term recommendation effectiveness and user satisfaction. It shows the great value of deep reinforcement learning in dynamic recommendation system.

In addition, DDPG only takes 2.3 hours to train, while the time of training DeepFM and GRU4Rec are 3.0 hours and 2.6 hours respectively. It shows that DDPG not only can keep a high accuracy, but also has a reasonable amount of training time. More importantly, the DDPG model can adjust dynamically according to the real-time feedback of users. It means that the model not only can work well in static data environment, but also can give users high-quality recommendations when meeting real-time user feedback in real application.

Moreover, DDPG model also achieved better performance than other recommendation algorithms in terms of precision (0.92) and recall (0.89) respectively. It shows that DDPG model can optimize the recommendation in multiple aspects. Specifically, it can enhance more correct recommendations, and achieve more users with needs. It has shown the benefits of reinforcement learning to adjust the recommendation strategy in real-time.

The experimental results have also shown that DDPG model can achieve better performance in dealing with complex data and meeting user needs change over time. To quantitatively evaluate its adaptive capability, we analyzed its performance during a simulated seasonal transition from summer to autumn, which typically involves a significant shift from cooling to heating energy demands. During this period, the DDPG-based model demonstrated a 32% faster recovery in recommendation accuracy (Precision and NDCG) compared to static models, as it dynamically reallocated recommendation weight from air conditioning-related plans to heating solutions based on evolving user interaction signals. This exemplifies the model’s effectiveness in fine-tuning strategies in response to behavioral shifts. The DDPG model can therefore provide a powerful and adaptive solution for personalized recommendation in power marketing and other fields.

In order to further verify the effectiveness of DDPG model and other recommendation methods, shown in Fig 6 are compared with AUC curve of different recommendation methods. As can be seen, the AUC curve of DDPG model is always higher than the other methods. It shows that DDPG model can provide more accurate and dynamic recommendations when dealing with complex user behavior and real-time feedback.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Comprehensive analysis of reinforcement learning recommendation models.

https://doi.org/10.1371/journal.pone.0340851.g006

With this experiment, we show the benefits of our DDPG recommendation algorithm. Compared with traditional static recommendation methods, DDPG optimizes recommendation strategy in a dynamic way and adapts to user behavior in real time. DDPG makes recommendation more accurate, recall, and user satisfaction. Through reinforcement learning, recommendation system is more flexible to adapt to the current user and market. By adding recommendation strategy optimized by DDPG, the recommendation system can adjust in real time, and the long-term user satisfaction will be greatly improved.

Comparison with industry benchmarks

To further validate the practical applicability of our proposed model beyond academic benchmarks, we conducted comparative analyses against two representative types of systems used in current power marketing operations. The experimental setup involved a retrospective A/B test on a historical dataset of 50,000 residential customers from a partner utility company, covering a 6-month period of marketing campaigns.

Experimental setup and benchmark systems:

• Benchmark A (Rule-Based Expert System): A production system used by our partner utility company, which segments customers based on monthly consumption volume (low/medium/high) and time-of-use patterns, then recommends pre-defined tariff plans or energy efficiency programs according to business rules.
• Benchmark B (Adapted E-Commerce Framework): A commercial customer engagement platform (Salesforce Marketing Cloud) configured for energy products, utilizing collaborative filtering with content-based features for product recommendations.
• Evaluation Metrics: We employed both business-oriented metrics (Conversion Rate, Click-Through Rate) and computational metrics (Precision, NDCG) to comprehensively evaluate performance.

As shown in Table 4, our model demonstrates significant advantages across all evaluation metrics. Specifically, it achieved a 126% higher conversion rate compared to the rule-based system and a 41% improvement over the adapted e-commerce framework. More notably, in cold-start scenarios for new energy products, our model’s ability to leverage graph-based relationships and multimodal data resulted in a 153% performance improvement over Benchmark A and a 73% improvement over Benchmark B.

Download:

• PNG
  larger image
• TIFF
  original image

Table 4. Performance comparison with industry benchmarks.

https://doi.org/10.1371/journal.pone.0340851.t004

The Adaptability Score, which measures how well each system maintains performance during market disruptions (e.g., sudden price spikes, extreme weather events), further highlights the value of our DDPG component. Our model’s score of 0.72 significantly outperforms both benchmarks, demonstrating its capability to dynamically adjust recommendation strategies in response to changing market conditions.

While direct comparisons with proprietary industrial systems present challenges due to data confidentiality and implementation differences, these controlled experiments demonstrate that our approach not only excels in academic metrics but also addresses critical limitations of current industry practices. The results suggest substantial potential for improving customer engagement and business outcomes in real-world power marketing operations.

Ablation studies

To analyze the influence of each model module (MLP, GCN, DDPG) on the final recommendation performance, we design the ablation experiments to remove each module one by one and analyze the change of model performance. In this experiment, we remove MLP, GCN, DDPG one by one and analyze the influence of each module on the recommendation performance, and we also remove different modules combination to eliminate more.

Table 5 clearly shows that removing any one module significantly impacts model performance. In particular, removing the DDPG module results in the most significant decrease in the model’s AUC and F1 score, demonstrating the crucial role of deep reinforcement learning in optimizing recommendation strategies, improving recommendation accuracy, and enhancing user satisfaction.

Download:

• PNG
  larger image
• TIFF
  original image

Table 5. Ablation experiment results.

https://doi.org/10.1371/journal.pone.0340851.t005

The ablation study results presented in Table 5 confirm that each component is indispensable for optimal model performance, and their absence leads to distinct failures in model behavior that directly impact user experience. Removing the DDPG component creates a static system that cannot adapt recommendations based on user feedback, resulting in persistent irrelevant suggestions that frustrate users. The absence of the GCN module severely degrades recommendation personalization, particularly for new users, by failing to leverage relationships within the user-product graph, leading to generic suggestions that miss opportunities for meaningful engagement. Eliminating the MLP component removes the model’s predictive capability, preventing proactive recommendations aligned with future consumption patterns and making the service appear less intelligent. The most severe performance drop occurs when the MLP and GCN modules are removed concurrently. The AUC metric plunges from 0.91 to 0.80, indicating that the synergistic effect of these two components is fundamental to the model’s efficacy.

In summary, the integrated architecture—combining MLP for prediction, GCN for relational inference, and DDPG for strategic optimization—is vital for achieving high recommendation accuracy, robust real-time processing, and effective user modeling. The significant performance loss from excluding any single module validates the advantage of this hybrid approach.

However, the integration of these distinct models also presented practical challenges. A primary difficulty was ensuring training stability, as the objectives of the supervised GCN/MLP (minimizing error) and the reinforcement learning-based DDPG (maximizing reward) could lead to conflicting gradient updates. To mitigate this, we employed a phased training strategy: we first pre-trained the GCN and MLP modules to obtain stable feature representations and a accurate behavior predictor, then froze them to train the DDPG agent, and finally conducted limited fine-tuning of the entire model with a low learning rate. This approach was crucial for achieving stable convergence. Furthermore, balancing exploration and exploitation in the DDPG component was critical in the power marketing context; an over-explorative strategy could degrade user experience with irrelevant recommendations, while an over-exploitative one might lead to policy stagnation. We addressed this by carefully designing a reward function that incorporated both immediate user feedback and long-term satisfaction metrics predicted by the MLP.

The key trade-off for the performance gains of this integrated framework is the increased computational overhead, which we managed through techniques like graph sampling and experience replay. Our training employed a phased strategy: first pre-training GCN and MLP separately, then freezing them to train DDPG, and finally fine-tuning the entire model with a low learning rate (1e-5) to avoid gradient conflicts. This approach, while computationally intensive (total training time: 18–24 hours), proved necessary as ablation studies confirmed the full GCN + MLP+DDPG integration outperformed all partial combinations, particularly in long-term user satisfaction and cold-start scenarios.