Survey design

In order to infer the preferences of the end-users and alleviate the previously mentioned Cold-Start Problem (CSP), a socio-economic survey was conducted in 7 tertiary buildings across Europe [28]. The survey took place in two phases, before and after the pilot intervention. The survey structure was slightly altered in the second phase (we removed some questions) based on the first phase results. The surveys included questions regarding demographics and work and non-work-related energy efficiency, e.g. confidence in the use of technology, potential barriers for energy-saving behaviour or intentions to join pro-environment actions, resulting in 9 predictor variables (The full survey can be consulted in S1 File). For a full immersion on the data collected in both surveys, these are publicly available on Zenodo; namely, the pre-pilot dataset [30], with 303 responses, and the post-pilot [31] dataset, with 65 responses. However, as 368 samples were insufficient, we decided to extend the evidence by running a second iteration of the survey through the Prolific platform (https://www.prolific.co/). The new data collected presents an additional 375 responses from the same four countries. Furthermore, we used a similar screening methodology employed in the first survey during the GreenSoul project. By doing so, the analysis was expected to have more objective results as the two datasets are entirely independent and with around two years of difference. The final dataset with N = 743 with all the answers already coded can be found also in Zenodo platform [32].

Datasets

The two datasets used in this that were merged are:

• the GreenSoul dataset with data from 368 users, and
• the Prolific dataset, with data from 375 users beyond the context of the EU project.

The records in each dataset consist of ten “Persuasion features” (see next section), i.e. the self-reported answers to questions about the appropriateness of the persuasion principles, used as the target variables in the developed models. Besides, nine “User features”, i.e. demographic and energy-related behaviour questions used as the predictor variables. Table 1 offers a listing of the persuasive principles included in this study, along with a brief explanation. For the developed models, these are the values that the “principle” categorical variable takes. The user features can be also consulted in Table 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Persuasive principles/strategies.

https://doi.org/10.1371/journal.pone.0303214.t001

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. The nine most important features extracted from the dataset and their descriptions.

https://doi.org/10.1371/journal.pone.0303214.t002

Persuasion principles.

In the aforementioned surveys, ten persuasion principles (PP) have been used as targets or persuasion features. All ten principles were selected from a larger set of principles composed through an extensive survey of the literature [33–37]. Specifically, as also mentioned in [38], the persuasion strategies collected from the above literature were grouped into 15 initial common persuasion principles that correspond to general modes of thinking that affect people to be persuaded, as identified by the literature experts. The survey participants were asked to rank these principles according to their perceived applicability to them. The ten PPs that are selected for the current study were the most influential for the end-users out of the original 15, based on their applicability, and were used to design and evaluate the proposed methods. Furthermore, the final ten principles were validated by two experts with a strong background in persuasion and nudge theory. The selected PP are the following: 1) Authority, 2) Cause and Effect, 3) Conditioning, 4) Physical Attractiveness, 5) Reciprocity, 6) Self-monitoring, 7) Similarity, 8) Social Proof, 9) Social Recognition, and 10) Suggestion. These PPs are also grouped into two clusters: 1) based on the interaction with a system, or 2) based on the interaction with other people. The objective is to examine the main driver behind different types of user profiles. Thus, to understand if people are more oriented to 1) a PP that requires more cognition and interaction with the system (central route of persuasion [39]); 2) or if the people prefer to be more influenced by peers as social actors (normative behaviour [40]). It is worth mentioning that the statistical analysis of the top 10 preferred principles by all the users in the full dataset was: “Conditioning”, “Social-recognition”, “Physical attractiveness”, “Self-monitoring”, “Reciprocity”, “Social-proof”, “Cause and Effect”, “Similarity”, “Suggestion” and “Authority”. See Fig 2.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. A boxplot of the top-rated persuasive principles in the dataset.

The medians are represented with diamonds. These PPs are represented in descending order.

https://doi.org/10.1371/journal.pone.0303214.g002

Pre-processing & feature selection

The input data features consist of the answers of the users to the energy-related questionnaire. Each row of the data corresponds to a user and each column to a questionnaire answer. The data are a mixture of ordinal and categorical variables, which are handled differently. Ordinal variables, such as age and education level (on a scale from 1 to 5), are directly transformed to numerical. Variables with different ranges for the scales, such as rankings from 1 to 10, are linearly transformed to the 1–5 scale.

A series of initial transformations were made to limit the number of original features (37) to a smaller number of features that contain most of the information for the users and are few enough to be easier to collect in future endeavours. The first transformation was performed towards ensuring inter-rater reliability. The features that provide the least amount of information about the users and require more effort from users to answer were removed. With this initial transformation, the user feature vector’s dimensionality was reduced from 37 features to 21.

Subsequently, feature selection algorithms were applied to detect the importance of each user feature in the classification and reduce the dataset’s dimension. This transformation’s objective was to identify the minimum subset of information that would represent a user to generate the best persuasive principle for each user while balancing the size of the dataset and the information loss.

Some details about the feature extraction methods used are as follows. Decision trees [41] were used as feature selection algorithms for this experiment as they have proven to be suitable for this task [42]. Two different classifiers, Random Forest and Extra Trees, were employed 10.000 epochs and 100 estimators, results were averaged and compared to decide which user-input features provide the most information for selecting the top principle of persuasion (i.e., a classification task). After this transformation, the user feature vector was reduced to the nine (9) most important user features shown in Table 2. We decided to maintain the features that provide 80% of the information regarding the users in the dataset as stated in [44]. This dimensionality reduction process also led to the Prolific survey’s design, which included only these nine most crucial user features as questions.

As a result of the feature selection process, nine predictor variables have been extracted, consisting of the user answers regarding demographics and behavioural aspects as can be observed in the final survey design in S1 File. These variables will be used as predictor variables by the models to predict the rankings and scores of the persuasion principles.

The ranks produced by the first model of the ensemble are also used as scores, by linearly transforming them to the 1–5 scale. The ensemble’s goal is to use the user profile, built from demographics and energy-related behaviour at work, to predict the rankings and scores and thus suggest the most appropriate persuasion principle for a user.

Proposed method

Based on the structure of these datasets, two different approaches, originally developed by the authors using preliminary data in previous publications, have been followed to identify the appropriateness of each persuasion strategy per user, each resulting in its recommendations, subsequently, these two sets of recommendations are merged in a step-by-step fashion, resulting in aggregated results. The first method [14], called here “Persuasion Principle Optimal Ranking”, is based on an extended version of matrix factorization (MF) [45] through Factor Machines based on the library LightFM [46]. Specifically, our approach extends traditional matrix factorization (MF) by incorporating latent factors that represent both users and items. For users, these latent factors are derived from socio-economic features, such as age, income level, and occupation, as specified in Table 2. For items, the latent factors correspond to the characteristics of the persuasive principles being recommended, like their complexity, type, and targeted behaviour change. The elements of the matrix to be factorized are the interactions between users and items, quantified based on the users’ responses to different persuasive principles. This extended MF approach allows us to capture the nuances of user preferences and strategy effectiveness, leading to more personalized and effective recommendations. This model results in personalised rankings of the selected persuasion principles from the most appropriate for a user to the least appropriate. The second method [25], called here “Persuasion Principle Score Prediction”, is based on Random Forest regression [47] and results in personalised predictions of the score that a user would give to a persuasion principle on a scale from one to five. The former model can predict the order in which a user prefers to receive a persuasive message (ranking), and the latter provides the impact that this persuasive message will have on the specific user through ratings.

This section briefly describes the methods in isolation [14, 25] for the sake of completeness and introduces the fusion scheme used to aggregate the results of the two models. It is also essential to state that the extended dataset is analysed in this manuscript, presenting additional results both for the individual and the fused methods. Various scenarios have been evaluated with the sequential fusion of the two models (https://github.com/morelab/st_recommender).

Persuasion principle score prediction.

The second model uses Random Forest regression to predict a user’s scores for each persuasion principle in a questionnaire. Information about the user demographics and her energy-related behaviour are used as predictors, while the scores of the users for the persuasion principles are used as the targets of regression. Details of the method have been presented in [25]; however, a brief description of the method is provided here for completeness. Fig 3 displays the overall workflow of the method.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Recommendation model workflow (Fig from [25]).

https://doi.org/10.1371/journal.pone.0303214.g003

The problem of predicting the persuasion principle scores is viewed as a regression problem. We aim to predict numerical values using a set of numerical and categorical predictors. We have used Random Forest (RF) regression [50, 51] for this task. Random Forests are ensembles of decision trees, each considering a different random subset of the input data and the input variables, while the final output is a majority voting over the decisions of all trees in the forest. Using multiple trees constructed from random subsets of the data leads to less overfitting and thus better accuracy when presented with novel samples. Random Forests are popular as non-linear prediction models, capturing the non-linear structure of the data and producing accurate predictions.

As a choice for regression, the Random Forest model is more complex than simpler alternatives, such as multi-variate regression, which can also be used to tackle such problems. However, it has been chosen due to its better prediction performance compared to other models. To demonstrate the better performance of this model over more classical alternatives, we have also considered a reduced rank multi-variate regression model (MVR). In MVR, all targets are modelled simultaneously, while the coefficient matrix is constrained to be of rank lower than the number of principles (10), to cope with any correlations between the targets. Apart from being a simpler alternative to Random Forest regression, MVR can also provide added insights into the importance of the predictors for predicting the outcomes.

There are multiple target variables in the current problem, one for each persuasion principle, so multiple RF models are trained. We build an RF model from the data for each persuasion principle, with the respective principle as the (scalar) target. When the input data of a new sample user are given, the multiple models will each produce a prediction of the score for the respective persuasion principle, which can then be used to suggest the most appropriate strategy.

To fit the RF models, the available data are split into training (75%) and testing (25%) sets using stratified sampling. The quality measure to determine the best split is variance reduction, with Mean Squared Error as the measure of variance. The number of trees in the forest is set to 1000.

Evaluation measures

Experimental scenarios

To evaluate and validate both individual and fused models, two specific scenarios have been employed, based on the final envisioned outcome to be predicted.

Persuasion principle optimal ranking

For this approach, and considering that there have not been any state-of-the-art findings trying to achieve similar objectives, a benchmark with three different baselines to evaluate the performance of the persuasive strategy recommendation system has been designed. The first baseline is a random classifier that selects five random principles from the possible ten for each user. The second baseline, called ‘Top5’, proposed the average top 5 principles to all the dataset users. Finally, the last baseline is the proposed model itself, with the user features extracted from it. We employed a 5-fold cross-validation methodology to evaluate the baseline’s performance and the model through all the data. In each fold, 80% of the data was used as the training set and the remaining 20% as the testing set. The average NDPM, as well as the standard deviation for each model for the testing set, are reported in Table 3.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. NDPM and standard deviation (σ) for optimal ranking prediction model with the cross validation approach.

https://doi.org/10.1371/journal.pone.0303214.t003

The results reported in the table show that the model with the data performs better than the other proposed baselines. However, the Top 5 baseline also achieved good results in this experiment. As we analysed the dataset, it can be observed that there are five popular principles that most of the users liked the most. Even if these principles are not consistently ranked in the same position, the NDPM for a ranking that includes the same principles as the ground truth will always be better than a ranking that does not include one of those principles. Therefore, baseline results will always be high for this dataset because of the popularity of some principles and the similar rankings between many users. These findings are in line with the observation that four different user groups can be found who ranked the top 5 principles differently in the dataset that we used. Conducting a T-SNE to the user embeddings obtained from the model, these user groups can be easily clustered using agglomerative or simultaneous approaches. The latter was employed using K-means, from which we obtained four clusters that represent user profiles, as can be observed in Fig 4. Please, bear in mind that in the next Figure, unlike some other dimensionality reduction techniques (like PCA, where axes can represent principal components), the axes in a t-SNE plot do not have an inherent, interpretable meaning. Thus, they do not correspond to specific original features of the data. In essence, the focus of Fig 4 should be on the relative positioning and clustering of points, rather than trying to interpret the axes themselves. The axes serve as a means to visualize the high-dimensional relationships in a more comprehensible two-dimensional space.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. T-SNE and K-means conducted to users’ embeddings.

Red dots show the centroids of each of the clusters.

https://doi.org/10.1371/journal.pone.0303214.g004

Persuasion principle score prediction

To evaluate the persuasion principle score prediction method, we have trained and tested the Random Forest regression (RF) and the Multi-Variate Regression (MVR) models, one for each principle, in 5-fold cross-validation, using the same folds as the ones used in the evaluation of the optimal ranking procedure. Furthermore, the usage of the same folds allows the comparison between the two methods and the fusion of their results. For the MVR model, the rank of the coefficient matrix has been set to 5, which achieved the highest accuracy scores.

The results of the evaluation are presented in Fig 5. The horizontal axis contains the target principles, while the vertical axis reports the RMSE errors of predicting the scores for each principle. The lines correspond to the mean RMSE across all cross-validation folds, for the RF and MVR methods, while the error bars correspond to the 95% inter-quantile range. We can see that the RMSE errors range from 1 to 1.5, indicating that the predictions deviate, on average, by 1–1.5 units from the true scores, which fall within the range of 1 to 5. We can also see that the RMSE values vary for all principles, with “Similarity” and “Social proof” achieving the lowest errors, suggesting more accurate predictions for these principles. The errors of the RF method are slightly lower than those of the MVR method, indicating that the RF method is more suitable for predicting individual principles, as expected given that MVR considers all principles simultaneously. However, it is important to note that even though MVR used a rank 5 coefficient matrix, it still achieved relatively high accuracy, especially when focusing on fewer “core” targets.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Evaluation results for the proposed recommendation model, using the RF and the MVR models.

RMSE values are shown for all persuasion principles. The principles are sorted by increasing RMSE values.

https://doi.org/10.1371/journal.pone.0303214.g005

The importance of each predictor in the final predictions is shown in Fig 6. It can be seen that aspects of energy-related behaviour such as profile information, education and barriers, are among the most important predictors.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Average importance of the predictors in the Random Forest regression models used for score prediction.

https://doi.org/10.1371/journal.pone.0303214.g006

Fusion of optimal ranking and score prediction

To evaluate the fusion of the two methods, we have first considered the problem of predicting the optimal persuasion principle to use for each user, i.e. the top-1 recommendation. Using the top-1 recommendation extends our previous work (where top-10 rankings and rating scores were computed) to bring it closer to the envisioned application of our methods, i.e. selecting the most appropriate strategy to use to persuade people. In a real-world setting, the system would need to select one strategy as the most appropriate, so being able to predict the top-ranked strategy is important. Considering only the top-1 principle makes the problem a multi-class classification problem so that measures such as F-score can be applied for evaluation. For example, if the ground truth for a user (i.e. the most appropriate principle) is “suggestion”, we want to evaluate if the proposed methods will be able to rank “suggestion” as the top-ranked principle out of all considered principles. For the experimental scenarios, we used the fusion of the two methods to compute the top-5 principles and then used the top-1 principle as the prediction for the most appropriate persuasion principle for a user. Next, we compared this prediction to the ground truth, i.e. the top-1 principle reported by the users in the questionnaires. As a baseline method for comparison, we have also used the most frequent top-1 answer in the training set to predict all users. In the majority of the cases, the top-1 strategy is: “Conditioning”. Recall that the top-5 principles among the whole dataset were: “Conditioning”, “Social-recognition”, “Physical attractiveness”, and “Self-monitoring”.

The results are summarized in Table 4. The columns of the table correspond to the compared methods: most frequent (baseline prediction), optimal ranking individually, score prediction individually, and sequential fusion. For the score prediction, both the RF and the MVR models are used, so there are two pairs of columns for score prediction and fusion. The rows of the table correspond to the evaluation measures used: macro F-score, weighted macro F-score, and micro F-score.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Comparison between the individual methods and the fused methods, for predicting the top-1 persuasion principle.

https://doi.org/10.1371/journal.pone.0303214.t004

We can see that for this type of problem, i.e. predicting only the top-1 result, the two individual methods perform rather poorly compared to the baseline of using just the most frequent result of the training set. This is more or less expected since the methods were trained with different goals in mind, i.e. predicting the complete ranking and the whole set of scores, respectively. However, when the two methods are fused sequentially, they surpass the baseline, providing better predictions, as verified by all the reported measures. The fact that the fused method (ensemble) performs better than the individual methods is a recurring pattern in the ensemble learning literature [54]. Combining different predictors increases the confidence in the predictions and reduces the error. In our case, keeping the top-5 principles from the first method probably acts as a filter that removes “noisy” principles that would reduce the accuracy of the second method. In this manner, the two methods act complementary, achieving better results when combined rather than when used individually.

Comparing the Random Forest (RF) and multi-variate regression (MVR) models for score prediction, we can see that the RF model achieves higher performance than MVR, suggesting that it can more accurately predict the top-1 principle. However, the fused models achieve comparable performance, which shows that fusion is able to overcome the shortages of even less accurate models.

Overall, although the numbers themselves are low, the results show that the user information encoded in the predictor variables contains information that can be used to provide better and personalised recommendations rather than recommending the same strategy for all users.

As a second experimental scenario, we have also evaluated the performance of the models in predicting the top-1 type of interaction, according to the types of interaction reported in Table 1. After we computed the top-1 principle, as in the evaluation above, we used Table 1 to map each principle to the corresponding interaction type and re-computed the evaluation measures for this type of outcome. There are only two outcomes this time, interaction with people and interaction with a system, i.e. the goal is to predict which kind of interaction would be most appropriate for each user.

The results are presented in Table 5. The absolute numbers are now larger than those for predicting the exact principle, which is expected since the number of classes is now only two. We can also see that again, the fused method performs better than either of the two proposed methods when used individually and also better than the baseline method where the most frequent type of interaction is used as the same prediction for all users. The RF model is again more accurate than the MVR model, although to a lower degree, while the fused models are again comparable in performance.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Comparison between the individual methods and the fused methods, for predicting the top-1 type of interaction.

https://doi.org/10.1371/journal.pone.0303214.t005

Enhancing the recommendation through a cascade of models

In the ensemble model provided in this paper, the former computational recommender system serves as filtering for the latter. That is, the first model can provide the order of preference that each user will have on the ten persuasive principles upon evaluation (see Table 1). Indeed, according to the Normalized Distance-based Performance Measure (NDPM), the first model outperforms the random assignment of persuasive principles and even the best top-5 principles to newcomers. However, having a ranking is insufficient to understand if the top principles will have a tangible impact on the user. Therefore, the latter model uses the top 5 five principles learned from the first model instead of all the ten initially evaluated (i.e. the first model can be understood as a dimensionality reduction approach). With these measures, the optimal scoring for each persuasion strategy is more likely to be obtained. Still, researchers using this model can fine-tune it to find better performance with, for example, a selection of a smaller subset of persuasion principles and facilitate the final selection of the latter model. The proposed model achieves an RMSE around 1–1.5 on a scale from 1 to 5, which, although not too low (relatively to the range of the score it is about “average”—not too poor or too good), is low enough to make the model useful in a real setting. In practical terms, this RMSE range suggests the model can provide useful, albeit not perfect, insights or predictions within the given context. While our model outperforms some benchmark models, the main goal of this paper was not purely to obtain the best performance metric, but mainly to propose a novel way to address the way messages are presented to end users to ensure that the desired action (e.g., switching off the PC while doing breaks) will eventually occur. With this ensemble, we demonstrate that cascading models seem to be a way to effectively address end-user tailoring. It is also worthwhile to notice that the model accuracy is different for different principles, with principles such as “similarity” and “social” proof achieving more accurate predictions. This means that when the model predicts these principles as most appropriate, it can be more confident about the correctness of this decision. Such an indication of confidence can also be of value to the policymaker when deciding how to interpret the model outcomes.

Framing the dimensions of persuasion per individual

To optimise the model in terms of the type of persuasion (see Table 1) that can be more effective to a specific user, an outcome is provided in terms of the type of motivational messages each user prefers between a more social-based interaction and system-based interaction. This idea is in line with existing literature where some authors explore the use of social practices for persuasion [55] and others use more system-oriented approaches such as visual interaction [56]. Having this outcome for each individual, scholars, researchers or intervention designers can have a clearer idea of what kind of contextual feedback to provide to end users in order to become more energy efficient (e.g., lowering their energy footprint) in tertiary buildings. This is an important outcome as people do not react equally to the same stimuli. Therefore, our method can provide hints of the types of interaction of messages that can be more relevant to a user depending on their personal traits. Some would prefer a message such as “your peers are already lowering their energy footprint” (i.e., social proof- People) while others would prefer a message such as “the system allows you to check your energy performance at any time” (i.e., self-monitoring—System). In essence, if instead of looking for the best persuasion strategy, a recommender system would like to know what kind of context is the most valuable to present a message to end-users, the model we presented in the paper can help to provide that with higher reliability.

Applying the model in real-world scenarios

The majority of the prior work reviewed conducted qualitative (focus groups, interviews) or quantitative research (questionnaires) to elucidate if certain socio-economic characteristics affect the persuasion potential of persuasive strategies [57, 58]. This is mostly done through the use of observational studies, structural equation models and systematic reviews of the literature [59–61]. However, when it comes to providing different persuasive strategies depending on the context, or the user traits, or depending on the success obtained from the previous experiences nudging a behaviour, the authors of this research have realised that the literature is scarce. We therefore aim to shed light on this area with the approach presented. Our main statement has as its basis the fact that the user archetypes are dynamic and changing. Consequently, if someone can obtain explicit information from users at certain moments, it will be much more effective to tailor a message to them [62]. The fact that the amount of information needed by the proposed method is not vast (only 9 features) makes it easier to get repeated measures throughout the time to capture such a dynamicity (e.g., using an app that facilitates the collection of data through rapid micro-surveys or online questionnaires without putting much burden to the user). Moreover, the fact that the context of the suggestions is personalised would guarantee that the suggestions keep up with the evolving user profiles. Therefore, we argue that the use of intelligence-based approaches for this purpose is novel. The only similar approaches we learned from the body of research are Kaptein’s persuasive profiles-based [59], Mogles’ agent-based models based on internal values [63] or Kang’s virtual healthcare [64]. However, computationally speaking, our approach seems to be more easily adaptable to real online services and mobile web or social apps. In essence, the proposed method could be used in a real setting as the back-end engine of a persuasion recommendation system, that would take this user information as input and compute the most appropriate strategy for each user. In this regard, other research works underscore the effectiveness of personalized normative messages in recommender systems for promoting energy conservation [65]. This aligns with the emphasis on contextualized persuasion principles, substantiating our approach with empirical evidence. Moreover highlights that tailored messages based on social norms can significantly influence user behaviour in energy efficiency. For this reason, applying this strategy would then require implementing each of the persuasion principles using specific actions, such as motivational messages (e.g., similarity or suggestion principles) and/or monitoring applications (e.g., self-monitoring or cause-effect principles), through web or mobile interfaces. This will be the topic of future endeavours and experimentation and we expect that other scholars can use our work to test these or other ideas in their research overall when aiming at understanding which persuasive principle elicits behaviour change in the mid and long-term.

In essence, it is important to address the practical challenges and considerations that may arise during the implementation of our proposed framework in real-world settings. Firstly, integrating our system into existing infrastructures requires careful planning and adaptation to ensure compatibility with different technological environments across the workspace. Secondly, collecting and handling user data, which is a cornerstone of our framework, must navigate the complex landscape of privacy laws and ethical considerations, as we will explore in the next sub-section. Thirdly, the effectiveness of our system depends on user engagement and acceptance. Additionally, the scalability of the system must be considered to ensure that it remains efficient and effective as it adapts to varying numbers of users and different environments. Finally, ongoing monitoring and evaluation are necessary to identify and address any unforeseen issues after deployment, and to ensure the system’s relevance and effectiveness in dynamically changing real-world scenarios as the workplace.

Privacy and Ethical considerations and model generalization

From a practical point of view, the fact that the proposed method can predict optimal personalised persuasion principles from a few input features makes it suitable for use in real-world applications. In essence, by just obtaining nine features of the user (e.g., gender, education or willingness to do green actions), our approach is able to provide bespoke messages to the users to motivate them to continue doing green practices related to energy efficiency in tertiary buildings (e.g. switching off a lamp when not in use, use the low energy mode in their PCs, or nudge people to change equipment to more energy efficiency options). However, special attention should be given to the privacy, legal and ethical aspects if the proposed system would be deployed in real-case scenarios. Another issue somehow connected to the previous, is that some people can be reluctant to provide personal data in a micro-survey (e.g., their age (although we asked the age range, as can be seen in the surveys uploaded to Zenodo), the country where they live, their gender or the education level). Others can find it cumbersome to answer a questionnaire several times asking for their intentions to behave pro-environmentally or their energy consumer archetype—PST—(these two constructs from Table 2 have an instrument of 12 and 9 items behind). Therefore, from the nine user features, we consider that only Intentions, Confidence in technology and PST can change over time whereas the rest are more static. In this regard, we challenge the research community to find shorter, yet still valid, instruments that can ensure that these constructs are measured correctly. Another option to lower the burden on the user would be to create a new model based only on the top user traits (i.e., from Fig 6: PST, Education, Barriers and Work culture) to see if still the model is effective predicting the preferred persuasive strategy or to create a new model with only static features so no repeated measures are needed as time passes. Another option would be to collect part of these data indirectly and unobtrusively through monitoring devices in the workplace (e.g., barriers or work-culture). The main issue that appears, apart from the extra cost, is the privacy concerns that installing new equipment would pose to the employees in the workplace [66, 67]. Again, the future of this work is in the direction of finding ways to create data-driven user profiles [68] without burdening the end-user and also preserving their privacy and ethical concerns at all levels (e.g., by using federated learning to separate the datasets per country as we did in a previous work [26]. It is worth mentioning that the Country where people live may affect their acceptance of energy-efficiency interventions [69]) and it should be taken into account to measure the effectiveness of a potential intervention based on the proposed system. Finally, in terms of generalization, our model indeed leans on data for a specific field of application (i.e., energy efficiency) and a specific context (i.e., the workplace). Nevertheless, we argue that the model is valid for any intervention related to sustainability issues (e.g., saving water or other resources, or improving the air quality). The only user feature that has to be removed as a predictor is “Work-culture” which is among the most relevant features (see Table 2) to accurately predict the persuasive principle. An option to tackle this issue would be to find another context-based predictor that will retain similar importance to ensure adequate inference.

Nudging vs. persuasion

While persuasion in literature employs strategic communication to influence beliefs and actions, nudging subtly steers behaviour in a desired direction without necessarily applying direct persuasion. As discussed by Wecker, Kuflik, and Stock [70], while not mandatory, both persuasion and digital nudging benefit from personalization, opening new opportunities for enhancing user experiences and bespoke interaction when used appropriately. According to Herrick [71], persuasion, in its broader sense, is an art of rhetoric, using specific formations to achieve desired effects on audiences. This can involve both rational and emotional appeals, influencing individuals’ knowledge, beliefs, and interests. Nudging, as defined by Thaler and Sunstein [72] and by Mols et al. [73] is identified as a distinct mode of governance, aiming to modify behaviour subtly without explicit persuasion strategies or incentives such as pecuniary ones. It uses choice architecture to make certain behaviours more likely without restricting freedom of choice. In this manuscript, we are focused on persuasion principles to influence energy-behavior activities. However, our approach indeed has a lot of similarities with nudging, as we are also trying to enact certain behaviours in a bespoke manner, aligning with insights from Eslambolchilar and Rogers [74] on how technology can leverage persuasion and nudging”.

Limitations of the current work

One of the most common limitations of behavioural studies is the sample size. In a recent systematic review [6] on the energy domain specifically, all of the 38 articles that have been extracted from the literature ranged from 4 to 651 individuals, introducing quite the doubt for the results. Although the sample size of this study is larger (N = 743), the population they belong to is very wide and therefore, more evidence is needed to observe if the results can be generalised to other actors, societies or cultures. Recall that the data from this study is obtained from four countries: Austria, Spain, Greece and the UK. The diversity in energy behaviours and cultural differences beyond these four European countries limits the extent to which our findings can be applied globally. Therefore, caution must be exercised in generalizing our results to populations in diverse geographic and socio-cultural contexts outside of Europe. This limitation underscores the need for further research in varied settings to enhance the universality of our findings. More particularly, provides us with a line for future research to remove the location factor (country) and explore if our computational model works better when people belong to the same society/culture.

The second major limitation of this study is that the factors/features obtained for training the presented models are tightly coupled to the energy efficiency context in the workplace (e.g. if the people would be willing to join an environmental campaign in the workplace, the barriers that people encounter in their workplace to act pro-environmentally, their current occupation or their behavioural archetype. Thus, our proposal is dependent on whether the end-users are considered as pinballs, shortcuts, or thoughtful which are stereotypes related to the way they use energy resources [43]. Therefore, to fit the model to other contexts beyond the workplace and energy efficiency, it is advisable to find the socio-economic and behavioural characteristics that are context-specific for the new field of application to be then linked to the selection of persuasive principles studied in this paper. One option, if the objective is to stick to the generalist approach (not case-specific), could be to try to link the persuasion principles to personal traits such as those obtained from validated questionnaires such as Big Five (Ocean) or DiSC [75].

Another limitation of this work is that it relies on the subjective appropriateness of the persuasion principles, as reported by the users themselves in the provided questionnaires. The end-users may have certain misconceptions about what would affect their behaviour, which are imposed by pre-existing beliefs or social influence that do not reflect reality [76]. Additionally, the subjective nature of the self-reported data in our study is a significant limitation, as it can introduce biases related to personal perceptions, potentially affecting the accuracy and reliability of our findings. This aspect is crucial in understanding the implications and applicability of our results. In the current work, subjective perceived appropriateness was used as a proxy for actual appropriateness, due to the current lack of an extensive study to evaluate the actual appropriateness. Such a study would require applying multiple persuasion principles to the same user over a large time span and monitoring their effects in terms of energy savings, through extensive monitoring. This is a plan for future endeavours. However, the results of the current study are a first step towards this direction.

Regarding the persuasion principles studied in this paper, the authors acknowledge that their selection of ten principles was based on expert knowledge and from a survey conducted during the process of a research project to observe which were most prevalent among respondents. However, putting together the principles from Cialdini (6 principles), Fogg (15) and Oinas-Kukkonen (25) aforementioned in this paper and adding Abraham & Michie (93) [77], a researcher will find him/herself with more than one hundred of options to evaluate. A better approach than the one followed in this article would have been to obtain the most effective principles in the context of sustainability reporting in the body of knowledge. Still, the evidence of success or failure is not always provided in the same way, making the meta-analyses on this field challenging.

Another limitation of the presented recommender system presented against other computational models reviewed is that our approach is not framed nor linked to any behavioural framework (e.g. Transtheoretical Model [78], Belief-Values-Actions [79], Elaboration Likelihood Model [80], Social cognitive theory [81] or Theory of Planned Behaviour [82]). Therefore, future work should be focused on expanding the cause-effect idea that the model proposes towards providing specific persuasive messages to determinants of behaviours based on existing behavioural frameworks.

Finally, whereas the results are enhanced in the two-step pipeline proposed with regard to the simplest isolated models, the outcomes reported are not as conclusive as the authors would have expected when they designed the study. Nevertheless, they provide room for improvement by exploring other features that might be more relevant when inferring the most impactful persuasive strategy for a specific user based on previous knowledge and context. For instance, the model ensemble approach presented in this paper opens the possibility of studying alternative options to combine the outputs of both models. In this way, together with the ranking obtained in the Persuasion Principle Optimal Ranking, additional scores could be obtained and combined with the scores of the Persuasion Principle Score Prediction stage. For that, a multi-objective optimization strategy could be applied in order to search for optimal solutions that maximize the output of both models and balance their trade-offs. Pareto multi-objective optimization analysis [83] could be applied to find the small-sized subset of solutions with the best performance [84]. This provides an example of the flexibility of the proposed approach.