Highways to happiness for autistic adults? Perceived causal relations among clinicians

The network approach to psychological phenomena advances our understanding of the interrelations between autism and well-being. We use the Perceived Causal Relations methodology in order to (i) identify perceived causal pathways in the well-being system, (ii) validate networks based on self-report data, and (iii) quantify and integrate clinical expertise in autism research. Trained clinicians served as raters (N = 29) completing 374 cause-effects ratings of 34 variables on well-being and symptomatology. A subgroup (N = 16) of raters chose intervention targets in the resulting network which we found to match the respective centrality of nodes. Clinicians’ perception of causal relations was similar to the interrelatedness found in self-reported client data (N = 323). We present a useful tool for translating clinical expertise into quantitative information enabling future research to integrate this in scientific studies.


Reviewer #1: Review PONE-D-20-15702 ### GENERAL COMMENTS
In general, this is a very interesting manuscript that has used a network methodology to shed a light on a new methodology (PCR) to catch clinical aspects regarding ASD. Nevertheless, to achieve the best shape, the manuscript needs some modifications.
We would like to thank the reviewer for appreciating this manuscript and for suggesting modifications to achieve its best shape. We respond to all points being raised below and have addressed all helpful comments in the revision. We are grateful for the time investment of the reviewer and are convinced that the opportunity to revise the manuscript based on the reviewer's suggestions has significantly improved our submission.
1. The authors tend to use several personal choices to justify some decisions (without a proper rationale to justify them). Two of the most problematic topics of the article are: a) the sample size (n=29 completed the questionnaire and n=16 were enrolled in the follow-up), We respond to this comment below, comment no. 23, where the reviewer explained their concern more extensively. In short, for a rater study, based on the literature of structured interviews, the current study's sample size can be considered large (de Villiers et al., 2005). While we are of course aware that this is considered small for empirical studies based on survey data. We have now clarified this with the addition of lines 570 -573.
1. and b) is the use of the term "causal". If by one side, the authors mention correctly "association network", by the other, they tend to classify associative relationships as causal ones. The authors should avoid using "causal" when data is correlational.
We agree with the reviewer that careful use of the term 'causal' is warranted. We have therefore revisited every single use of the word throughout the text and have either removed the word to avoid confusion or added the word "perceived" to stress that the discussed association is the results of a rating technique focused on perceived causality (the Perceived Causal Relation method).
2. The authors should have more care regarding the organization of the text. For example: the tables are out of order, the methodology and discussion presents redundant parts, and more than once, the reader needs to read again a part of the text to understand the intentions of the authors.
We thank the reviewer for spotting these inconsistencies and have carefully reorganized the text and tables to make sure there are no redundant parts anymore and to generally improve its comprehensibility.
3. Rather than using the term "empirical data", it would be more appropriate to rename this sort of information as "subjective data" (gathered directly from patients) in opposition to "objective data" (gathered from clinicians regarding their patients). Because, as a matter of fact, this study compares personal observation of patients with observations from clinicians. Since "empirical data are facts that are obtained by observation or experiment", both data used in the research could be considered empirical. The definition used by the authors does not consider as empirical the second cluster of data. Nevertheless, this assumption is not correct.
We thank the reviewer for this suggestion and have given it extensive thought. In this context, the notion of subjective vs. objective may come with problematic implications: the clinicians' data (now labeled as such, or as raters data) is also subjective in nature, as it is based on individual perception. At the same time, we agree that a pool of experienced clinicians as information source will be more objective than self-reported data as their judgments are based on different cases and years of experiences with autistic individuals. Based on this line of thought, we have, however, decided to refrain from using the 'subjective' vs. 'objective' phrasing but instead changed the 'empirical' label throughout the data description into 'selfreported data' and used the term 'association network' instead of 'empirical network'. We hope to have accommodated the reviewers concern with these changes.

5.
In a more general comment, to improve the comprehensibility of the text, it would be very interesting if the text was built having blocks of information in mind. Them, pairing each part of the text with the next section. Example: The concepts and gaps mentioned in the introduction should be paired with the aims of the study. Then, these previous parts being paired with the corresponding part on the methods used for answering each of these questions (in a step-by-step fashion). Followed by the results you have for each of those questions. And, finally, the discussion of these results (still pairing them with the previous blocks), and the presentation of a conclusion We would like to thank the reviewer for this excellent suggestion to improve the comprehensibility of the text. We were trying to structure the introduction with first/second/third before but are much happier with this content-related step-by-step fashion. We chose the following headlines for the three aims of the current study (i) Perceived causal pathways in the well-being system, (ii) Clinical validation of networks based on self-report data, and (iii) Integration of clinical expertise into empirical network studies. We have implemented this structure throughout the whole manuscript, i.e. the introduction, the statistical analysis part of the method section, and the results. For the discussion section we decided to stick to the structure of integrating the reported findings in the broader context of the theoretical background. We have rewritten this section to make it more concise.

### INTRODUCTION
6. LINES 65 to 68 -I suggest the authors place the primary purpose of the study at the end of the introductory section. The same could be said regarding information regarding LINES 90 to 93.
We thank the reviewer for this suggestion and have moved lines 65-68 and lines 90-93 accordingly. 7. LINES 85 to 93 -The authors should think about dividing the sentence in two at least. Since it is too long, it is difficult to follow, and the reader has to read again to catch all the information it provides.
We thank the reviewer for this suggestion and have divided the sentence into two parts. We hope this improves its clarity: "Due to the explorative character of network analysis, it is unclear whether relations between variables identified in these studies actually reflect causal interactions (rather than, e.g., the effect of unmeasured common causes). In addition, it remains unclear what the direction of these causal pathways is, and to what extent the relevant pathways are also identifiable by other, independent modes of observation." 8. LINES 94, 106, and 119 -The authors should reconsider the use of these numbered steps ("First", "Second", "Third"). It is unnecessary. Moreover, the paragraphs they begin with were the continuity of the literature review on the field brought by the introductory section. So, it doesn't make sense the use of them… We thank the reviewer for pointing this out and have changed this to the (previously suggested) subheadings listed above, see our response to comment no. 5. 9. LINES 108 and 109 -The authors make an imperative affirmation. Nevertheless, it is a supposition. So that they should use a conditional verbal form instead of such an illation.
-"This suggests that professionals working in the clinical field already WOULD/COULD/MAY conceptualize psychological constructs as a set of causal relations between symptoms and other factors" We thank the reviewer for this suggestion and have changed this into a conditional verbal form: "This suggests that professionals working in the clinical field may already conceptualize psychological constructs as a set of causal relations between symptoms and other factors." 10. LINE 110 -It is not adequate mention "in our view". If the authors made a brief review and from that, they concluded that the sort of information is underused, so mention that. Avoid this sort of postulation. E.g.: "A BRIEF

SEARCH IN PUBMED DEMONSTRATES THAT THERE IS LACK OF STUDIES…"
We thank the reviewer for this suggestion and have changed our wording accordingly: 'A brief review of the existing network literature shows that this source of information has not yet been integrated in network approaches. It remains unclear, for instance, whether the network structures shown in self-reported data actually resemble those networks that clinicians would report. ' (line 112-114) 11. In general, it is interesting the use of questions to incite the readers toward some aspects of the literature gap. Nevertheless, some of them could be placed as affirmations (and their respective references). The overuse of the questions may give the impression that the authors did not perform a good literature review (what doesn't seem to be the case). But to prevent this imprecise impression, use the questions only for the most important points raised in the text (and only those the authors intend to answer with the present manuscript). In LINE 133-134, for example, the authors mention: "The aim of the current study is to take the first steps towards addressing these questions…". The problem is: there were so many questions… which one of them do they intend to answer? All of them?
We thank the reviewer for this comment and we agree that the overuse of questions can give the wrong impression. We have, therefore, rephrased some of the questions in the introduction into affirmations, see e.g. line 103-105: ' That is, the field is in dire need for a toolbox that can help us determine which connections in the network represent directed causal effects that arise from reciprocal causation or coupled equilibria, and which associations are due to the effect of unmeasured variables.' In (now) lines 129-134 however, we raise two very specific questions that we intend to answer with the current study: a) how to meaningfully represent clinical expert knowledge in network studies and b) how to combine existing methods assessing such expert knowledge with networks of self-reported data. Directly after raising these two questions, we provide a detailed explanation of how we plan to answer these questions by the current study. We thank the reviewer for this suggestion and have divided the too-long sentence presenting the current study's aims to make it easier to read: "The first aim of the current study is to take the first steps towards addressing these questions, by constructing a symptom network on the basis of expert judgments (Frewen et al., 2012) to visualize the relationships among characteristics of ASD and multiple facets of outcome and well-being. The second aim is to combine this information with the information obtained from statistical analyses of survey data."

LINE 147 -I suggest changing the term "client" for "patient"…
We thank the reviewer for this suggestion, and have changed our wording accordingly.

LINE 150 -Suppress "exploratory".
We deem it important to clearly stress the exploratory character of this study. Since the reviewer did not share their reasons for this comment, we have, for now, decided to refrain from suppressing 'exploratory'.

LINE 151 -Although the proposal of look into Deserno et al. 2017 is interesting, that proposition would be more suitable for the reader if the article would be presented in a medium with hypertextual tools (which is not the case of PLOS ONE). So, explain very briefly, in the introduction, before presenting the aims, the domains of well-being as presented in Deserno et al., 2017.
We agree with the reviewer that it comes with disadvantages for the text's general comprehensibility to work with important information that is drawn from another article. We have, therefore, followed the reviewers' suggestion to add information on the ASD factors important to well-being from Deserno et al., 2017, see line 116-118.
17. LINES 155 -156 -Why do the authors would like to explicitly mention the PCR as an objective of the paper, deviating from the core results? ("…provide an example of how to use the PCR toolbox to integrate the knowledge of clinicians in empirical studies"). Is it necessary? Wouldn't this interesting work itself be a proper example?
We thank the reviewer for this comment and have rephrased this aim without a focus on the PCR methodology: '...provide an example of how to integrate the knowledge of clinicians in empirical studies.'

LINES 156 -158 -More suitable for the "Methods section".
We thank the reviewer for this suggestion and have moved the sentence to the Methods section, lines 163 -165.

We agree with the reviewer and have restructured the introductory section, by moving parts to the Methods section, and removing redundant paragraphs.
### METHODS 20. In general, this section is prolix, presents redundant parts, and is not straightforward. It needs to be reformulated to provide to the reader a better understanding of the step-by-step of what was done.
We have revisited every single use of the term 'causal' to make sure it is clear that we talk about perceived causality, see our response to the reviewer comment no. 2.
22. Concerning the sample, how many clinicians were invited to participate, and how was the acceptance rate among all clinicians invited? How the authors calculated (and eventually decided) that the number of raters was enough (n=29)? How do the authors think about the number of responders in the follow-up phase?
As we sent out the survey through mailing lists of the clinical departments of the three mentioned institutions, we do not know how many clinicians have read the email and decided not to respond. However, please note that whereas N=29 is a small sample size for quantitative statistics based on survey data (where each case represents one individual) the current sample size is particularly large for rater studies. Common rater samples often consist of less than 10 raters.
We chose to select a very specific sample since we wanted to select a group of experts in the autism field based on their extensive experience. The follow up intervention ratings were obtained from a subgroup of 16 participants as only 16 of 29 clinicians responded to this second online assessment (although we sent it to all 29 clinicians with multiple reminders). We regret to report that we are unable to assess more specific characteristics of those 13 clinicians not completing the follow up assessment regarding intervention targets-and how this affecting the results. We have added a more explicit comment on the potential bias due to attrition to the discussion, lines 595-597 ('This leaves us unable to assess whether the subgroup asked to choose intervention targets has very specific or, instead, a diverse range of characteristics.').
We mention the limitations related to limited information on sample in lines 589 -596: "Third, in this study we were able to reach out to highly experienced experts in the autism field: the knowledge that we combined into the PCR network was based on about 14 years, on average, of clinical work with people with an ASD diagnosis. This is a very specific sample of experts, of course, which limits the generalizability of the ratings to, for example, other mental health professionals or general practitioners (Nicolaidis et al., 2015). Also, although we know that the majority of our informants works for clinical institutes that have a tradition for over 40 years in specialized autism teams, we did not specifically ask for more detailed information about their background and type of experience."

It is not clear how the selection of the institutions and the raters was done.
Do the services investigated were the only facilities treating ASD in the region? Provide a glimpse both the selection of the sample. The simple mention that the authors "knew all the clinicians" is not descriptive enough and does not allow replications. Provide objective criteria. Why the authors have included raters with different backgrounds (psychologists, psychiatrists, behavioral scientists, and social workers)? Since their formation is quite different from each other some bias on the appreciation of causal relations could certainly occur. Moreover, the PCR intends to access causal relations based on their clinical expertise. Even though the patients are the same, the clinical perception of a professional is biased by people's academic background.
In the same direction, mention that because they work in such facilities, "we know that all clinicians participating in the current study are involved in diagnostic work, consultation, and intervention services" is not informative enough. Besides that, how the authors guarantee that some of the respondents do not work only in the administration or coordination of such settings rather than as clinicians.
We distributed the questionnaire through the three leading clinical networks within the autism realm in the Netherlands, targeting specific individuals of which we knew that they are clinically speaking the authority in the Netherlands. Their affiliation with those institutions means that all clinicians participating in the current study are involved in diagnostic work, consultation, and intervention services -which is also the reason we selected these three leading clinical networks. Also, it is a Dutch standard that these institutes all work with multidisciplinary teams. We added this information in lines 167-176 of the manuscript. In addition, we assessed how many hours per week they engaged in clinical work and have added this information on page 13 of the revised manuscript. Although we know that the majority works for a clinical institute that have a tradition for over 40 years in specialized autism teams, we did not specifically ask for more detailed information about their background and type of experience. We, therefore, agree with the reviewer that there are some limitations due to the sample of the current study. We explicitly mention those limitations in line 589 -598. Although individual differences related to professional background would be an interesting future research avenue, our intention was to find a way to find a meaningful way to represent clinical expert knowledge in network studies. For this purpose, it is beneficial to have a sample with a diverse background as this represents the range of expertise usually present at a clinical 'case conference', in Dutch clinics.
24.The authors should avoid expressions such as: "[they] consider 'advanced experience with ASD'". Such subjective evaluation weakens the methodology of the study. For example, mention only that "we knew that they are [specialists]" is too fragile. Objective criteria would be preferable.
We thank the reviewer for this comment. We consider these clinicians advanced, as the average clinical experience in the field was 14 years. We have now changed our wording to 'years of experience' in order to avoid subjective evaluations implied by 'advanced', see lines 167.

What is CASS18+? I suppose it is an institution/facility for the treatment of ASD. Explain previously used abbreviations.
We thank the reviewer for pointing us to this missing definition, and have added it to the Method section. CASS18+ is a national network for professionals specialized in healthcare for autistic adults.

Provide a table with the demographic characteristics of the group.
We have provided all demographic information that we collected in lines 182 -189 since we otherwise would have exceeded the  Figure  28. LINES 204 to 210: That is "Statistical Analysis" rather than "Measure and Procedure". FIGURE 1 and 2: The quality of the images does not allow us to read the texts inside the nodes or in the figure legends. Figure 2

needs a legend.
We regret to learn that the image quality was not sufficient. We will upload another format and hope that this will resolve the issue. Also, we have now added a legend to Figure 2.

LINES 243 -247: The authors mention that: "Therefore, we decided to split the EMPIRICAL network in three (overlapping) parts with an (almost) equal number of nodes (j1= 14, j2=13, j3=13) based on their clustering in the EMPIRICAL network to ensure study feasibility." Is that correct?
Yes, the reviewer is correct. We have changed the wording from empirical to 'association network' now.

LINES 243 -252: This was the less comprehensible part of the methodology. Reformulate it to allow a better understanding. It would help a lot providing a diagram/draw/scheme with the step-by-step of the methodology.
We thank the reviewer for this suggestion and have created a scheme of our methodology. We hope this helps to clarify what the reported results are based on:

LINE 259 -Why the authors have asked the raters to choose THREE targets? Based on what rationale?
This choice was based on the following rationale: We were aware that all factors included in the network somehow matter for well-being of autistic individuals, but we wanted the clinicians to make a selection of factors that they deem the most important intervention targets in the given network. In order to avoid the impression that we expected clinicians to rank-order the factors in the network, we limited the space of potential answers to three. The exact number of three answers was arbitrary but the goal was to limit the answers to avoid complete orderings of all factors. We have added this information to the manuscript in lines 260 -261. Also, we planned to distribute the follow-up question regarding intervention targets in a second assessment round, i.e., after the rating assessments, as we wanted the intervention rating to be accompanied by a visual representation of the averaged clinician network. The intervention rating consisted of one simple question asking the participant to choose three intervention targets from the complete list of nodes depicted in the averaged clinician network.

LINE 276 -Why the authors have used a threshold of 6? Based on what rationale?
Please see our response to the related comment below, comment no. 37.
35. It also is not clear why, having in mind that the empirical network reflects partial correlations scaled between -1 and 1, why the authors have used clinicians' ratings from 0 to 10. Why not using a Likert scale, for example?
We thank the reviewer for the opportunity to clarify. Since the association network based on self-report data represents partial correlations, we decided on a continuous rating scale. An average of 7 on the rating scale would then be comparable to a partial correlation of 0.7.

Moreover, comparing two different networks with two different methodologies would not be problematic?
We regret to say that we are not sure what the reviewer is asking here. The two networks we compare are based on another assessment method, and therefore differ in some regards. We discuss this difference in the methods section (e.g., line 301-306) and have presented three different approaches to comparing these network structures, each focusing on different aspects of the networks (i.e. absence/presence of edges, weights, integration as a scaffolding structure in network estimation). In these steps, our methodology is completely transparent in what we are comparing.
37. I wonder if the authors could have further commentaries on the following sentence from LINES 276 and 277: -"Manually thresholding the visual representation was necessary since the raters tended to attribute very high values to edges, they thought were present." Do the authors did not expect that ceiling effect (since they search for "perceived causal effects", those how seem to the raters more intense would be valued)? Do the way the questions were asked did not favor that behavior? Moreover, in LINE 278, the authors mention that "did not specify such thresholds for any of the analyses". Nevertheless, they mention that (LINE 275-275) "only those causal relations endorsed by the raters with an average rating of at least 6 (on a scale from 0 to 10) on the PCR were included in the visual representation of the network". It seems that one sentence contradicts the other.
We thank the reviewer for pointing this out and have rephrased these sentences to avoid this confusion. We do specify a threshold in the visual representation of the network, i.e. the figure in the manuscript. Figure 2 depicts a regularized partial correlation network, which means that we used lasso regularization to control for Type 1 error, pushing small edges to zero. Since there is no similar technique for the rated network (as the connections are rated rather than estimated), we had to manually adjust for the fact that clinicians tended to attribute high values to edges they judged present (but not strong), resulting in a small, but high range, of edge weights. We did not, however, apply this threshold for further analyses. In the presented results all present edges are included, even if their average rating was smaller than 5. Furthermore, we do think that it was to be expected that the clinicians would rarely rate edges between these factors to be completely absent (i.e. 0). This is also in line with the theoretical assumption this whole endeavor is resting on: that these factors form a multicausal network of interrelations (see also our response to the last reviewer comment). We thank the reviewer for this suggestion and have reworked the results section with a focus on the main findings, moving the broader contextual information to the discussion section.

The authors do not present numerical results in this section. This data is important and not presented only as supplementary material (LINE 359).
We thank the reviewer for this suggestion and have moved this information from the supplement to the main text, see 372-375.

LINE 337 -The authors should provide the list of abbreviations in the text and not only in the supplementary material.
We indeed solely provided an extensive list of the node abbreviations as supplementary material, but we agree with the reviewer that it is more convenient to have a We thank the reviewer and have corrected this typo.

Where is Figure 4 that is mentioned in the text?
We regret to learn that Figure 4 was not included in the reviewer's manuscript version, and have thoroughly checked again that it is included in this submission.

LINE 496 -499: This paragraph is not part of the Results section and should be placed in the Discussion section.
We appreciate the reviewer's suggestion and have removed this paragraph from the Results section.

### DISCUSSION
49. In general, the Discussion is easier to read than the rest of the text, although repetitive. Nevertheless, the authors tend to stretch the line in some conclusions to explain some findings. For example, I did not understand the sentence in LINE 526-528: "This would be consistent with the plausible idea that ASD symptoms arise from sources external to the current networks (e.g., from problems associated with brain development)." How's that, "external to the current network"? Are not all the ASD symptoms part of a neurodevelopmental problem? Moreover, having assessed only about 30 clinicians from a limited number of facilities is too forced, and -why not say -presumptuous, mention that "we found that the way clinicians perceive cause-effect relations between ASD symptoms".
We thank the reviewer for the opportunity to clarify. By 'current networks' we mean that there are more factors relevant to ASD than the ones included in the networks of this study. For example, early brain parameters are not assessed in the current study, but without any doubt relevant to atypical development (see Johnson et al., 2017 for an elaborate review of this line of thought). We have now explicitly added this information to line 545-549 in order to clarify what we mean by 'external to the current network'. The reviewer's second comment is in line with previous comments we have responded to in general comment no. 2. We have rephrased the sentence in line 563-565: "Third, we found that the way this clinicians sample perceive causeeffect relations between ASD symptoms, well-being, and domains of daily functioning is fairly similar to the interrelatedness of these factors found in self-reported data." 50. Again, in Discussion, the problem of "causal" inferences persists. We thank the reviewer for suggesting these articles and have re-structured the discussion section of our manuscript according to the suggested structure as much as possible. We have moved the section discussing weaknesses of the study up so that the discussion of its relation to the broader literature comes just before the section discussing unanswered questions and future research.
We thank the reviewer for spotting this and have changed our wording accordingly.
53. LINE 562: I do not think is fair the authors mention that they have used "large rater groups". The sample size is quite small and its subdivision smaller. This should be mentioned as a limitation.
Please see our response to similar comments above.
54. LINE 568: It is interesting the authors mention that "it is important to note that the centrality of nodes in the empirical network indicates the importance of a node…", but they relegate centrality measures to supplementary material.
See our response to an earlier comment by the reviewer, we have now moved this information to the main text.
55. LINE 572 -573: This is one of the most important sentences in the manuscript that deserves more attention from the authors: -"…assuming that the variables in question indeed form a causal system with symmetric effects…".
It is important to mention that, based on an assumption that variables investigated to form a causal system according to the current literature, the networks were built. So, it is necessary to caution regarding the conclusions of the study.
We would be keen to discuss this assumption more extensively with the reviewer and have now highlighted this again in the results section. Overall, we hope that our introduction of the network approach to psychology is a sufficient theoretical background for the reader to understand that this is the assumption under investigation.