Reader Comments

Post a new comment on this article

Quality Issue in this Article

Posted by Domino on 17 Jan 2012 at 11:36 GMT

I just posted a comment on a NYTimes article, referring to, but not naming this PLOS article, so feel I should now comment directly on the article on PLOS.

I fail to understand why the article cited below failed to meet the inclusion criteria for this article. If the article below should have been included, how many other articles were selectively excluded that would have changed the resulting view?

Damiani G, Pinnarelli L, Colosimo SC, Almiento R, Sicuro L, Galasso R, Sommella L, Ricciardi W. The effectiveness of computerized clinical guidelines in the process of care: a systematic review. BMC Health Serv Res. 2010 Jan 4;10:2. Avail at: http://www.biomedcentral....

No competing interests declared.

RE: Quality Issue in this Article

ashlydblack replied to Domino on 09 Oct 2012 at 12:16 GMT

If one were to read the abstract only then yes it would seem as though we had erroneously excluded the systematic review by Damiani et al. They state that "this systematic review assesses the impact of CCG on the process of care compared with non-computerized clinical guidelines" and that "from the research that has been carried out, we can conclude that after implementation of CCG significant improvements in process of care are shown." This is rather different in body of the report where the results pertaining to impact of CCG on processes of care are limited to calculating the "Proportions of studies with Positive and Negative Effect of CCG versus NCCG". Where a positive effect proportion of CCG was found to be 0.64 (p = 0.053).

Our inclusion criteria stated that those systematic reviews not primarily concerned with assessing impact on patients, professionals, or the organisation, but nonetheless intervention focused, would be drawn on to provide additional contextual information. The balance of this particular review was tipped in favor of that of a supplementary review being primarily concerned with assessing which design feature(s) of CCG made for success or failure (i.e. socio-technical or human factors). The results in this regard are indeed more elaborate, as shown here:

Automatic provision of recommendation in electronic version as part of clinician workflow (proportion = 0.82) and Degree of automation (proportion = 0.80) were the most frequent features used in the CCG software described in the selected articles. On the contrary, the least frequent features were Recommendation executed by noting agreement, Provision of a recommendation not just an assessment, Promotion of action rather than inaction (proportion = 0.11).(Italic)

The multivariable analysis highlighted two variables as statistically significant predictors of CCG positive impact on the process of care: Automatic provision of recommendation in electronic version as part of clinician workflow (Odds Ratio [OR]= 17.5; 95% confidence interval [CI]: 1.6-193.7) and Publication Year (OR = 6.7; 95%CI: 1.3-34.3). Besides, the feature Justification of recommendation via provision of reasoning (OR = 14.8; 95%CI: 0.9-224.2) resulted marginally significant in logistic analysis.(Italic)

They then conclude that their "contribution provides an updated, systematic and quantitative analysis aiming to understand the design factors which are responsible for the success or the failure of computer-based guidelines compared with NCCG."

I hope this clarifies the reasoning behind the categorization of this particular review as supplementary.

Ashly B.

Competing interests declared: Author on article