Abstract
Data sources
Cochrane Central Register of Controlled Trials (CENTRAL), the Cochrane Effective Practice and Organisation of Care (EPOC) Group Specialised Register, Medline, Ovid; Embase, CINAHL, EBSCO, Science Citation Index and Social Sciences Citation Index, ISIWeb of Science
Study selection
Randomised trials of audit and feedback that reported objectively measured health professional practice or patient outcomes.
Data extraction
Data were abstracted by two independent review authors using a data extraction form. The following factors were examined as possible explanations for the variation in the effectiveness of interventions across comparisons; format of feedback, source of feedback, frequency of feedback, instructions for improvement, direction of change required, baseline performance, profession of recipient and risk of bias within the trial itself. Risk of bias was assessed using EPOC criteria. The degree of confidence in the estimate of effect across studies was assessed using the GRADE approach. All outcomes were expressed as compliance with desired practice and both professional and patient outcomes were assessed, though separately.
Results
One hundred and forty studies were included. Using 70 of these 108 comparisons were made. After excluding studies at high risk of bias there were 82 comparisons from 49 studies with dichotomous outcomes, and the weighted median adjusted RD was 4.3% (interquartile range (IQR) 0.5% to 16%). For studies with continuous outcomes the weighted median adjusted percentage change relative to control was 1.3% (IQR = 1.3%to 28.9%). For patient outcomes the weighted median RD was 0.4% (IQR −1.3% to 1.6%) in studies reporting dichotomous outcomes, and the weighted median percentage change was 17% (IQR 1.5% to 17%) for studies with continuous outcomes. Multivariable meta-regression indicated that feedback may be more effective when baseline performance is low, the source is a supervisor or colleague, it is provided more than once, it is delivered in both verbal and written formats and when it includes both explicit targets and an action plan.
Conclusions
Audit and feedback generally lead to small but potentially important improvements in professional practice.
Similar content being viewed by others
Commentary
The Cochrane Effective Practice and Organisation of Care (EPOC) Group focuses on interventions to improve the delivery, practice and organisation of healthcare services1. A proportion of their reviews are concerned with how to encourage professional behaviour that is in accordance with best available evidence or a professional standard of practice. As well as this review on the effect of audit and feedback, there are also reviews on the effectiveness of educational meetings2, educational outreach visits3 and local opinion leaders4 in changing professional behaviour.
The purpose of audit and feedback is to measure a clinician's performance, compare it to a professional standard and then to feed the results back to the clinician with the intention of improving practice. The feedback may be given verbally, in writing, or both. In the 140 studies included in this review some of the participants were simply given their results whilst in others they were also set targets or an action plan to help them improve.
The vast majority of the included studies involved doctors, mainly in a practice setting, however, there were two that involved dentists; one that involved recording periodontal care5 and another compliance with a guideline for impacted molars6. Most trials measured professional practice, such as prescribing or the use of laboratory tests, but in 112 of the 140 studies it was unclear how the feedback was given. Thirty-one percent of trials were judged to have been at low risk of bias, 51% had an unclear risk of bias and 18% had a high risk of bias.
The authors use the risk difference (RD – an absolute outcome) for dichotomous outcomes and the weighted median adjusted percentage change (a relative outcome) for continuous outcomes. For studies with dichotomous outcomes the overall RD was a 4.3% increase in compliance with desired practice. But using meta-regression they looked at the expected effect of a number of variables on the outcome.
There were relatively large differences in effect size when comparing these characteristics: feedback presented in both verbal and written format versus only verbal (expected difference in adjusted RD= 8%); delivered by a supervisor or senior colleague versus the investigators (expected difference in adjusted RD= 11%); frequency of monthly versus once only (expected difference in adjusted RD = 7%); containing both an explicit, measurable target and a specific action plan versus neither (expected difference in adjusted RD= 5%).
So whilst the effect of audit and feedback may be small overall this review suggests that it can be improved by ensuring the person responsible for the audit and feedback is a supervisor or colleague, that it is provided more than once, it is given both verbally and in writing and it includes clear targets and an action plan.
How does audit compare to other ways of changing clinician performance? The EPOC reviews suggest that there could be a positive risk difference with educational meetings of 6%, with the use of local opinion leaders of 9% and with educational outreach visits, 5.6%. Audit is a component of clinical governance in the National Health Service here in the UK as it is seen as part of the drive to improve care quality7. This review suggests that its effect in such a process may be small but given the large number of patients treated even small changes can result in large improvements in patient care.
References
Cochrane Effective Practice and Organisation of Care Group. 2012 [updated 16/10/2012; cited 2013 23/02/2013]. Available from: http://epoc.cochrane.org/.
Forsetlund L, Bjorndal A, Rashidian A, et al. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2009; 15: CD003030. PubMed PMID: 19370580.
O'Brien MA, Rogers S, Jamtvedt G, et al. Educational outreach visits: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2007; 17: CD000409. PubMed PMID: 17943742.
Flodgren G, Parmelli E, Doumit G, et al. Local opinion leaders: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2011; 10: CD000125. PubMed PMID: 21833939.
Brown LF, Keily PA, Spencer AJ . Evaluation of a continuing education intervention “Periodontics in General Practice”. Community Dent Oral Epidemiol 1994; 22: 441–447. PubMed PMID: 7882660.
Bahrami M, Deery C, Clarkson JE, et al. Effectiveness of strategies to disseminate and implement clinical guidelines for the management of impacted and unerupted third molars in primary dental care, a cluster randomised controlled trial. Br Dent J 2004; 197: 691–696. PubMed PMID: 15592551.
Scally G, Donaldson LJ . The NHS's 50 anniversary. Clinical governance and the drive for quality improvement in the new NHS in England. BMJ 1998; 317: 61–65. PubMed PMID: 9651278. Pubmed Central PMCID: 1113460.
Author information
Authors and Affiliations
Additional information
Address for correspondence: Noah Ivers, Department of Family Medicine, Women's College Hospital, Toronto, Canada. E-mail: noah.ivers@utoronto.ca
Ivers N, Jamtvedt G, Flottorp S et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012; 13: Issue 6. CD000259. DOI:10.1002/14651858.CD000259.pub3.
This paper is based on a Cochrane Review published in the Cochrane Library 2016, issue 6 (see www.thecochranelibrary.com for information). Cochrane Reviews are regularly updated as new evidence emerges and in response to feedback, and the Cochrane Library should be consulted for the most recent version of the review.
Rights and permissions
About this article
Cite this article
Hurst, D. Audit and feedback had small but potentially important improvements in professional practice. Evid Based Dent 14, 8–9 (2013). https://doi.org/10.1038/sj.ebd.6400910
Published:
Issue Date:
DOI: https://doi.org/10.1038/sj.ebd.6400910