Clinical guidelines

When is a guideline not a guideline? The devil is in the details


Clinical Practice Guidelines are not meeting guideline development standards, involve conflicts of interest, and are not always rooted in evidence, reports Cosgrove, Shaughnessy and Shaneyfelt in their recent study, “When is a guideline not a guideline? The devil is in the details”. Clinical Practice Guidelines are intended to facilitate the practice of evidence-based medicine through streamlining healthcare delivery and as a result improving the process and outcomes of patient care. They are believed to represent the best evidence and are therefore adopted as standards of care. Therefore, they have great influence over medical practice. However, the authors argue that the problems with guideline development, such as the vulnerability to biased judgments in guideline development, commercial sponsorship of guidelines, and conflicts of interest of panel members remain an important public health issue that may result in CPGs being a driver of overtreatment.

There have been efforts for increased transparency in the development of Clinical Practice Guidelines. The authors point to the 2011 Institute of Medicine (National Academy of Medicine) standards for enhancing the integrity of the guideline development process, the US Agency for Healthcare Research and Quality requiring stricter standards for CPGs to be included on the National Guideline Clearinghouse, BMJ’s collaboration ‘Rapid Recommendations’, the AGREE II, emphasising the importance of editorial independence as an indicator of high quality, and checklists that have been developed to assist guideline developers, clinicians and journal editors. However, the authors argue that despite these efforts, guidelines that fail to meet standards continue to be developed and published in well-respected medical journals.

The authors point to 6 criteria included in the Institute of Medicine “trustworthy” guidelines.

(1) The process by which guidelines are developed should be detailed explicitly and publicly accessible

(2) CPG developers should use systematic reviews that meet standards set by the IOM

(3) the guideline development group should be independent from industry, and at a minimum, neither the chair nor the majority of members should have financial conflicts of interest

(4) guideline development groups should be interdisciplinary, balanced and be comprised of a variety of methodological experts and stakeholders

(5) for each recommendation, there should be a rating of the strength of, and level of confidence in, the recommendations; and

(6) the guideline should be sent out for external review before release.

The study draws upon an example of guideline development to illustrate the “economies of influence that contribute to the development and dissemination of untrustworthy guidelines”. The guideline was described as being a first-ever for patients with MDD with mixed features: a guideline on the recognition and management of mixed depression published in CNS Spectrums.

The authors first look to the ways in which the development of the guideline did not adhere to guideline development standards. The guideline fails to meet any of the IOM standards for trustworthy guidelines, and the authors write that the 2.5 pages of disclosure of conflicts of interest by the majority of the authors raises questions of potential bias. The authors also highlight the significant financial conflicts of interest among members of the guideline development group.

Concerns with the guideline development include that the guideline was written by a group of experts but it is unclear who convened the group or how they were selected; the lead author of the guideline is also editor-in-chief of the journal in which the guideline was published; there is no transparency around the process of developing the recommendations; no representatives of stakeholder groups were identified; and finally, there was no external review.

The authors also raise concerns about the “poorly established evidence base to support treatment recommendations”. The guidelines cited seven studies as evidence for the recommendation of an antipsychotic for mixed depression. Only two were randomised controlled trials on which a recommendation to treat depressed patients with lifelong antipsychotics was based. The authors found that four of these studies were post-hoc analyses and write that because of multiple testing there is a significant increased risk of false positive errors. Finally, the remaining study was a clinical trial for a different condition.

This example is only one of many examples of guidelines that have been developed under industry’s influence and that could result in negative health consequences. The authors write that “a guideline is not a guideline if it fails to adhere to standards put forth by public health institutions such as IOM, NICE, National Health and Medical Research Council, G-I-N and AHRQ”.  The authors call for steps to be taken to ensure guidelines are trustworthy. Their recommendations include:

  • that the United States consider adopting a national guideline assessment process;
  • multidisciplinary panels to prevent ‘group-think’ and guild interest from unduly influencing the development process
  • membership of Methodologists on each guideline panel or, at the very least, having them serve as consultants.
  • the consolidation of guideline development efforts.
  • that guideline development be commissioned to panels with demonstrated technical and clinical expertise.
  • the centralisation of guideline development under the guidance of AHRQ in the USA or similar groups in other countries.

In conclusion, the recommendations should be adopted in order to ensure a transparent and evidence-based process for guideline development – ensuring improved health for all.

  • Heather Webster


Photo credit: Nick Youngson CC BY-SA 3.0 ImageCreator