Navigation and service

Project on a Framework for Rating Evidence in Public Health (PRECEPT)

Project coordinator: Dr. Thomas Harder

Project partners

Members/institutions of the Operational Group:

  • Robert Koch Institute (RKI), Berlin, Germany

Members/institutions of the Technical Advisory Group:

  • Department of Technology and Social Change, Linköping University, Sweden
  • Department of Clinical Epidemiology & Biostatistics, McMaster University Health Sciences Centre, Hamilton, Ontario, Canada
  • German Cochrane Centre, Freiburg, Germany
  • Glasgow Caledonian University, Glasgow, United Kingdom
  • Grading of Recommendations Assessment, Development and Evaluation Working Group (GRADE)
  • Developing and Evaluating Communication Strategies to Support Informed Decisions and Practice Based on Evidence (DECIDE)
  • Institute for Medical Informatics, Biometry and Epidemiology, Ludwig-Maximilians-University Munich, Germany
  • National Institute for Health and Clinical Excellence (NICE), United Kingdom
  • Norwegian Institute of Public Health, Oslo, Norway
  • Scottish Intercollegiate Guidelines Network (SIGN), United Kingdom

Funding

European Centre for Disease Prevention and Control (ECDC).

Background

The Project on a Framework for Rating Evidence in Public Health (PRECEPT) has been established to develop a methodology for evaluating and grading evidence in the field of infectious disease epidemiology, prevention and control to be applied in the European region. The project – under the lead of the Robert Koch Institute – is carried out by a multidisciplinary team involving experts with well-documented experience in the fields of infectious disease epidemiology and prevention, evidence-based public health (EBPH), systematic reviews and meta-analyses as well as development of evidence appraisal and grading systems. Members of the project team are engaged in a variety of international institutions, organizations, and working groups in the Public Health area, thereby providing profound expert knowledge in the field of EBPH methodology, methods development, and application of evidence appraisal instruments.

Public health recommendations in the area of infectious disease epidemiology, prevention, and control should draw on reliable research to inform decisions. In the field of clinical medicine, particularly regarding treatment options, guidance is developed using the approach of Evidence-based Medicine (EBM). In recent years, debates have centered on the applicability of the clinical EBM model to the field of public health and infectious disease prevention and control. Though EBM started out as an attempt to track the best available external evidence to address healthcare questions, this framework has resulted in evidence-appraisal practices that assume a hierarchy of study designs, with randomized controlled trials being ranked high, whereas observational studies a priori are considered to provide lower quality of evidence. In the field of Evidence-based Public Health, however, often more complex interventions or evidence exclusively derived from non-interventional studies have to be assessed. In particular, in infectious disease epidemiology potentially relevant information often comes from studies which differ considerably in their design from RCTs, such as case series, incidence studies, surveillance data and outbreak reports. For a number of such study designs no established criteria exist for rating of study quality, causing difficulties in grading evidence based on these study types. Given this background, there is a need for a comprehensive framework for rating evidence in the field of infectious disease epidemiology, prevention and control that can assist European public health institutions in developing recommendations.

PRECEPT I

(August 2012 – July 2014)

Aim of PRECEPT I was to develop and pilot a systematic, transparent and comprehensive evidence assessment framework for rating the evidence and strength of recommendations in the area of public health based on published literature and the expertise of a multidisciplinary team in close collaboration with the ECDC. The developed framework will focus on topics and questions that are relevant for guidance development and decision-making in the area of infectious diseases prevention and control, and will not be limited to topics and questions related to interventions.

Results: During the first project phase, the project team conducted a systematic review of quality appraisal tools, provided an overview on previous applications of the GRADE methodology in the field of infectious disease prevention and control and organized an international expert meeting to discuss a draft framework that was developed by the project team (for details see Publications). According to the results of the peer review process which included the expert meeting and further rounds of consultations, a final version of the framework was developed and submitted to ECDC in March 2014.

PRECEPT II

(November 2014 – June 2018)

The second project phase builds upon this first version of the framework. Aim of PRECEPT II is to further elaborate the framework for rating the evidence and strength of recommendations in the area of Public Health / infectious diseases prevention and control developed by PRECEPT, in close collaboration with ECDC, and using the expertise of the multidisciplinary team. Furthermore, aim of the project is to suggest a methodology to translate evidence into recommendations, and to develop a user-friendly handbook and learning materials for future users of the framework.

Results: After a process of further refinements, the PRECEPT framework has been accepted for publication (forthcoming soon in Publications). In its current version, the framework is intended to rate scientific evidence related to four major domains of studies: burden of disease, risk factors for disease, diagnostics and intervention. The framework is grouped into four steps, starting from a complex public health question and ending with an evidence statement for each relevant domain. In step 1, approaches are described for identification of relevant questions. In step 2, methodological guidance is provided for the conduct of systematic reviews for these questions. For appraisal of methodological quality of identified studies, 15 different quality appraisal tools are proposed and an algorithm is given to match a given study design with an appropriate tool. In step 3, a standardized evidence grading scheme using the GRADE methodology is provided to rate the certainty of a body of evidence. The findings are documented in evidence profiles and summary of finding tables. The final step (step 4) consists of the preparation of a narrative evidence summary.

Currently, a user-friendly handbook for the framework is being developed.

Date: 29.09.2017

Publications

  • Harder T, Remschmidt C, Haller S, Eckmanns T, Wichmann O (2016): Use of existing systematic reviews for evidence assessments in infectious disease prevention: a comparative case study.
    Syst. Rev. 5 (1): 171. Epub Oct 11. doi: 10.1186/s13643-016-0347-9. more

  • Harder T, Abu Sin M, Bosch-Capblanch X, Bruno Coignard, de Carvalho Gomes H, Duclos P, Eckmanns T, Elder R, Ellis S, Forland F, Garner P, James R, Jansen A, Krause G, Lévy-Bruhl D, Morgan A, Meerpohl JJ, Norris S, Rehfuess E, Sánchez-Vivar A, Schünemann H, Takla A, Wichmann O et al. (2015): Towards a framework for evaluating and grading evidence in public health.
    Health Policy 119 (6): 732–736. Epub Mar 23. doi: 10.1016/j.healthpol.2015.02.010. more

  • Harder T, Takla A, Rehfuess E, Sánchez-Vivar A, Matysiak-Klose D, Eckmanns T, Krause G, de Carvalho Gomes H, Jansen A, Ellis S, Forland F, James R, Meerpohl JJ, Morgan A, Schünemann H, Zuiderent-Jerak T, Wichmann O (2014): Evidence-based decision-making in infectious diseases epidemiology, prevention and control: matching research questions to study designs and quality appraisal tools.
    BMC Med. Res. Methodol. 14 (1): 69. Epub May 21. doi: 10.1186/1471-2288-14-69. more