Hostname: page-component-65f69f4695-q7d28 Total loading time: 0 Render date: 2025-06-27T23:00:50.365Z Has data issue: false hasContentIssue false
  • English
  • Français

Evaluation criteria to assess the value of identification sources for horizon scanning

Published online by Cambridge University Press:  29 June 2010

Joanna Smith ,
Alison Cook  and
Claire Packer
Joanna Smith
Affiliation:
University of Birmingham
Alison Cook
Affiliation:
University of Birmingham
Claire Packer
Affiliation:
University of Birmingham
Get access Rights & Permissions [Opens in a new window]

Abstract

Objectives: The English National Horizon Scanning Centre routinely scans thirty-five sources to identify new and emerging health technologies. The aim of the study was to develop and apply evaluation criteria and scores to assess the value of sources, and to identify a cutoff score below which sources would be recommended for removal from routine horizon scanning. Criteria to evaluate each source scanned could result in a more efficient approach in the selection process.

Methods: Evaluation criteria were developed following a review of the literature and discussions with horizon analysts. Proposed criteria were piloted on a random selection of six sources, and then applied to all thirty-five sources. The criteria were assessed using the Analytic Hierarchy Process.

Results: Eight criteria were identified as being most relevant for assessing the value of scanning sources. The three most important (primary) criteria were coverage (approximate percentage of relevant information), quality (reliable, accurate, objective), and efficiency (estimated time to identify one potentially significant health technology or other relevant information). Seven sources fell beneath the cutoff score and were recommended for removal from routine scanning.

Conclusions: The criteria were considered useful in the assessment of current sources, and have the potential to be used to assess new ones. These criteria may be useful for other horizon scanning centers to pilot and validate.

Keywords

Horizon scanning systemsHealth technology assessmentNew and emerging health technologiesPriority setting

Information

Type
METHODS
Information
International Journal of Technology Assessment in Health Care , Volume 26 , Issue 3 , July 2010 , pp. 348 - 353
Copyright
Copyright © Cambridge University Press 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

REFERENCES

1. Douw, K, Vondeling, H. Selecting new technologies for evaluation: Can clinical experts predict which new anticancer drugs will impact Danish health care. Soc Sci Med. 2007;64:283–286.CrossRefGoogle ScholarPubMed
2. Douw, K, Vondeling, H, Eskildsen, D, et al. Use of the Internet in scanning the horizon for new and emerging health technologies: A survey of agencies involved in horizon scanning. J Med Internet Res. 2003;5:e6.CrossRefGoogle ScholarPubMed
3. Formulary. Advanstar Communications. http://www.formulary-journal.modernmedicine.com/ (accessed May 6, 2009).Google Scholar
4. Packer, C. The National Horizon Scanning Centre (NHSC): Early warning for new and emerging health technologies in England. Evid Based Healthc Public Health. 2005;9:410–413.CrossRefGoogle Scholar
5. Perras, C, Topfer, LA. A review of sources of new drug information used for horizon scanning. The Canadian Agency for Drugs and Technologies in Health (CADTH). Book of Abstracts, IV Annual Meeting Health Technology Assessment International (HTAI), Barcelona; 2007:92, S22–01.Google Scholar
6. Publichealthconferences.org.uk. Modified Portsmouth Scorecard. http://www.publichealthconferences.org.uk/health_services_2007/downloads/Modified_Portsmouth_Scorecard_v1.pdf (accessed May 6, 2009).Google Scholar
7. Robert, G, Gabbay, J, Stevens, A. Which are the best information sources for identifying emerging health care technologies? An international Delphi survey. Int J Technol Assess Health Care. 1998;14:636–643.CrossRefGoogle ScholarPubMed
8. Saaty, TL. The analytic hierarchy process. New York: McGraw Hill; 1980.Google Scholar
9. Stevens, A, Milne, R, Lilford, R, et al. Keeping pace with new technologies: Systems needed to identify and evaluate them. Br Med J. 1999;319:1291.CrossRefGoogle Scholar
10. Thygesen, M, Vondeling, H, Douw, K, et al. Development of an Internet search strategy to identify new medical technologies in prostate cancer. Ugeskr Laeger. 2007;169:1912–1916.Google ScholarPubMed
11. Wild, C, Langer, T. Emerging health technologies: Informing and supporting health policy early. Health Policy. 2008;87:160–171.CrossRefGoogle ScholarPubMed