Very few practical frameworks exist to guide the formulation of recommendations at hospital-based health technology assessment (HTA) units. The objectives of our study were: (i) to identify decision criteria specific to the context of hospital-based health technologies and interventions, (ii) to estimate the extent to which the expert community agrees on the importance of the identified criteria, (iii) to incorporate the identified criteria into a decision-aid tool, and (iv) to illustrate the application of a prototype decision-aid tool.

Relevant decision criteria were identified using existing frameworks for HTA recommendations, our past experience, a literature search, and feedback from a survey of diverse stakeholders.

Based on the survey results, twenty-three decision criteria were incorporated into the final framework. We defined an approach that eschewed a scoring system, but instead relied on a visual means for arriving at a final recommendation, by juxtaposing the importance rating for each criterion against the results of the health technology assessment. For a technology to be approved, a majority of criteria considered important should also have received favorable findings.

We created a simple and practical decision-aid tool that incorporates all decision criteria relevant to a hospital-based HTA unit. With its ease of use and accessibility, our tool renders the subjective decision-making process more structured and transparent.

As healthcare decision makers continue to face challenges in health services delivery to their patients, disinvestment programs are being established for a sustainable healthcare system. This study aimed to collect data and information by means of a survey of disinvestment candidates and ongoing disinvestment projects in the health technology assessment (HTA) community.

An online survey was conducted to collect information on disinvestment candidates and activities from members of the Health Technology Assessment International Disinvestment & Early Awareness Interest Group, the EuroScan International Network and International Network of Agencies for Health Technology Assessment.

Among the 362 invitees, twenty-four unique responses were received, and almost 70 percent were involved in disinvestment initiatives. The disinvestment candidates identified represented a range of health technologies. Evidence or signaling of clinical ineffectiveness or inappropriate use typically led to the nomination of disinvestment candidates. Health technology assessments and reassessments were usually conducted to evaluate the technology in question, and decisions usually led to the limited use of the technology. Barriers to disinvestment decisions included the strength of interest and advocacy groups, insufficient data for assessments, a systematic decision process and political challenges, while obstacles to their implementation were clinicians’ reluctance and insufficient funding and incentives.

The survey results suggested that disinvestment activities are occurring in the HTA community, especially in the public sector. Future research can further investigate the processes and methods used to reach and implement disinvestment decisions from our survey respondents and explore to form closer ties between the HTA and clinical communities.

There is no established methodology to assess the feasibility of medicine price data sources. Against this backdrop, a framework to guide the selection of most appropriate price data sources for pharmacoeconomic research has been developed.

A targeted literature review was carried out. Dimensions discussed in literature as relevant for medicine price comparisons and practical experience of the authors in medicine price studies informed the conceptional work of the framework development. A draft version of the framework was reviewed by peer pricing experts. The feasibility of the framework was tested in case studies.

According to the developed framework (called Re-ADAPT), a medicine price data source should meet the following criteria: reliability and sustainability; accessibility at a cost that users can afford; provision of medicine price information at the date(s) required; information for the defined geographic area, or at least in a representative way; coverage of the pharmaceuticals and at the price type(s) required. Easy handling and provision of additional information were defined as supportive assets of candidate data sources (secondary criteria). The case studies confirmed the feasibility of the Re-ADAPT framework. In some cases, however, it can be difficult to disentangle assessment criteria (particularly geographic area, scope of pharmaceuticals and price types) for separate consideration, given their interlinkage.

While selection of the most appropriate data sources will remain a challenge, the Re-ADAPT framework aims to provide practical guidance and thus contribute to a more careful, balanced, and evidence-based selection of data sources for medicine price studies.

Social and cultural aspects are rarely assessed in health technology assessments (HTA), despite being part of most HTA definitions. One hypothesis for the reason why they are hardly considered in HTA is that we lack relevant assessment methods. Accordingly, this review aims at providing an overview of methodological approaches to address social and cultural aspects related to health technologies in HTA.

We conducted a comprehensive literature search by searching fourteen databases and a hand-search of two pertinent journals. Additionally, we sent a query to all member agencies of the International Network of Agencies for Health Technology Assessment (INAHTA) asking them for methods they use to assess social and cultural aspects.

A total of 125 publications met our inclusion criteria. We grouped the methodological approaches into checklists for experts, literature reviews, stakeholder participatory approaches, primary data collection methods, and combinations of methodological approaches.

There is a wide variety of methods available for assessing social and cultural aspects of health technologies, some of which have been applied in HTA. The presented overview of the different approaches and their merits can facilitate the assessment of these aspects, and improve the knowledge regarding (potential) success and failure of the implementation of a health technology.

The aim of this study was to develop a feasible and effective strategy to involve patients in the Spanish Network of Agencies of Health Technology Assessment (RedETS).

The framework for patient involvement (PI) in the assessment activities and processes of RedETS were developed through a research project that included: (i) a systematic search of the international literature describing a strategy and/or a methodology linking health technology assessment (HTA) and PI; (ii) a qualitative study through interviews with RedETS members to analyze the perceptions of PI among HTA managers in the Spanish context; (iii) a Delphi consultation with three large platforms of patients, carers and consumer organizations in Spain about their perspectives of PI; (iv) a consensus process with the members of the RedETS Governing Council to define the final strategy.

Three main themes were identified in the literature and Web site review: (i) PI methods for the different HTA phases; (ii) Participant definition and selection; (iii) Resources needed. A three-step implementation strategy was proposed: (i) short-term actions: piloting and testing patient participation in HTA and building patients' capacity; (ii) medium-term actions: broadening the participation of patients, and building internal capacity; (iii) long-term actions: consolidating and mainstreaming patient involvement

Patient participation can be incorporated into almost all the HTA phases and products with greater or lesser degrees of difficulty. However, a progressive implementation strategy is suggested for a feasible PI process.

Population-based colorectal cancer (CRC) screening programs that use a fecal immunochemical test (FIT) are often faced with a noncompliance issue and its subsequent waiting time (WT) for those FIT positives complying with confirmatory diagnosis. We aimed to identify factors associated with both of the correlated problems in the same model.

A total of 294,469 subjects, either with positive FIT test results or having a family history, collected from 2004 to 2013 were enrolled for analysis. We applied a hurdle Poisson regression model to accommodate the hurdle of compliance and also its related WT for undergoing colonoscopy while assessing factors responsible for the mixture of the two outcomes.

The effect on compliance and WT varied with contextual factors, such as geographic areas, type of screening units, and level of urbanization. The hurdle score, representing the risk score in association with noncompliance, and the WT score, reflecting the rate of taking colonoscopy, were used to classify subjects into each of three groups representing the degree of compliance and the level of health awareness.

Our model was not only successfully applied to evaluating factors associated with the compliance and the WT distribution, but also developed into a useful assessment model for stratifying the risk and predicting whether and when screenees comply with the procedure of receiving confirmatory diagnosis given contextual factors and individual characteristics.

Evidence requirements and assessment methods access differ between health technology assessment (HTA) agencies. The HTA Core Model® provides a standardized approach to HTA, targeting evidence sharing and collaboration between participating HTA bodies. It is fit for purpose from an industry perspective and was used by pharmaceutical company Roche to develop a framework for internal assessment of evidence required for market access and coverage/reimbursement (“access evidence”).

Tools were developed to systematically scope, assess, plan, and summarize access evidence generation. The tools were based mainly on the first four HTA Core Model® domains and rolled-out in selected development teams in 2017. Five months after full implementation, the impact of tools was assessed in an internal survey.

Systematic access evidence generation started with the Access Evidence Questionnaire, to scope evidence requirements and identify evidence gaps. Findings were summarized in the Access Evidence Metric, which assessed the alignment of available/planned evidence against HTA bodies’ requirements and developed scope mitigation strategies. The Access Evidence Plan was then used to plan and document (additional) evidence generation. Once generated, evidence was summarized in the Access Evidence Dossier. A survey of twenty-seven Roche employees involved in evidence generation showed that the tools made discussions around access strategies and evidence more efficient and transparent.

The HTA Core Model® provided a useful framework around which to optimize internal evidence generation and assessment. The benefits of using a standardized HTA approach in industry mirror those expected from implementing the HTA Core Model® in HTA agencies.

Early assessment can assist in allocating resources for innovation effectively and produce the most beneficial technology for an institution. The aim of the present study was to identify methods and discuss the analytical approaches applied for the early assessment of innovation in a healthcare setting.

Knowledge synthesis based on a structured search (using the MEDLINE, Embase, and Cochrane databases) and thematic analysis was conducted. An analytical framework based on the stage of innovation (developmental, introduction, or early diffusion) was applied to assess whether methods vary according to stage. Themes (type of innovation, study, analysis, study design, method, and main target audience) were then decided among the authors. Identified methods and analysis were discussed according to the innovation stage.

A total of 1,064 articles matched the search strategy. Overall, thirty-nine articles matched the inclusion criteria. The use of methods has a tendency to change according to the stage of innovation. Stakeholder analysis was a prominent method in the innovation stages and particularly in the developmental stage, as the introduction and early diffusion stage has more availability of data and may apply more complex methods. Barriers to the identified methods were also discussed as all of the innovation stages suffered from lack of data and substantial uncertainty.

Although this review has identified applicable approaches for early assessment in different innovation stages, research is required regarding the value of the available data and methods and tools to enhance interactions between different parties at different stages of innovation.

The aim of this study was to describe patient level costing methods and develop a database of healthcare resource use and cost in patients with AHF receiving ventricular assist device (VAD) therapy.

Patient level micro-costing was used to identify documented activity in the years preceding and following VAD implantation, and preceding heart transplant for a cohort of seventy-seven consecutive patients listed for heart transplantation (2009–12). Clinician interviews verified activity, established time resource required for each activity, and added additional undocumented activities. Costs were sourced from the general ledger, salary, stock price, pharmacy formulary data, and from national medical benefits and prostheses lists. Linked administrative data analyses of activity external to the implanting institution, used National Weighted Activity Units (NWAU), 2014 efficient price, and admission complexity cost weights and were compared with micro-costed data for the implanting admission.

The database produced includes patient level activity and costs associated with the seventy-seven patients across thirteen resource areas including hospital activity external to the implanting center. The median cost of the implanting admission using linked administrative data was $246,839 (interquartile range [IQR] $246,839–$271,743), versus $270,716 (IQR $211,740–$378,482) for the institutional micro-costing (p = .08).

Linked administrative data provides a useful alternative for imputing costs external to the implanting center, and combined with institutional data can illuminate both the pathways to transplant referral and the hospital activity generated by patients experiencing the terminal phases of heart failure in the year before transplant, cf-VAD implant, or death.

This study investigated which databases and which combinations of databases should be used to identify economic evaluations (EEs) to inform systematic reviews. It also investigated the characteristics of studies not identified in database searches and evaluated the success of MEDLINE search strategies used within typical reviews in retrieving EEs in MEDLINE.

A quasi-gold standard (QGS) set of EEs was collected from reviews of EEs. The number of QGS records found in nine databases was calculated and the most efficient combination of databases was determined. The number and characteristics of QGS records not retrieved from the databases were collected. Reproducible MEDLINE strategies from the reviews were rerun to calculate the sensitivity and precision for each strategy in finding QGS records.

The QGS comprised 351 records. Across all databases, 337/351 (96 percent) QGS records were identified. Embase yielded the most records (314; 89 percent). Four databases were needed to retrieve all 337 references: Embase + Health Technology Assessment database + (MEDLINE or PubMed) + Scopus. Four percent (14/351) of records could not be found in any database. Twenty-nine of forty-one (71 percent) reviews reported a reproducible MEDLINE strategy. Ten of twenty-nine (34.5 percent) of the strategies missed at least one QGS record in MEDLINE. Across all twenty-nine MEDLINE searches, 25/143 records were missed (17.5 percent). Mean sensitivity was 89 percent and mean precision was 1.6 percent.

Searching beyond key databases for published EEs may be inefficient, providing the search strategies in those key databases are adequately sensitive. Additional search approaches should be used to identify unpublished evidence (grey literature).

When making decisions in health care, it is essential to consider economic evidence about an intervention. The objective of this study was to analyze the methods applied for systematic reviews of health economic evaluations (SR-HEs) in HTA and to identify common challenges.

We manually searched the Web pages of HTA organizations and included HTA-reports published since 2015. Prerequisites for inclusion were the conduct of an SR-HE in at least one electronic database and the use of the English, German, French, or Spanish language. Methodological features were extracted in standardized tables. We prepared descriptive statistical (e.g., median, range) measures to describe the applied methods. Data were synthesized in a structured narrative way.

Eighty-three reports were included in the analysis. We identified inexplicable heterogeneity, particularly concerning literature search strategy, data extraction, assessment of quality, and applicability. Furthermore, process steps were often missing or reported in a nontransparent way. The use of a standardized data extraction form was indicated in one-third of reports (32 percent). Fifty-four percent of authors systematically appraised included studies. In 10 percent of reports, the applicability of included studies was assessed. Involvement of two reviewers was rarely reported for the study selection (43 percent), data extraction (28 percent), and quality assessment (39 percent).

The methods applied for SR-HEs in HTA and their reporting quality are very heterogeneous. Efforts toward a detailed, standardized guidance for the preparation of SR-HEs definitely seem necessary. A general harmonization and improvement of the applied methodology would increase the value of SR-HE for decision makers.

The HTA Core Model® was developed to improve the transferability of health technology assessment (HTA) between settings. The model has been used by HTA agencies but is also of interest to manufacturers, for improving internal evidence generation and communicating with other HTA stakeholders. To establish if the model is fit for purpose from an industry perspective, the pharmaceutical company Roche, collaborating with the European Network for HTA (EUnetHTA), conducted an assessment of the model.

A questionnaire was developed to evaluate all assessment elements in the HTA Core Model v2.0 for their usefulness in meeting payers’ evidence needs and demonstrating value. The questionnaire was completed by country affiliate teams working in evidence generation and reimbursement submissions for pharmaceuticals. Survey results were discussed in workshops to ensure consistency and alignment between teams.

The questionnaire was completed by six teams. An additional team from global pricing and market access participated in workshops. Model domains pertaining to the health problem and current technology use, technology description, clinical effectiveness, and economic value were considered most important because they meet payers’ evidence needs. Overall, the model was considered useful to improve the efficiency of HTA evidence generation, share evidence internally, and communicate value to payers and HTA agencies.

From an industry perspective, the HTA Core Model provides a useful framework and common terminology for efficient generation of transferable HTA evidence. The timeliness, efficiency, and transparency of HTA processes could be improved by a more standardized approach to HTA across settings.

Integration of ethics into health technology assessment (HTA) remains challenging for HTA practitioners. We conducted a systematic review on social and methodological issues related to ethical analysis in HTA. We examined: (1) reasons for integrating ethics (social needs); (2) obstacles to ethical integration; (3) concepts and processes deployed in ethical evaluation (more specifically value judgments) and critical analyses of formal experimentations of ethical evaluation in HTA.

Search criteria included “ethic,” “technology assessment,” and “HTA”. The literature search was done in Medline/Ovid, SCOPUS, CINAHL, PsycINFO, and the international HTA Database. Screening of citations, full-text screening, and data extraction were performed by two subgroups of two independent reviewers. Data extracted from articles were grouped into categories using a general inductive method.

A list of 1,646 citations remained after the removal of duplicates. Of these, 132 were fully reviewed, yielding 67 eligible articles for analysis. The social need most often reported was to inform policy decision making. The absence of shared standard models for ethical analysis was the obstacle to integration most often mentioned. Fairness and Equity and values embedded in Principlism were the values most often mentioned in relation to ethical evaluation.

Compared with the scientific experimental paradigm, there are no settled proceedings for ethics in HTA nor consensus on the role of ethical theory and ethical expertise hindering its integration. Our findings enable us to hypothesize that there exists interdependence between the three issues studied in this work and that value judgments could be their linking concept.

The aim of this study was to identify guidelines and assessment tools used by health technology agencies for quality assurance of registries and investigate the current use of registry data by HTA organizations worldwide.

As part of a European Network for Health Technology Assessment Joint Action work package, we undertook a literature search and sent a questionnaire to all partner organizations on the work package and all organizations listed in the International Society for Pharmaco-economics and Outcomes Research directory.

We identified thirteen relevant documents relating to quality assurance of registries. We received fifty-five responses from organizations representing twenty-one different countries, a response rate of 40.5 percent (43/110). Many agencies, particularly in Europe, are already drawing on a range of registries to provide data for their HTA. Less than half, however, use criteria or standards to assess the quality of registry data. Nearly all criteria or standards in use have been internally defined by organizations rather than referring to those produced by an external body. A comparison of internal and external standards identified consistency in several quality dimensions, which can be used as a starting point for the development of a standardized tool.

The use of registry data is more prevalent than expected, strengthening the need for a standardized registry quality assessment tool. A user-friendly tool developed in conjunction with stakeholders will support the consistent application of approved quality standards, and reassure critics who have traditionally considered registry data to be unreliable.

Multi-analyte assays with algorithmic analyses (MAAAs) use combinations of circulating and clinical markers including omics-based sources for diagnostic and/or prognostic purposes. Assessing MAAAs is challenging under existing health technology assessment (HTA) methods or practices. We undertook a scoping review to explore the HTA methods used for MAAAs to identify the criteria used for clinical research and reimbursement purposes.

This review included only non-companion (stand-alone) tests that are actionable and that have been evaluated by leading HTA or insurer/reimbursement bodies up to September 2017.

Twenty-five reports and articles evaluating seventeen MAAAs were examined, most of which have been developed in oncology. The two main models used were the EUnetHTA Core model and the Evaluation of Genomic Applications in Practice and Prevention ACCE framework. Clinical validity and utility criteria were used, as were economic, ethical, legal, and social aspects. Economic evidence on MAAAs was scarce, and there is no consensus on whether the perspectives used are sufficiently broad to include all relevant stakeholders.

Clinical utility and efficiency were the most used criteria, with stronger evidence needed linking the use of the algorithm with the clinical outcomes in real-life practice. HTA bodies must as well consider questions related to the analytical validity of MAAAs or with organizational aspects. The two main models, the EUnetHTA Core model and the ACCE framework, could be adapted to the assessment of MAAAs.

A primary outcome in oncology trials is overall survival (OS). However, to estimate OS accurately requires a sufficient number of patients to have died, which may take a long time. If an alternative end point is sufficiently highly correlated with OS, it can be used as a surrogate. Progression-free survival (PFS) is the surrogate most often used in oncology, but does not always satisfy the correlation conditions for surrogacy. We analyze the methodologies used when extrapolating from PFS to OS.

Davis et al. previously reviewed the use of surrogate end points in oncology, using papers published between 2001 and 2011. We extend this, reviewing papers published between 2012 and 2016. We also examine the reporting of statistical methods to assess the strength of surrogacy.

The findings from 2012 to 2016 do not differ substantially from those of 2001 to 2011: the same factors are shown to affect the relationship between PFS and OS. The proportion of papers reporting individual patient data (IPD), strongly recommended for full assessment of surrogacy, remains low: 33 percent. A wide range of methods has been used to determine the appropriateness of surrogates. While usually adhering to reporting standards, the standard of scholarship appears sometimes to be questionable and the reporting of results often haphazard.

Standards of analysis and reporting PFS to OS surrogate studies should be improved by increasing the rigor of statistical reporting and by agreeing to a minimum set of reporting guidelines. Moreover, the use of IPD to assess surrogacy should increase.

In 2007, the Ontario Health Technology Advisory Committee (OHTAC) developed a decision framework to guide decision making around nondrug health technologies. In 2012, OHTAC commissioned a revision of this framework to enhance its usability and deepen its conceptual and theoretical foundations.

The committee overseeing this work used several methods: (a) a priori consensus on guiding principles, (b) a scoping review of decision attributes and processes used globally in health technology assessment (HTA), (c) presentations by methods experts and members of review committees, and (d) committee deliberations over a period of 3 years.

The committee adopted a multi-criteria decision-making approach, but rejected the formal use of multi-criteria decision analysis. Three broad categories of attributes were identified: (I) context criteria attributes included factors such as stakeholders, adoption pressures from neighboring jurisdictions, and potential conflicts of interest; (II) primary appraisal criteria attributes included (i) benefits and harms, (ii) economics, and (iii) patient-centered care; (III) feasibility criteria attributes included budget impact and organizational feasibility.

The revised Ontario Decision Framework is similar in some respects to frameworks used in HTA worldwide. Its distinctive characteristics are that: it is based on an explicit set of social values; HTA paradigms (evidence based medicine, economics, and bioethics/social science) are used to aggregate decision attributes; and that it is rooted in a theoretical framework of optimal decision making, rather than one related to broad social goals, such as health or welfare maximization.

Severe hypercholesterolemia is a major cause of death in coronary heart disease. New adjunctive drug therapies have passed authorization processes and been launched recently. So far it is not known which properties of the new treatment options generate the highest benefit for patients. The aim was to evaluate patient priorities in the field of adjunctive drug therapy with apheresis. Therapy characteristics were examined as to their relevance to hypercholesterolemia patients.

To identify all potential patient-relevant treatment characteristics, a systematic literature review and ten interviews with patients were conducted. Seven key characteristics were identified from the patient perspective. Patients’ priorities were elicited using an analytic hierarchy process (AHP).

In total, N = 61 patients diagnosed with severe hypercholesterolemia and undergoing apheresis participated in the study. The analysis showed predominance for the attribute “reduction of LDL-C level in blood” (Wglobal:0.362). The “risk of myopathy” (Wglobal:0.164), “risk of neurocognitive impairment” (Wglobal:0.161) and “frequency of apheresis” (Wglobal:0.119) were ranked second, third and fourth. Subgroup analyses revealed that “frequency of apheresis” is of greater importance to younger patients, men and/or patients who indicated a reduction in quality of life due to apheresis.

The essential decision criteria for optimal therapy from the patients’ perspective were obtained. “Reduction of lipoprotein in blood” was ranked highest compared with the “mode of administration” and “side effects” characteristics. The study offers a transparent approach for the identification of patient priorities for adjunctive PCSK9-inhibitor therapy in apheresis-treated hypercholesterolemia. The project can be used by healthcare decision makers to understand the importance of each patient-relevant endpoint.

Current health technology assessment (HTA) methods guidelines for medical devices may benefit from contributions by biomedical and clinical engineers. Our study aims to: (i) review and identify gaps in the current HTA guidelines on medical devices, (ii) propose recommendations to optimize the impact of HTA for medical devices, and (iii) reach a consensus among biomedical engineers on these recommendations.

A gray literature search of HTA agency Web sites for assessment methods guidelines on devices was conducted. The International Federation of Medical and Biological Engineers (IFMBE) then convened a structured focus group, with experts from different fields, to identify potential gaps in the current HTA guidelines, and to develop recommendations to fill these perceived gaps. The thirty recommendations generated from the focus group were circulated in a Delphi survey to eighty-five biomedical and clinical engineers.

Thirty-two panelists, from seventeen countries, participated in the Delphi survey. The responses showed a strong agreement on twenty-seven of thirty recommendations. Some uncertainties remain about the methods to accurately assess the effectiveness and safety, and interoperability of a medical device with other devices or within the clinical setting.

As medical devices differ from drug therapies, current HTA methods may not accurately reflect the conclusions of their assessment. Recommendations informed by the focus group discussions and Delphi survey responses aimed to address the perceived gaps, and to provide a more integrated approach in medical device assessments in combining engineering with other perspectives, such as clinical, economic, patient, human factors, ethical, and environmental.

The Health Technology Expert Review Panel is an advisory body to Canadian Agency for Drugs and Technologies in Health (CADTH) that develops recommendations on health technology assessments (HTAs) for nondrug health technologies using a deliberative framework. The framework spans several domains, including the environmental impact of the health technology(ies). Our research objective was to identify articles on frameworks, methods or case studies on the environmental impact assessment of health technologies.

A literature search in major databases and a focused gray literature search were conducted. The main search concepts were HTA and environmental impact/sustainability. Eligible articles were those that described a conceptual framework or methods used to conduct an environmental assessment of health technologies, and case studies on the application of an environmental assessment.

From the 1,710 citations identified, thirteen publications were included. Two articles presented a framework to incorporate environmental assessment in HTAs. Other approaches described weight of evidence practices and comprehensive and integrated environmental impact assessments. Central themes derived include transparency and repeatability, integration of components in a framework or of evidence into a single outcome, data availability to ensure the accuracy of findings, and familiarity with the approach used.

Each framework and methods presented have different foci related to the ecosystem, health economics, or engineering practices. Their descriptions suggested transparency, repeatability, and the integration of components or of evidence into a single outcome as their main strengths. Our review is an initial step of a larger initiative by CADTH to develop the methods and processes to address the environmental impact question in an HTA.