We list here details on the many collaborative projects which are led by the Hubs, and funded by the Network. Details on current funding opportunities can be found here.
Network-funded projects have included workshops, primary research, development of guidance and training events. The summary below captures the projects by year of funding and as projects are completed supporting documents and outputs will be added for more information on this research.
We have also funded a cohort of PhD studentships in trials methodology research, and support eight Working Groups.
Recruitment and retention challenges are well documented and have been identified as key methodological research priorities by UK CTUs. Whilst this focus has led to an increase in the quantity of research directed at these challenges, evidence for effective interventions is limited and navigating the literature to identify strategies relevant to different types of trials remains difficult. The flagship output from the HTMR recruitment working group was the launch of an online searchable database of recruitment research in 2016, helping trialists to identify interventions and areas for future methodological research. Project delivery engaged 24 researchers from 7 institutions and 3 countries. 1139 users from 18 countries have undertaken 1061 searches during the first nine months. The database is currently being updated with 2015-2016 publications (completion Nov 2017). However, no such resource exists for retention research. Retention may be influenced by recruitment methods but this aspect is often unexplored. We will develop ORRCA2 to organise and map retention research, linking with ORRCA to explore connections and overlap between recruitment and retention research.
Search strategies for major databases will be adapted from the Cochrane Methodology Review of retention interventions. Eligible studies will include retention strategy evaluations, descriptive studies identifying risk factors, qualitative research and relevant case reports. Articles exploring treatment adherence or statistical methods for handling missing data will be excluded.
A matrix of retention research domains and a searchable database will be developed as per ORRCA. A mapping exercise will explore current methodological research topics and the potential to evaluate recruitment and retention simultaneously outcomes in future SWATs or nested randomised studies. Text mining methodology will be explored to increase sustainability of ORRCA and ORRCA 2.
A significant challenge in the evaluation of all phases of surgical device development is outcome reporting. Numerous outcomes are reported, making data synthesis difficult and risking outcome reporting bias. RCTs have methodologically improved in part due to the development of COSs. However, these have focused on specific conditions and/or surgical procedures. The COMET database provides details of COSs that have been developed or are under development. At present, there are no COS specifically developed to aid earlier and later phase studies on innovative surgical technologies and devices. Consequently, it is difficult for stakeholders to compare key outcomes. Adverse events may be under reported (reporting bias) or ill-defined. It is therefore difficult to judge whether to proceed to full evaluation or abandon new technology.
The proposed workshop is intended for the first time to bring together key stakeholders in the development and evaluation of surgical technologies and devices to consider mandated core outcome reporting. It will describe the current problem with outcome reporting in this field and the pitfalls, present COSs as a solution to this problem to use from early to later phase trials and consider methods for live outcome reporting in registries and on-line journals to optimise learning. The workshop will establish whether it will be feasible to develop a generic COS for the reporting of all surgical devices and establish next steps to achieve this.
Classically, single-arm trial designs have been the standard approach in oncology research for conducting a phase II clinical trial. In recent years however, acknowledgement of the limitations associated with single-arm studies has seen an increased call for the use of randomisation in phase II. Indeed, numerous discussion articles have now put forward arguments for and against the use of single-arm designs in this setting, and several simulation studies have been conducted to try to identify which approach should be preferred. At the very least, these presentations together indicated that the percentage of phase II trials using randomisation should increase.
Nonetheless, a large number of phase II trials continue to be conducted using single-arm designs, with a contemporary review indicating at least 50% of UK Clinical Trial Units (CTUs) had recently been involved in such a study (Jaki 2013, Clin Trials 10:344-46). Therefore, it is possible that the design of many publicly funded trials remains sub-optimal for a reason that could easily be rectified.
Accordingly, in this project, we will first circulate a questionnaire to each of the CTUs within the UKCRC Registered CTU Network, with the aim of establishing key factors behind their choice to utilise either a single-arm or randomised approach in any phase II clinical trials they have been involved in. Following the completion of this survey, amongst those expressing interest, the CTUs responsible for conducting the largest number of phase II trials will be visited for more in-depth follow-up meetings.
Finally, the results of these discussions will, in combination with the expertise of several researchers, aid the development of a guidance document on the recommended contemporary design of phase II trials. This document should be of great value to the trials community for helping ensure future phase II trials are designed in the most appropriate manner.
Randomised trials typically involve collecting pre-randomisation information about key demographic and clinical characteristics of participants. Randomisation of treatment ensures that participants in the different arms of the trial are comparable, which means that an unadjusted comparison of outcomes between arms provides an unbiased estimate of the treatment effect. However, adjusting for pre-randomisation measurements within a regression model may increase the statistical power. Whether an adjusted or unadjusted analysis should be performed as the primary analysis remains a contentious issue.
Current guidelines regarding the optimal way to incorporate covariates into the primary analysis of randomised trials recommend including a limited number of pre-specified covariates, and explicitly recommend against data-adaptive model selection procedures and including interactions between treatment and covariates into the model. However, some theoretical work suggests that data-adaptive approaches may have benefits, and including interactions can help protect against model misspecification. Further, most of the theoretical work underpinning guidelines is based on continuous outcome measures, but many randomised trials have time-to-event outcomes.
The aim of the proposed work is to re-analyse a small number of published trials, using a range of covariate-adjustment analysis strategies, focusing particularly on time-to-event outcomes, in order to explore the benefits and limitations of each approach. Key issues uncovered will be subsequently explored in simulation studies. We aim to provide practical guidance to trialists regarding the optimal way to account for baseline covariates in the primary analysis of a randomised trial.
To identify the key content and develop a template for health economic analysis plans that will guide analysts in conducting economic evaluations alongside randomised controlled trials (RCTs).
The use of SAPs, drawn up in advance of the analysis phase of a trial, is an accepted means of reducing bias in reporting the results of RCTs. However, while health economic analysis plans (HEAPs) to guide trialists in conducting economic evaluations alongside RCTs are becoming more widespread, they lag behind SAPs in terms of standardisation and acceptance. In October 2015, an HTMR-funded workshop involving fifty (predominantly academic) participants was held to discuss some of the issues associated with HEAPs. Feedback from the workshop suggested that health economists would value guidance and clarity on the appropriate content of a HEAP.
Building on recent work led by the NW Hub to create guidance for SAPs, we propose to develop a template for a HEAP. We plan to use ‘real time’ Delphi methodology, which involves presenting dynamic feedback to participants, to gain a consensus opinion on the relevant content of a HEAP.
A literature review will identify published HEAPs and additional examples will be sought from practising health economists in order to derive a list of items for inclusion in an electronic Delphi survey. A panel of experts will be recruited and, after seeding the survey with responses from randomly selected attendees of the workshop, the survey will be opened to the panel for a period of one month. The project team will convene at the end of the survey to discuss the results with invited participants in a consensus meeting. The final list of items will be developed into a template HEAP, which will be disseminated widely.
Previous work has identified the need for reporting guidelines for trials using adaptive designs. The CONSORT guidance framework has substantially improved the reporting of randomised controlled trials. However, there is no guidance tailored for the reporting of adaptive trials. This proposal is part of a larger project that will develop and disseminate a CONSORT extension for adaptive trials.
This proposal will improve the quality and international impact of the developed guidance through three activities.
The first is a consensus workshop that will bring together around 30 multidisciplinary experts in adaptive trials from around the world. The workshop participants will consist of representatives from academia, industry and regulatory agencies. By including international representatives in person (rather than via telephone), we believe the developed guidance will be of much higher quality and more relevant to a worldwide audience.
The second is a dissemination workshop that will present the guidance to a UK audience. We will invite around 60 individuals that represent a variety of stakeholders including medical journal editors, academic trialists and industry representatives.
The third is to present the guidance at the SCT conference in 2018, which will help disseminate the guidance beyond the UK.
We expect the development of the guidance to raise important gaps in methodology that need to be addressed. Thus, the dissemination workshop will also involve discussions to identify these. In the longer term we hope that this may lead to additional useful methodology research.
Based on this work we will be able to provide guidance for researchers on the most suitable approach to use. We will also aim to identify gaps in methodology that future work can address.
Within the last decade, significant developments in the field of mixed methods mean that literature on integration is now widely available. Moreover, in the last year, clinical trials experts have predicted that the use of mixed methods will gather pace over the next few years where the key challenge will lie in integration. As this juncture, there is an opportunity for the HTMR to accelerate this development.
Up to twenty experts in mixed methods and clinical trials will attend a two day summit on the integration of qualitative and quantitative methods in RCTs. Day one will involve presentation and facilitated discussion leading to an authoritative overview of the current strengths and weaknesses in the integration of mixed methods in clinical trials. On day two, attendees will identify the next steps required to provide the trials community with guidance on integration. Project outputs will comprise (i) an open access position paper on integration in clinical trials and (ii) an application to the MRC Methodology Research Programme in June 2017 for funds to develop guidance. By convening the summit, the HTMR will help equip the trials community with new skills and techniques in the integration of quantitative and qualitative methods that can be applied by researchers in their own practice.
Aims: This project aims to 1) identify the key elements leading to successful trials conducted by surgical TRCs and 2) synthesise these findings to develop strategies to enhance clinician engagement in trials across other clinical specialties, including clinician training.
Methods: A qualitative research study to inform a one day trialist stakeholders workshop. The research will include: 1) non-participant observation of TRC-linked surgical trials and TRC meetings, and 2) in-depth semi-structured interviews with key clinicians and trials units staff. Trials and linked personnel will be purposefully sampled across three geographical locations to include a variety of surgical specialties, clinicians and TRCs. Meetings and interviews will be audio-recorded and transcribed. Thematic analysis of transcripts will use the constant comparative method.
Outputs: Results will be synthesised and reviewed by co-applicants. These will inform a meeting with key trialists and clinical stakeholders to develop strategies to enhance clinicians’ engagement in trials which could subsequently be tested in other specialties and inform post-graduate clinician training. Peer-review publications and presentations.
We plan a piece of literature work (analyses of NIHR HTA funded main trials with an internal pilot study to explore their decision making in relation to progression criteria) and a two-day event. The combined activities will build on our individual expertise and lead to guidance documents to be published, material to be made available on the web and further meetings with clinical trialists and funding bodies to widely implement the recommendations for when to conduct each type of pilot and/or feasibility study design.
In the 2013 World Health Report, there was an unequivocal statement that unless low- and middle-income countries (LMICs) become the generators and not the recipients of research data, there will never be any real improvements in public health outcomes in these most underserved regions of the world. This lack of data is the result of too few studies being conducted in low-income settings. Therefore undertaking methodology research is required to identify the gaps and issues in order to optimise design, ease and quality with the aim of making research relevant and accessible to health care workers in these regions. The MRC HTMR Network has established a portfolio of methodology research of relevance to the UK trials community. It is important, and timely, to establish the relevance of this work to LMIC settings, but also to identify gaps where further research would benefit clinical trials in LMICs. The aim of this project is to identify trials methodology research priorities in LMICs. We predict each region can gain valuable lessons from the other.
The MRC/NIHR EME Programme funds clinical studies testing both if an intervention works in a defined population of patients, and providing opportunity to understand disease or treatment mechanisms. However, there remain significant methodological challenges in performing mechanistic evaluations and accounting for departures from randomised allocations in order to assess efficacy (rather than effectiveness). Uptake of the available methods has been limited, though increasing in popularity.
Our aim is to organise a workshop and training day centred on how to use causal methods to understand mechanisms of action for treatments. The workshop will invite experts in the area to identify important topics and challenges and to discuss ways in which these methods might be utilised more in clinical research studies. A representative from the EME board will give an overview of the scheme, and how they consider mechanisms evaluation when evaluating applications. The results of this workshop will inform a training day approximately six months later for researchers who are interested in incorporating mechanistic methods into their clinical studies.
We will hold a one-day workshop in Lancaster in 2016. We will invite key stakeholders from the area, including representatives from the EME board, and discuss issues and suitable methods for different types of intervention: pharmaceutical, psychotherapy, behaviour change, and surgery. The outcomes of the workshop will be used to develop a training day for researchers in clinical studies.
Six months after the one day workshop we will hold a one day training workshop, also provisionally in Lancaster. This workshop will be aimed at clinicians and methodologists new to the area and who are in the process of designing mechanistic studies, and intending to apply for EME funding. We plan to use organisations such as the NIHR Clinical Research Network, the RDS, and EME to advertise to find applicants.
Cost-effectiveness models are used in health economic decision making to compare the costs and effects of competing strategies for the management of disease. These decision recommendations are uncertain due to limitations in the available evidence. Value of information calculations measure the expected improvement in our decision recommendations, on the monetary scale, if we reduce (EVSI) or eliminate (EVPPI) uncertainty by gathering further evidence. EVPPI and EVSI can therefore be used to guide research funding decisions, and inform trial design. However, as EVPPI and EVSI involve the expectation of a maximum of a conditional expectation, 2-level nested Monte Carlo simulation and sometimes additional Markov chain Monte Carlo simulation is necessary. This is very computationally intensive and often impractical.
This project aims to assess the potential of efficient sampling techniques to reduce the computational burden of EVPPI and EVSI. Simulations where the decision doesn’t change contribute nothing to EVPPI and EVSI. One approach is therefore to use importance sampling and stratified sampling schemes to sample more frequently in the space where decisions change, with appropriate re-weighting.
We will develop importance sampling methods for use in the computation of EVPPI and EVSI, and explore their performance on a range of examples. Another approach is to use a method from pricing financial derivatives, such as simple call and put options, which also rely on the estimation a maximum of several processes. We will explore whether numerical techniques developed in this area of mathematical finance, in particular for non-Normal underlying stocks, can be applied to the estimation of the EVPPI and EVSI. We will meet with technical experts in simulation methodology and pricing financial derivatives, to explore how these techniques can be applied to EVPPI and EVSI.
We will then apply the methodology to some illustrative examples, and present the work in a focused meeting.
Phase I dose-escalation studies are essential to determine the safe dosing range of a novel compound. Despite the poor operating characteristics of algorithmic methods such as the 3+3 design, superior model-based strategies are still rarely used. One of the main reasons why these Bayesian adaptive designs are not implemented is the lack of easy-to-use and accessible software. This project seeks to develop software for model-based dose-escalation studies. A standalone, fully documented system will be developed that allows investigators to plan, explore and conduct such studies without the need for technical expertise in the underlying methods. To ensure that the software is fit for purpose, it will be rigorously tested by different user groups (clinical experts, principal investigators, trial managers, statisticians) and training workshops will be held to facilitate uptake.
Stratified medicine has the potential to significantly improve the benefit-risk ratio in the treatment of many diseases. A randomised controlled trial (RCT) design is perceived as the gold standard for demonstrating the clinical utility of a biomarker-guided approach to treatment. To this end, a large number of biomarker-guided trial designs have been proposed in the literature, but navigating through this literature can be challenging and there is little guidance on which design is optimal in a given setting.A systematic review of biomarker-guided trial designs has recently been undertaken by Antoniou, Jorgensen and Kolamunnage-Dona (co-applicants on this proposal), which identified over 200 relevant papers. The review identified that there is significant variability between authors in terms of the terminology used and descriptions of the different designs, which has resulted in significant ambiguity and confusion regarding biomarker-guided trial designs. To address this problem, in this project, we propose to develop a website using interactive visualisation to provide a user-friendly and easily accessible resource for informing those embarking on a biomarker-guided trial on the most optimal design. The website will initially mirror the findings of the systematic review, but in a much more accessible format, and will subsequently be extended to provide a truly interactive tool allowing searches for the optimal design in a given setting as well as sample size estimates. The idea for the project has stemmed from feedback from attendees of conferences and meetings where the systematic review work was presented, which suggested a real need for information on the different trial designs to be available in an easily accessible and user-friendly format.
Slow recruitment and poor retention are common challenges to the successful delivery of clinical trials. Patient and public involvement (PPI) in research has the potential to enhance recruitment and retention in clinical trials, but there have been few attempts to investigate this experimentally. The aim of this project is to develop a PPI intervention aimed at improving recruitment and/or retention in surgical trials.
The project will consist of 4 stages:
The use of statistical analysis plans (SAPs), drawn up in advance of the analysis phase of a trial, is an accepted means of reducing bias in reporting the results of randomised controlled trials (RCTs). However, while health economic analysis plans (HEAPs) to guide trialists in conducting economic evaluations alongside RCTs are becoming more widespread, they lag behind SAPs in terms of standardisation and acceptance, and there is a fundamental question over whether they add value to the trial process at all. In the collective experience of members of the Health Economic Resource Use and Costs Working Group, there is currently substantial variation in the structure, format and content of HEAPs, and no real agreement on either their purpose or appropriate methods of oversight. Clarity on the need for, and appropriate usage of, HEAPs would be advantageous.
We therefore propose to hold a workshop on HEAPs for about 50 attendees in Bristol, with three key aims. First, we plan to review the limited number of currently available guidelines addressing aspects of HEAPs. We also intend to collate information about the current usage of HEAPs in terms of their structure, content and purpose. Finally, we aim to provide a forum in which health economists and other interested parties engaged in applied economic evaluations can open a dialogue on appropriate methods of standardisation with a view to creating guidance in future work.On 20th October 2015, a workshop was held in Bristol to discuss issues with 'Health Economics Analysis Plans (HEAPs)'. This cross-Hub collaboration between Bristol, Bangor and Oxford universities was well-attended; 50 participants heard presentations from speakers covering a range of topics before breaking into smaller groups to discuss the appropriate content, oversight and potential changes to HEAPs.
Resource-use measurement in economic evaluations conducted alongside randomised controlled trials (RCTs) is commonly carried out by asking patients to provide information via a questionnaire or diary. However, there is no well validated instrument available that is flexible enough to work across different health conditions and care settings. Health economists tend to use unvalidated bespoke instruments for each trial, which results in unnecessary repetition of work, and hinders comparisons between trials.
To identify a core set of economically important resource-use items that are suitable for future inclusion in a modular patient-reported resource-use measure.
Instruments currently lodged in DIRUM (the Database of Instruments for Resource-Use Measurement, www.dirum.org), and additional instruments sourced from health economists, will be examined to determine the items of resource use that are commonly collected in RCTs. A comprehensive list of care items encountered will be compiled; items will then be categorised into ‘domains’ describing different types of healthcare (e.g. inpatient care, community care or medication). The item list will be systematically reduced to 10-20 key items per domain to form the basis for the Delphi survey.
A Delphi panel comprising patients, health economists and trialists from varying backgrounds will be engaged. In round 1 of the Delphi process, professional participants will be asked to rate the items according to their economic importance in a trial context, while patients will be asked to assign ratings based on the item’s relevance. Participants will also be asked to suggest additional items of healthcare resource use for consideration.
Items deemed insufficiently important according to predefined criteria encompassing both professional and patient responses will be dropped. A second Delphi round will be undertaken in which feedback from the first round will be presented to participants. A third Delphi round may be conducted if significant differences of opinion remain.
People doing clinical trials and other types of health research often struggle when trying to choose the outcomes to measure which would be of most use to the patients, practitioners and policy makers who will use their research to help them make decisions. These difficulties for trialists are passed on to those producing and reading systematic reviews of trials, many of whom have experienced the frustration of finding that the original researchers either did not measure certain outcomes or measured them in such different ways that it is difficult or impossible to compare, contrast or combine the studies. Much could be gained if there was an agreed minimum set of core outcomes for each medical condition, which were measured and reported in all clinical trials in that area. The COMET Initiative, launched in 2010, has brought together an international network of individuals and organisations interested in the development, application and promotion of such core outcome sets (COS).
A review of the literature has identified core outcome sets in nearly 200 medical conditions, and we are also aware of over 30 ongoing projects. COMET has put this information together in a publically available, searchable database. To date there has been no formal quality assessment of these studies and there is a pressing need to do so using internationally recognised criteria, both for COS developers and for trialists using COS.
This proposal requests funds to host a consensus meeting which is part of a larger study to develop the quality assessment instrument for studies developing COS. The outputs could impact immediately on the increasing number of ongoing and planned COS studies.
The HTMR Network has provided funding to support a workshop focussed on new Chief Investigators. The workshop is likely to be run again in 2016. Dates are to be confirmed. To add your name to our mailing list please email us.The workshop is targeted to recently funded Chief Investigators on RCTs.