×

Clinical effectiveness research: a critical need for health sector research governance capacity

The barriers to conduct of clinical research will require solutions if we are to implement evidence-based health care reform

Reforms in the funding of health services, such
as “activity-based” funding initiatives, seek to facilitate changes in how health care is delivered, leading to greater efficiency while maintaining effectiveness. However, often these changes in treatment strategies and service provision evolve without evidence demonstrating effectiveness in terms of patient outcomes. The pressures on health care expenditure (currently around 9% of gross domestic product1) make such an approach untenable and unsustainable. The evidence necessary to support these initiatives can only be derived through carefully conducted clinical research. Most readers would immediately think of clinical trials in terms of pharmaceuticals or clinical devices, and this type of research is critically important, although continuing to decline, in Australia.2 Other questions relate to the effectiveness of changes in health practice or policy, usually (but not always) based on sensible ideas that seem self-evident. However, in order to function with an evidence base, these ideas need to be proven to be clinically effective and cost-effective. Such research can be costly, and many of the questions to be addressed are not ones that would be the subject of an industry-sponsored trial. Researchers, clinicians and health administrators are therefore faced with the problem of how best to measure the outcomes of changes to health care strategies, without the necessary resources to ask and answer the question.

The MJA Clinical Trials Research Summit held in Sydney on 18 May 2012 included a working group addressing issues of research governance and ethics. The key discussion outcomes of that group were:

  • confusion exists regarding the differences between ethics and governance;

  • variability continues in state and federal legislation and regulations, despite attempts at harmonisation;

  • processes for improvement at government and institutional levels are underway but are not yet complete or implemented;

  • hospital boards and chief executive officers need to have incentives to make the infrastructure work;

  • substantial challenges exist when working with international investigator-initiated trials;

  • trials involving the private health sector include specific difficulties such as insurance and contracts; and

  • national accreditation of researchers and training should be considered.

Costs are not the only barrier. Efforts to rationalise health care provision on the basis of evidence provided through the conduct of clinical research are also hampered by existing or perceived obstacles in the form of cumbersome institutional research governance and ethics approval processes. Substantial changes and streamlining of the processes of ethical review are underway across Australia, addressing inconsistencies and inefficiencies of human research ethics committee approval, financial processes, and contractual clinical research governance processes. Nevertheless, the system remains complex, slow and expensive. Unfortunately, the old adage of “good, quick or cheap: pick two” still applies.

Many researchers fail to distinguish between research governance and ethics. Clinical research in Australia is governed by the National Health and Medical Research Council (NHMRC) National statement on ethical conduct in human research3 and the Australian code for the responsible conduct of research.4 Research governance can “be understood as comprising distinct elements ranging
from the consideration of budgets and insurance, to the management and conduct of scientific and ethics review”.5 Research governance thus includes oversight of all processes of ethics review, but also includes responsibilities of both investigators and institutions for the quality and safety of their research.3

The Harmonisation of Multi-centre Ethical Review (HoMER) initiative by the NHMRC is a significant step forward, enabling a single ethics review process that has been adapted for several states. This process, if used effectively, should reduce the resources required to obtain ethics approval for multicentre research, but it has also created some challenges in ensuring that research governance obligations are maintained within various health service jurisdictions.6 Currently, no incentives or requirements exist for health services or hospital chief executive officers to ensure that appropriate infrastructure is in place and working. Similarly, a different set of challenges arises when considering performing research in the private sector, where insurance and contractual issues may differ substantially from those in the public sector.

Much of the non-industry-sponsored clinical research performed in Australia is investigator-initiated research, supported by funding organisations such as the NHMRC, state governments, and other non-government organisations such as Cancer Council Australia, the National Heart Foundation of Australia and cooperative clinical trial groups. At present, investigator-initiated trials require comparable levels of research governance and are certainly subject to the same requirements for good clinical practice as industry-sponsored trials. The research questions addressed by these studies are based on clinical imperatives, a broad understanding of the underlying science, and a necessary ability to work on a shoestring — the latter being the main point of distinction from industry-sponsored trials. Current models of competitive research grant funding do not recognise the complexities, duration, costs and distribution of costs across the length of a clinical trial, especially when considering late clinical outcomes that are often the most clinically relevant ones. As an example, an NHMRC project grant can be funded for at most 5 years and therefore necessitates a focus on end points occurring within end-point time frames. The clinical questions that we and the community recognise as important might not be able to be answered with such designs. The resources required to meet these requirements continue to escalate and we currently run the risk that these trials will soon be untenable in Australia. Anecdotally, many academic clinical research units are already questioning what level of involvement they should have in such relatively underresourced trials or if they should be involved at all, for the most part purely for financial reasons.

Within the current Australian health care environment, clinical research is being conducted in the face of significant headwinds. These inefficiencies arise from resource costs due to complex governance arrangements combined with those of research conduct (Box). Processes to be considered that will improve clinical research capacity might include:

  • continued adoption of electronic health records that span clinical, investigative (ie, pathology and radiology) and therapeutic information (eg, the Australian Orthopaedic Association National Joint Replacement Registry);

  • data-linkage techniques to obtain clinical outcomes
    (eg, hospital readmission data, Medicare Benefits Schedule and Pharmaceutical Benefits Scheme use data, the National Death Index);

  • better integration of research into routine clinical practice;

  • national accreditation of investigators;

  • standardised good clinical practice training;

  • increased profile for research participation at the clinician–patient level, enabling the conduct of studies that are more representative of a wide spectrum of patients;

  • development of a clinically relevant strategic research agenda led by collaborations between clinicians, researchers and health policy decisionmakers;

  • a culture shift where lawyers and hospitals communicate and quantify the risk or research appropriately.

Research developed through partnerships between health policymakers and health service providers should lead to outcomes that are more immediately relevant and translatable to the care we provide, the outcomes we achieve and the costs incurred by the health system. Reinvestment of financial and efficiency gains realised from initial research outcomes back into the next relevant translational research question provides a model for a sustainable health system that evolves with the support of a robust clinical research-driven evidence base. These financial windfalls currently go back into government coffers and ideally should be seen as a potential funding stream to support future clinical research.

As the demands on our health system continue to mount, the need for clinical effectiveness research to build a robust evidence base upon which to reform care has become even more acute. It will be critical to align the clinical and policy research agenda while strengthening the governance structures that facilitate the conduct of research within the clinical space if we are to develop “an agile, responsive and self-improving health system for future generations”.7

Key points

Barriers to clinical research include:

  • regulatory complexity

  • inflexibility of ethical review and oversight

  • funding models that are not designed to support clinical trials

  • lack of incentive for engagement of health services in research support.

Solutions may include:

  • different funding models, including support for longer time frames

  • simplification of ethical and governance processes recognising the different goals of industry- versus investigator-initiated research

  • better involvement by health services in supporting research

  • return of savings from clinical research to support further research

  • clinical research key performance indicators for health service administrators.

A meta-analysis of “hospital in the home”

To the Editor: Caplan et al1 include in their meta-analysis a trial by Mather et al that compared home care with intensive care management of patients with acute myocardial infarction (AMI) between 1966 and 1968.2 A joint working party of the Royal College of Physicians and British Cardiac Society dismissed the results of this study because of design defects.3,4

Kalra et al5 performed a randomised trial with three arms for patients with acute stroke: stroke unit care, general ward care with stroke team support, and domiciliary care. Stroke units achieved a significantly lower mortality than general ward or domiciliary care. Caplan et al ignore the heterogeneity of the hospital arms, and sum their mortalities, creating a non-existent advantage for domiciliary care over hospital care. This meta-analytic technique is simplistic and invalid.

Hill et al describe home versus hospital management for patients with suspected AMI,2 as do Mather
et al.2 Studies of obsolete treatments, such as home management of patients with AMI, should have been excluded from the meta-analysis.

Rudd et al studied the effect of early discharge after stroke using a 1976 clinical definition of stroke.6 No details of imaging or comorbidities were given. The assumption of equipoise in the trial arms regarding morbidity is not met, and the study is not suitable for inclusion in the meta-analysis.

Indredavik et al7 studied the effect of early supported discharge versus ordinary care in patients with stroke, with 13 deaths at 26 weeks in the experimental group against 15 deaths in the control group. However, Caplan et al incorrectly report this as 21 and 26 deaths, respectively.

If these five most heavily weighted studies are excluded, no significant difference in mortality is seen
(243 hospital-in-the-home deaths [n = 2747] v 245 hospital deaths
[n = 2435], two-sided P = 0.14). Moreover, meta-analysis of the effect of location on mortality where the circumstances of the location are
not defined and not expected to be homogenous is invalid and makes
the mathematical exercise futile.

A meta-analysis of “hospital in the home”

In reply: Dickson argues for exclusion of randomised controlled trials (RCTs) if treatments have changed, but treatments are constantly changing so, following
this rule, meta-analysis would be impossible. Similarly, diagnosis
has changed — stroke was
a clinical diagnosis, then computed tomography was required, and
now magnetic resonance imaging
is needed. Equipoise is not a requirement for inclusion in a
meta-analysis.

Complaints about research being simplistic because it aggregates patients and groups demonstrates a misconception of research, which is designed to aggregate one factor while other factors differ — for example, study arms may have different mixtures of ages but similar average ages. The meta-analysis studied effects of two systems of care — hospital and hospital in the home (HITH) — not a particular diagnosis or treatment.1,2 Therefore it is legitimate to aggregate hospital patients and compare them with HITH patients.

Location and heterogeneity were mathematically defined, there was no heterogeneity for mortality data, and other outcomes were adjusted appropriately.

Results from the study by Indredavik et al were published in several reports, but (due to space limitations) only the primary report was cited. The data that Dickson refers to are in a report by Fjaertoft et al.3

Although the prevailing opinions
of the Royal College of Physicians
of London and the British Cardiac Society criticised the study by Mather et al in the 1970s, no contradictory facts or trials were cited at the time.4 Considering that other prevailing practices that were initially not examined by adequate RCTs led
to many iatrogenic deaths (eg, prophylactic use of antiarrhythmic drugs5), such practices should be examined and evidence of patient harm taken seriously, rather than simply dismissing evidence as obsolete.

The impact of age and sex on person-level Medicare costs

To the Editor: The impact of population ageing on health care expenditure was a hot topic in the late 1990s in Australia.1 Previous studies have adopted various modelling approaches to estimate age- and sex-specific health care costs to inform population health care expenditure projections.24 Medicare funds about 3800 medical services, including consultations provided by general practitioners and specialists, medical diagnostic services (such as medical imaging and pathology), dental surgery, optometry and selected allied health services through the Medicare Benefits Schedule (MBS). It also funds about 2100 pharmaceutical therapies, or about 80% of all prescription medications dispensed in Australia, through the Pharmaceutical Benefits Scheme (PBS).

This letter reports annual Medicare costs at the person level for 10-year age and sex groups, drawing on de-identified Medicare linked data. Medicare randomly selected 10 000 men and 10 000 women from each of the following 2009 age groups, irrespective of their associated Medicare expenditure: 15–24, 25–34, 35–44, 45–54, 55–64, 65–74 and 75–84 years. Health care cost data for each of the samples were sourced from the MBS and PBS databases for the period 1 July 2005 to 31 December 2008. Costs were annualised and mean costs per person were calculated for each sample in 2007 Australian dollars.5

Medicare costs increased with each 10-year age increment, with the exception of a decrease in costs observed in the 75–84-year age group, which is likely to have resulted from the death of patients who previously had heavy health care use. Medicare costs per person peaked for both sexes in the population aged 65–74 years (women, $4518; men, $4215), where costs were 3.7 times and 5.6 times higher than equivalent costs for women and men aged 15–24 years. Medicare costs per capita were consistently higher for women. The differential was most pronounced for the younger age groups, particularly the 25–34-year and 35–44-year age groups where costs for women were more than double the equivalent costs for men (Box).

Observed health care cost data for demographic groups can be used to inform future Australian health care expenditure projections.

Mean annual Medicare costs per person by age and sex*

* Costs in 2007 Australian dollars.

Secondary prevention of coronary heart disease in Australia: a blueprint for reform

Authorship statement omitted: In “Secondary prevention of coronary heart disease in Australia: a blueprint for reform” in the 4 February 2013 issue of the Journal (Med J Aust 2013; 198: 70-71), a full statement of authorship was omitted. Julie Redfern and Clara Chow coauthored the editorial on behalf of the Executive Committee and all participants in the National Secondary Prevention of Coronary Disease Summit held in December 2011.