Comparative effectiveness research — the missing link in evidence-informed clinical medicine and health care policy making

To change practice, we should move beyond trial-based efficacy to real-world effectiveness

Meaningful health care reform requires robust evidence about which interventions work best for whom and under what circumstances. The Institute of Medicine in the United States has estimated that less than 50% of current treatments are supported by evidence and 30% of health care expenditure reflects care that is of uncertain value.1 In studies testing established clinical standards of care, more than half reported evidence that contradicts standard care or is inconclusive.2 Many Medicare Benefits Schedule services lack comprehensive evidence of comparative safety or effectiveness, while many that have been evaluated have been shown to be ineffective, harmful or of uncertain value compared with alternative forms of care.3

Filling the void — the rise of comparative effectiveness research

Comparative effectiveness research (CER) compares new
or existing interventions (or a new dose or intensity of an intervention) to one or more non-placebo alternatives, which may include “usual care”. It can be used to evaluate
a broad spectrum of clinical interventions, including diagnostic tests or strategies, screening programs, surgical procedures, pharmaceuticals, prostheses and medical devices, quality and safety improvement interventions, behavioural change and prevention strategies, and care delivery systems. While CER is not a new process — many past trials have compared different interventions — it represents a new focus and consolidation of approaches
in clinical and health services research.

In 2009, the US Congress authorised $1.1 billion for CER and, in 2010, the Patient-Centered Outcomes Research Institute was established to identify CER priorities and develop appropriate methodologies. In the United Kingdom, the National Institute for Health Research (which was established in 2006) commissions and disseminates CER that informs clinical decision making, overseen by
the National Institute for Health and Clinical Excellence. However, in Australia, there is no comparable group or agency with CER as the prime focus of activity.

For CER to realise its full potential, the research community must accommodate four prerequisites in
the following order.

1. Involvement of all relevant stakeholders in setting the research agenda

Research has often lacked meaningful engagement of health care providers and patients in the choice of research questions and design and implementation of the research effort. Researchers and consumers of research must collaboratively identify important unanswered questions among current systematic reviews and clinical guidelines.4 Questions should be selected for CER on the basis of: the perceived needs of key stakeholders (clinicians, patients and health care managers); factors related to potential impact (eg, disease burden, cost of care and variation in outcomes); paucity of effectiveness data among specific populations; and emerging concerns about undisclosed harm.4 In the US, the Agency for Healthcare Research
and Quality has developed iterative and transparent methods for defining and prioritising future research
needs that involve a wide spectrum of stakeholders
(http://www.effectivehealthcare.ahrq.gov). Quantitative modelling methods which calculate the potential value of information in filling existing gaps in knowledge can also assist in prioritisation.5 The Institute of Medicine has issued an initial list of 100 national CER priorities derived by consensus, which includes patient-level and health system-level interventions.6

2. Flexible approach to evidentiary standards

To be useful, CER must use the best possible data sources and methods to provide credible, timely and relevant evidence. The analytic scope of CER includes reanalysing existing data from available studies (in the form of systematic reviews, meta-analyses or decision analyses)
or, if these fail to provide answers, generating additional data from new studies.

The aim of CER is to determine intervention benefit among unselected patients in real-world practice settings (ie, measure effectiveness), as opposed to doing so among highly selected patients in tightly controlled experiments (ie, measure efficacy). The design and conduct of CER studies must reflect this aim (Box).7

CER encounters the vexed question regarding the relative clinical utility of observational studies versus experimental trials. Randomised controlled trials (RCTs) have high internal validity, but narrow patient selection criteria limit their generalisability. Observational studies use data on care delivered routinely to unselected populations in various settings, but their results are
more vulnerable to confounding and bias owing to the absence of randomisation. The way forward for CER is
to encourage more large-scale, real-world RCTs (pragmatic trials) and more rigorous observational
studies (see Appendix).

In RCTs, the inclusion of as-treated and per-protocol analyses (in addition to intention-to-treat analyses) can help expose patient-specific differences in intervention uptake and response. More head-to-head RCTs that fairly compare appropriately administered alternative interventions are needed. Network (or mixed-treatment) meta-analysis enables direct and indirect comparisons of different treatments to be combined into one synthesis, which enables greater use of all available RCT evidence than traditional meta-analysis.

In observational studies, several design features improve rigour: prospective and standardised data collection, blinded outcome assessors, prespecified matching or stratification of patient groups, and analytic techniques that minimise confounding, such as risk-adjusted regression modelling and interrupted time series analysis. Multiple high-quality studies related to a single question which consistently show large intervention effects that persist after discounting all important sources of bias confer a high level of credibility. High-quality observational studies may fill evidence gaps more proficiently than RCTs in situations where:

technologies are rapidly evolving (ie, there are moving targets)
technologies cannot easily be randomised
no head-to-head trials exist
RCTs exclude certain types of patients or conditions
information is being sought about modification of treatment effects due to

variation in patient adherence and tolerance
use of concomitant treatments
dosing or intensity of treatments
selection or switching of treatments according to provider and patient preferences.

The choice of study design will depend on the context in which CER results will be applied, the necessary level of rigour according to the consequences of incorrect inferences from the sample studied (eg, potential harm or waste from introducing mass screening program versus small-scale niche therapy on the basis of invalid analyses), the feasibility and costs of different study designs, and the urgency of the need for evidence. The overarching goal is to describe methods that, if consistently applied, provide decisionmakers with a reasonable level of confidence that one intervention is more effective than or equally effective as another.

3. Investment in and redesign of research infrastructure

To fill evidence gaps quickly and definitively, CER will require substantially increased investment in current research infrastructure, both human and technical. This includes expanding existing and adding new research teams, developing new research methods and establishing collaborative research partnerships across multiple sites.
It also requires data linkage at the patient level (involving administrative databases and clinical registries for public and private patients), which enables more patients to be studied and facilitates better quality and diversity of studies. Such data linkage will require establishment of data standards and common vocabularies, unique patient identifiers, data quality control and privacy protection systems, and informatics grids which connect practice-based research networks.

Of even greater importance is tackling the current bottlenecks in clinical research: ethics approvals,
contract negotiations and incentives for organisations
to participate. Truly harmonised and rapidly responsive
ethics approval procedures across multiple jurisdictions, standardised contract language, and logistical and financial support for institutions to collect and share data are required.

In attracting researchers, CER also requires a dedicated funding stream for investigator-led CER, as is the case
with basic biomedical science research and RCTs that predominantly or solely measure efficacy.8 In 2011, the National Health and Medical Research Council spent less than 5% of its $800 million budget on CER, compared with 47% on biomedical research.9

4. Implementation of CER in changing clinical practice

Generating or revising clinical guidelines using CER results will, by itself, have minimal impact in changing clinical practice quickly. Implementation drivers that have greater impact include: redesign of care processes, professional roles and systems of care; financial incentives that reward better practice; performance reporting and feedback; health information technology and clinical decision support; mandates for shared decision making with patients; and better training of clinicians in CER and its application.10 The interdisciplinary field of implementation science, used to study successful diffusion of innovation, will become an important tool,11 aided by CER trials designed to simultaneously evaluate intervention effectiveness and optimal methods of implementation.12

The biggest implementation challenge is reconciling clinicians to important shifts in who delivers what care, and to whom, under different circumstances. The Medicare Benefits Schedule and Pharmaceutical Benefits Scheme will need to move towards greater investment in efficiently priced interventions that CER shows to be effective and disinvestment in interventions that are not. These reforms will require strong political endorsement, independent researchers, early and ongoing engagement with stakeholders around reimbursement decisions, and demonstrable commitment to evidence-informed best practice. However, CER should not be perceived as a means to substantially reduce overall health care spending. In the US, estimated cost savings from CER are less than 0.1% of total expenditure.13 Instead, the aim is to facilitate better return on investment. In fact, CER may lead to recommendations to adopt new interventions.

Current status of CER and its impact on
clinical practice

In a recent review of 231 CER studies (37% on drugs, 29% on behavioural interventions and 16% on procedures), only 35% favoured the new intervention; in contrast, 79% of 804 non-CER studies favoured the new intervention.14 More than 70% of the CER studies relied on non-commercial funding, but less than a quarter evaluated safety and cost.14

CER is informing health care policy and changing clinical practice in Australia and overseas. Australian researchers have reported sentinel CER trials comparing saline infusions with albumin infusions in intensive care15 and early dialysis with late dialysis in end-stage renal failure.16 In Norway, an entire colorectal cancer screening program has been set up as a series of adaptive randomised trials testing different screening tests and procedures.17

Conclusion

CER has the potential to reform health care and transform health care research. The research community needs to accommodate a greater emphasis on CER and address challenges regarding optimal methods for selecting stakeholders, prioritising research questions, selecting study designs that best answer the clinical question posed, determining funding and governance arrangements, and implementing CER findings into practice and policy making.

Elements of clinical and health services research that distinguish efficacy and effectiveness studies*

Study elements

Efficacy studies†

Effectiveness studies†

Intervention

Protocol strictly enforced; treatments masked; cross-overs discouraged

Highly flexible, as used in routine health care; treatments not masked; crossovers permitted

Patient population

High disease risk, highly compliant, few comorbidities

Anyone with the condition of interest

Study sites

Academic settings with well resourced research specialists

Routine clinical practice settings

Outcome measures

Often short-term surrogates or composite outcomes

Outcomes that are clinically relevant to patients, clinicians and health care managers

Duration of study

Often short (eg, several months to a year)

Often long

Intensity of monitoring

Intense

Intensity depends on condition of interest and practice setting

Data sources

Specific to trial

Various, including administrative databases

Data collection

Ceases when study is discontinued

Continues as part of routine care

Analysis

Typically intention to treat

Various, depends on study aims and design

* Adapted from Gartlehner et al.7 † When the study design is a randomised controlled trial (RCT), efficacy studies are termed explanatory RCTs and effectiveness studies are termed pragmatic RCTs.

Dr Burnt’s consultation

A day in the life of an oncology registrar

It’s Monday morning and I arrive at the oncology outpatients department to do a clinic with my consultant. It’s overbooked — yet again. The reminder alarm on my smartphone starts beeping. The alert flashes that I have to submit my oncology advanced training paperwork today and try to get hold of my supervisor to sign it off, and that I’m doing a journal club presentation tomorrow.

So, things are already sounding busy for the next 24 hours at least. I take a deep breath and get ready to start my clinic.

Before the consultation

The outpatients’ waiting room is already full and the reception staff seem to be panicking already. I pick up the case notes of a 29-year-old man who is attending the clinic to be told the results of his restaging scan for melanoma and to discuss treatment options. I start thinking that there have been a lot of developments recently in the field of melanoma.

Meanwhile, my consultant, who knows this patient very well, walks into the clinic central meeting room. He asks me, “Dr Burnt, what do you know about the mechanism of action and efficacy of targeted therapy and immunotherapy (with ipilimumab) in advanced melanoma?” I say I have heard they are good, but to what extent, I don’t know exactly. Obviously, had I read the pile of recent issues of the New England Journal of Medicine lying on my shelf still covered in the plastic wrapping, I might have been able to answer his question.

My consultant tells me that I need to stay up to date with the latest developments in the field of oncology and to read the literature regularly.

He then asks me to compare the latest scans for this patient with previous scans for any disease progression. After about 5 minutes, I still have not located the retroperitoneal nodes and external iliac nodes mentioned in the report in either scan, let alone compare them for progression.

I am further told to build up my radiology skills to be able to accurately compare the scans and also for reporting on clinical trials.

My consultant then shows me a few slides of ipilimumab Phase III trial results in advanced melanoma, commenting that the response rate seems to be low (10%–15%) and, looking at the graphs, that there is no difference in progression-free survival — however, there is a survival benefit. His next questions are: “What is the significance of the delayed separation of Kaplan–Meier survival curves in this trial? And what about the prognostic markers on multivariate analyses?” My replies, as before, are not convincing.

I am told to work on my statistics, as this skill is needed to better make sense of the data.

Finally, after a quick 5-minute advanced melanoma-treatment update session from my consultant, I go and see the patient.

The consultation

The patient is accompanied by his wife and mother. I tell him: “Unfortunately, you have widespread metastatic disease and we have identified a brain metastasis as well.” This spreads a huge wave of disappointment and grief throughout the room. Both wife and mother start crying.

The consultation continues …

Patient: I have heard on television about good results with a magic tablet (targeted therapy) in advanced melanoma.

Dr Burnt: You do not have a specific mutation, so that treatment will not work for you.

Patient: What about a clinical trial option?

Dr Burnt: Unfortunately, because brain metastasis is present, this would exclude you from any current trial.

Mother: Will chemotherapy work?

Dr Burnt: I’m sorry, but this is unlikely, given poor response rates to this treatment.

There is a long, silent, unhappy pause in the room …

Wife: How long does my husband have? Will he be there at the birth of our second child in 5 months’ time?

Dr Burnt (in shock, unaware that the couple were expecting a second child): It’s very difficult to say, but probably less than 6 months and it may even be less than that.

Patient: Is there anything, doctor, that you can do for me? What about the immune-stimulating drug I read about on the internet? I don’t want to die! I have a 14-month-old son and we’re expecting our second child … I’m only 29, I have many years yet to live … what you are saying can’t be true … who will look after my family and young kids … I have to keep working to pay my mortgage.

Dr Burnt: Unfortunately, we can’t access this drug in Australia as it’s very expensive. What I have told you about your prognosis is also, unfortunately, true.

The family leaves the room in tears.

After the consultation

I move on to the next patient. After seeing 10 patients, I realise it is now 3 pm. As usual, I’ve not had my lunch break yet. But at the back of my mind, I can still feel the sorrow and pain of the young couple that I saw earlier today. There was nothing very promising that I could offer them.

I felt sad and depressed. I kept thinking about the consultation for the rest of that day. I also felt the need and pressure to enhance my professional skills.

What does it all mean?

This is just one example of the many unfortunate scenarios that most of my colleagues, junior and senior, looking after patients with cancer, frequently have to face. Difficult consultations like the one above, administrative pressures, social dilemmas, working long hours and the constant demand to perform at high efficiency are a recipe for what many call “burnout”. This is especially relevant within the oncology subspecialty, where we are frequently dealing with terminally ill patients and their families.

Burnout is defined as

a syndrome … present in many individuals under constant pressure. Physicians in particular are overloaded with the demands of caring for sick patients … The symptoms and signs of burnout include emotional exhaustion, cynicism, and perceived clinical ineffectiveness, and a sense of depersonalization in relationships with co-workers, patients, or both.1

These feelings can manifest as irritability, increased anger and detachment from the social environment, very much in contrast to the individual’s normal behaviour.1 However, it is usually not the case that individuals can self-diagnose the early signs of burnout.

Burnout in oncologists has been reported in the literature. A nationwide survey of 740 (56%) members of the Clinical Oncology Society of Australia was conducted to determine the prevalence and predictors of burnout in the Australian oncology workforce.2 High levels of emotional exhaustion were present in 33% of participants. In the United States, a survey of 598 members of the American Society of Clinical Oncology found that 56% reported burnout, with feelings of sense of failure (56%), depression (34%), disinterest in practice (20%) and boredom (18%).3

Burnout has also been linked to low patient satisfaction, errors in decision making, unprofessional behaviour and deterioration of clinical skills. It may also affect the personal life of physicians in the form of substance abuse, depression and divorce.

I believe there is an urgent need to recognise and formulate interventions for at-risk medical staff in chronic stress and relational trauma settings. Early recognition and rectification of burnout in oncologists is needed to ensure the wellbeing, not only of doctors but also of our patients, who deserve optimal medical and psychological support provided by caring and competent medical practitioners.

Risks of complaints and adverse disciplinary findings against international medical graduates in Victoria and Western Australia

In reply: Nguyen and Nguyen question our finding that international medical graduates (IMGs) are at higher risk of attracting complaints than Australian-trained doctors (ATDs),1 based on recalculations using the descriptive statistics we reported. They infer, from the fact that IMGs accounted for 37% of doctors and 30% of total complaints, that IMGs were at lower risk of complaints.

Comparisons of simple proportions as above mislead for a couple of reasons. First, doctors were observed for different periods of time — accounted for by conducting the analysis at the doctor-year level. Nguyen and Nguyen’s reanalysis rests on the incorrect assumption that duration of follow-up was comparable between the two groups. On average, observation periods for IMGs were significantly shorter than those for Australian-trained doctors. Secondly, comparisons of simple percentages do not account for confounders — characteristics that differ systematically between IMGs and Australian-trained doctors and are also associated with the probability of complaints. We adjusted for several of these characteristics with multivariate analysis. Nguyen and Nguyen’s calculations do not take them into account.

Nguyen and Nguyen’s other criticism is that we should have modelled the incidence of complaints as a count variable in Poisson regression, rather than as a binary variable in logistic regression. A count model is not clearly the superior method for this analysis, but it is a reasonable alternative. That is why we used one. We reran our analyses using negative binomial regression, a close cousin of Poisson regression. This sensitivity analysis produced very similar results, as reported in our original article and Appendix. Nguyen and Nguyen’s Poisson calculations may have produced different results for many reasons, including use of simple proportions rather than individual-level data, their erroneous assumption about equal follow-up time, and no adjustment for confounders.

Chaturvedi argues that there are practical and moral imperatives to better support IMGs in acclimatising to work and life in Australia. We agree. A recent parliamentary inquiry2 into overseas-trained doctors recognised this need, as Zubaran notes, and has called for immediate action.

Akre and Achhra object to our use of the term “high risk”. We used this term in a specific way — to denote groups of doctors with significantly higher odds of complaints and adverse findings — and it is accurate. Additionally, Akre and Achhra speculate that the higher risk of adverse findings observed for IMGs as a whole may have been driven by IMGs from English-speaking countries with relatively “severe” complaints against them. We did not report IMGs’ risks of adverse findings by country of training because the data were sparse (373 adverse findings across 21 specified countries), confidence intervals were wide, and the analysis was underpowered. However, the data do not support Akre and Achhra’s theory. We checked by running a multivariable model (with the same specifications as the complaints model) to predict adverse findings among doctors from the three English-speaking countries and three non-English-speaking countries (excluding Germany) with the most IMGs. Odds ratios (ORs) for doctors trained in the United Kingdom/Ireland (OR, 1.37; 95% CI, 0.66–2.85), New Zealand (OR, 0.69; 95% CI, 0.24–1.94) and South Africa (OR, 0.92; 95% CI, 0.33–2.59) were similar to those produced by the original risk-of-complaints model, as were ORs for IMGs from India (OR, 1.94; 95% CI, 1.17–3.23), Sri Lanka (OR, 1.93; 95% CI, 0.77–4.82) and Egypt (OR, 1.93; 95% CI, 0.87–4.25).

Breen notes that complaints to medical boards are uncommon, often do not result in adverse findings, and may stem from complainant-related biases. Our article addressed each of those points. Breen also suggests that reporting complaint risks by country of training was unnecessary and ethically questionable. This strong claim is difficult to square with the standard public health approach. Research and prevention activities are routinely targeted at certain groups of people. Consider, for example, efforts to combat youth suicide, falls among the elderly, obesity in low-income households, HIV infection among injecting drug-users and renal disease in Indigenous populations. Such targeting is widely accepted and occurs because foundational research has previously determined that these subgroups faced disproportionately high risks.

Why should complaints to medical boards be approached differently? Breen suggests that uncertainty about causal factors, particularly biases in complaint-filing behaviour and medical board adjudications, demands it. These and other factors (eg, lack of support, cultural misunderstandings) warrant exploration, and may help explain IMGs’ heightened risks. However, disentangling causal factors will take years and will require many studies. Publication of risk differences should not wait for this. Where would the impetus for such further research come from without initial identification of risk differences? Being less than candid about our findings would have done a disservice to IMGs and their patients.

Risks of complaints and adverse disciplinary findings against international medical graduates in Victoria and Western Australia

To the Editor: The article by Elkin and colleagues1 has certainly reheated the long-simmering debate about bringing highly trained international medical practitioners from other countries to provide health care in Australia. Yet, it does not discuss the clinical and cultural acclimatisation they need. There are many moral and social issues associated with condemning an IMG without helping them to adapt to their locality. The study indicates that an effective orientation process of integration for individual doctors from overseas is needed.

Australia actively recruits doctors from overseas on limited registration to provide adequate services in rural and remote areas. These doctors have limited access to Medicare, to free public education for their family members and to other services that Australian residents enjoy. This is not conducive to good medical practice.

IMGs are required to pass a series of examinations that test their language and technical competency, but this does not guarantee employment. Supervised positions, vital for clinical competency, are scarce for these doctors. Most high-level governance and administrative positions do not include IMGs who could represent other IMGs in deciding policy matters. Furthermore, complaints against IMGs appear to be more likely to attract publicity than similar cases involving local graduates, which affects their reputation.

Lastly, the countries from which IMGs have come, many of them less privileged than Australia, have significantly invested in their training. In proper acknowledgement of this, they should be able to work here as doctors. The path of professional development of IMGs should be monitored by the government to ensure this is possible. The Australian Indian Medical Graduates Association strongly objects to the current trend for IMGs in Australia to remain wastefully unemployed or underemployed for prolonged periods, and considers this trend to be unethical and inequitable.

Emerging inequality and potential unconstitutionality — the case for reform of the intern priority system

Internship, the gateway from the primary medical degree to further medical training in Australia, is becoming increasingly competitive. We question the fairness and legality of the “priority system”, in which states give preference to Australian students graduating from medical schools in their state over all other applicants, including interstate Australian graduates, who form part of lower priority groups (Box 1 and Box 2). The implications of this system for international graduates have been considered7 and, irrespective of ethical and policy reasons for offering international students internships, we agree with the conclusion that the legal position of international students is weak. The impact of the priority system on interstate applicants has not been addressed.

Previously, interstate movement of medical graduates at internship level was feasible due to a national shortage of medical graduates. However, the increasing scarcity of positions has made such movement much more challenging, and has caused friction between some jurisdictions. This was illustrated by the recent removal of Australian Capital Territory hospitals from the New South Wales priority system, with ACT graduates now considered to be interstate applicants in NSW.8 The priority system, while administratively convenient, is inequitable, potentially unconstitutional, and inconsistent with medical school selection policies and recent registration changes facilitating free movement of medical professionals nationwide.

The priority system generally means that even the most meritorious interstate candidates cannot be selected for internship positions until the least meritorious “local” candidate is selected. This system has a negative impact on a proportion of the 27% of Australian medical students who complete their medical degree outside their home state and wish to return due to strong personal or professional links (Mary Solomon, Executive Officer, Medical Deans Australia New Zealand, personal communication, 13 March 2012). Such profound “handicapping” does not occur at postgraduate year (PGY) 2 or 3 levels. However, graduate movement at these levels will become more difficult. Internal candidates have the advantage of known referees and an established relationship, and in some states, such as NSW, interns generally fill PGY2 positions due to extended contracts. Given the forecast bottleneck of a 400-place shortfall in first-year training positions within 4 years,9–11 securing an internship in the state in which individuals wish to train long-term will become more critical.

As recent growth in graduate numbers has not been proportional to population size,12 applicants in more competitive states may receive job offers from lower-preferenced hospitals than applicants of equivalent merit in other states. It would be advantageous for a candidate keen on competitive or rarer specialties, such as cardiothoracic surgery or pathology, to gain employment at a less competitive interstate hospital offering such rotations. There is a policy advantage in facilitating exposure of interested graduates to rotations as soon as possible, particularly in specialties which have a workforce shortage.13 References from rotation supervisors are important in the selection process for many training programs,14,15 but these possibilities are greatly restricted within the current priority system. Furthermore, some states do not place domestic full-fee students in their highest (sub)category and guarantee them an internship (Box 3); these students are then further disadvantaged if they seek an internship interstate.

Noting that medical schools and colleges do not discriminate on the basis of state of residence in their selection of students and trainees, the best principle to use is that all Australian graduates should be able to apply to any hospital that best meets their career aspirations, and no citizen should be advantaged over another simply by virtue of residential location. The priority system may infringe a clause of the Australian constitution which enshrines this principle and prohibits such discrimination.

Is the priority system unconstitutional?

Section 117 of the Commonwealth of Australia Constitution Act 1900 (Cwlth) seeks to strengthen the nation’s federal structure through prevention of state protectionism:

A subject of the Queen, resident in any state, shall not be subject in any other state to any disability or discrimination which would not be equally applicable to him if he were a subject of the Queen resident in such other state.

This section could operate to prevent, for example, an applicant for internship in Queensland who was trained in Tasmania from being treated differently to a candidate trained and residing in Queensland.

The Street case

Section 117 was considered by the High Court of Australia in the key case of Street v Queensland Bar Association, concerning the admission rules to become a barrister in Queensland.16 The appellant, a barrister registered in NSW, had applied for admission to practise in Queensland, and was refused on the basis of his lack of intention to practise principally in Queensland (shown by his continued domicile and practice in NSW). The Court unanimously found that the Bar Association of Queensland rules contravened s. 117; though the justification for this decision varied among the judges.

Each judge confirmed that the s. 117 test of discrimination required an analysis of how the affected individual was treated compared with a hypothetical person sharing all of their qualities except for interstate residence. Justice Deane noted that

neither [of the] requirements [for admission] can properly be seen as flowing naturally from, or being a natural incident of, the privilege of practising as a barrister in Queensland in the sense of being a necessary professional qualification or safeguard.17

and, therefore, interstate applicants were subject to discrimination under the section. Pertinently, Justice Brennan concluded that

[this] law … opens too the doors of state universities, hospitals and other institutions for entry by subjects of the Queen resident in other states on the same terms as residents of the relevant state.18

As s. 117 was not intended to remove all state autonomy, there will be some circumstances in which the discrimination is justified if it is reasonably appropriate and adapted to the attainment of some legitimate objective.19 However, arguments that interstate lawyers would lack familiarity with local rules, or would be unable to perform their professional duties as satisfactorily as resident lawyers, have been consistently rejected due to lack of evidence.20

Legal analysis

Those defending the priority system might argue that the system discriminates on the basis of the location of the student’s medical school rather than state of residence and that s. 117 does not apply; or, if s. 117 does apply, that a sufficient justification for the discrimination exists.

We doubt that the High Court would find the first argument compelling, as it would be almost impossible to undertake mandatory clinical rotations without living in the state in which medical training is being undertaken. Today’s High Court would likely look at the substance of the underlying discrimination rather than the form of wording. In Street, Chief Justice Mason confirmed that an analysis of the effect of the relevant law is necessary to avoid depriving s. 117 of practical effect, observing that

a disability or discrimination based upon grounds apart from residence is effectively removed if those grounds relate to characteristics which are … concomitants of the individual’s notionally changed residence.21

Our view is that the modern test reveals what the discrimination is truly based on: the state of the applicant’s residence, which is a necessary concomitant of location of their medical school.

Supporting this conclusion, in a subsequent case concerning size of a state insurance payout to an individual, the High Court held unanimously that a clause designed to limit damages for an interstate resident but not for an “in-state” resident was invalidated by s. 117.22 Chief Justice Mason also noted in Street that “s. 117 is not susceptible of ‘colourable evasion’ by state legislatures” who may seek to hide discriminatory practices under creative legal drafting.23

In Sweedman v TAC, although the Gleeson High Court did not consider it an infringement to discriminate on the basis of the state a car was garaged in for the purposes of determining compensation recovery of the Victorian Transport Accident Commission, this interpretation primarily turned on payment of a transport accident charge rather than on residential location itself.24 There was no differential treatment on the basis of residence that would infringe s. 117, as a vehicle could be registered (and the relevant charge paid) in a state irrespective of whether its owner resides there. In contrast, the residence of a person while in medical school is intrinsically related to their enrolment at a medical school; one cannot simply submit an interstate address for the purpose of internship applications to “cure” the discrimination arising from the priority system.

The modern High Court’s unsympathetic attitude to state protectionism was demonstrated in a case invoking s. 92 of the Constitution involving free movement of goods and commerce between states, in which the High Court found unconstitutional Western Australian attempts to prevent a Tasmanian-based internet gambling exchange from operating in Western Australia.25 The unanimous decision de-emphasised geographical boundaries, with Justice Heydon noting that part of the legislation was designed to create “a smokescreen”.26 The case is particularly relevant given Chief Justice Mason’s observation in Street that s. 117 was intended to operate as “a counterpart” to other constitutional provisions, including s. 92.27

The argument regarding sufficient justification for the discrimination is more uncertain. We wrote to the federal and state health ministers, drawing attention to s. 117 and explaining the conflict, inviting them to provide a rationale or justification for the priority system. Most ministers provided general information regarding the priority system, or its historical genesis from the 2006 Council of Australian Governments meeting in which health ministers undertook to guarantee sufficient internship places for Australian citizens in Commonwealth-supported places at Australian medical schools.28 These responses did not provide specific information which could be readily evaluated against the types of justifications the High Court has previously considered. Two ministers asserted their belief in the legality of the scheme. One minister stated that as an internship is considered the final year of training of a medical degree, the [state] government believes that prioritising places for [state] graduates in their final year of learning in [state] provides a continuum of learning for those who have studied in [state].

If this rationale were argued before the High Court, we doubt it would be accepted. Students in Australia graduate from medical school before they become interns. Interns are paid a salary and given the title “doctor”; students are not. International medical graduates who complete their internship in Australia cannot continue on a student visa, and must obtain a temporary business visa. Moreover, even if internship were the final year of training for a medical degree, it seems an insufficient justification for discrimination that ranks interstate candidates unfavourably. There may be some benefit in continuity in working in the same state of medical schooling, but graduates moving interstate for internship would undertake the same core rotations in surgery, medicine and emergency medicine, and access the same nationally endorsed prevocational curriculum.29 Medical schools nationwide are all currently accredited by the Australian Medical Council to train medical students to the standard required to begin internship, so no state could credibly claim that its graduates were more appropriate than or superior to interstate graduates.

It could be argued that the priority system allows a state to better forecast and budget for intern positions, and plan for the health needs of their population. While an empirical analysis of this argument is not possible without more information, it is difficult to see how it would be compelling. Provisions such as ss. 117 and 92 were drafted to promote Australia’s new federal structure by allowing the free movement of people and goods nationwide. As medical schools are not established with the intention of producing doctors purely for the benefit of the state they are in, and graduate numbers are disproportionate to internship places in certain states, the argument is bolstered that state-based discrimination in internship selection undermines national unity and infringes s. 117. If the priority system were removed, states would be free to employ the same number of interns as they do now, minimising budgetary impacts but allowing interns to move more freely, promoting national unity.

States might be able to show that such discrimination is proportionate and adapted to some other legitimate interest, but based on the replies we received from the health ministers and the blanket nature of discrimination in the priority system, we doubt that any justifications would withstand the High Court’s scrutiny.

Conclusions

The departure from the principle of equality of citizens in competing for jobs promulgates more than mere academic concern. Erosion of rights may start with seemingly minor irregularities, tolerated on pragmatic grounds, before further changes increase the differences in treatment between citizens.

The recent changes to the priority system in Victoria are a predictable next step in this process, in which temporary residents (international students) graduating from Victorian medical campuses are now the second priority category.2 The outcome for Australian citizens is that any interstate applicant (including a Victorian secondary or undergraduate student who moved interstate to study medicine) is now in a lower priority category than every Victorian temporary resident, and has a greatly diminished chance of gaining an internship position in Victoria. As many universities wish to be attractive to international students, other states may be tempted to mimic Victoria’s changes in order to remain competitive by offering a bolstered chance of internship to such students.

We argue that distinctions between intrastate and interstate applicants should be abolished; these applicants should be all considered of the highest and equal priority. Movement towards better coordination of intern recruitment, via the National Intern Allocation Working Party and a recently proposed online national intern allocation portal,30 could be used to promote wider reforms of the system. If this did not occur, litigation could be considered by individuals or groups, as a mechanism of achieving change. Any selection rules found to be unconstitutional would be void and therefore have no lawful effect. Some previous High Court cases upholding constitutional rights have led to the swift redirection of government resources.31

1 Intern priority system as applied by Queensland Health for 2013 intake1

Priority	Explanation

1	Medical graduates of Queensland universities who are Australian citizens, Australian permanent residents or New Zealand citizens
2	Medical graduates of interstate or New Zealand universities who: completed year 12 in Queensland, and are Australian or New Zealand citizens, or are Australian permanent residents
3	Medical graduates of interstate or New Zealand universities who: completed year 12 outside Queensland and are Australian or New Zealand citizens, or are Australian permanent residents
4	Medical graduates of Queensland universities who are overseas permanent residents or overseas citizens, other than New Zealand citizens, and are permitted to remain in Australia for internship
5	Medical graduates of interstate or New Zealand universities who are overseas permanent residents or overseas citizens, other than New Zealand citizens, and are permitted to remain in Australia for internship
6	Medical graduates of international universities who have successfully completed the Australian Medical Council (AMC) written and clinical examinations and have obtained the AMC certificate who: are Australian or New Zealand citizens, or are Australian permanent residents, and require a year of supervised training to meet general registration requirements of the Medical Board of Australia (MBA)
7	Medical graduates of international universities who have successfully completed the AMC written and clinical examinations and have obtained the AMC certificate who: are overseas permanent residents or overseas citizens (other than New Zealand citizens), and require a year of supervised training to meet general registration requirements of the MBA

2 Variation in definitions of intern priority 1 groups

State

Priority 1 definition

Victoria

Group 1: Australian permanent resident graduates of Victorian universities (including domestic full-fee graduates and New Zealand citizens)2

Queensland

Priority category 1: Medical graduates of Queensland universities who are Australian citizens, Australian permanent residents or New Zealand citizens1

New South Wales*

Category 1.1: Commonwealth-supported place (CSP) and domestic full-fee graduates of a NSW university

Category 1.2: CSP and domestic full-fee graduates of an interstate or New Zealand university who completed year 12 in NSW

Category 1.3: CSP and domestic full-fee graduates of interstate or New Zealand universities

Tasmania

Australian citizens or permanent resident [CSP] graduates of the University of Tasmania3

Western Australia

Priority category 1: all graduates of WA medical schools who are Australian citizens or permanent residents or New Zealand citizens4

South Australia†

1. CSP and SA Bonded Medical Scholarship Scheme graduates from an SA university who are Australian citizens or permanent residents or New Zealand citizens

1.1. CSP or domestic full-fee graduates of an Australian or New Zealand university outside SA who completed year 12 in SA and who are Australian citizens or permanent residents or New Zealand citizens

1.2. Domestic full-fee graduates of an SA university who are Australian citizens or permanent residents or New Zealand citizens

* Currently only Category 1.1 are guaranteed positions in NSW.5 † Group 1 (not groups 1.1 or 1.2) are guaranteed a position in South Australia.6

3 Internship allocation systems for priority 1 applicants*

State

Allocation methodology

Further details

New South Wales

Computer algorithm

Allocated to hospitals by a program maximising preferences of the group via optimised preference matching

Victoria

Merit

Selection by hospitals, eg, using curriculum vitae (CV), group or individual interviews, referee reports

Queensland

Computer ballot

Some randomly selected applicants to oversubscribed hospitals are allocated to undersubscribed hospitals

South Australia

Computer ballot

Some randomly selected applicants to oversubscribed hospitals are allocated to undersubscribed hospitals

Western Australia

Merit

Applicants provide written answers to set questions regarding:

ability to work effectively in a health care environment
commitment to working in WA
commitment to continued learning

Tasmania

Merit

Selection by hospitals based on CV, interview and referee reports

Northern Territory

Merit

Selection by hospitals after centralised application process; informal preference given to Flinders University graduates and others who completed rotations and electives in the NT

Australian Capital Territory

Merit

ANU graduates are the highest priority, remaining positions are filled on merit as determined by CV and interview.

* Details were checked with relevant state and territory authorities.

Public reporting of health care-associated infection data in Australia: time to refine

National health care-associated infection indicators require validation, stakeholder input and risk adjustment to reflect quality improvement adequately

In December 2011, the Australian Institute of Health and Welfare (AIHW) launched the MyHospitals website, allowing national reporting of safety and quality indices for Australian hospitals, including health care-associated infection (HAI) indicators.1 Unlike approaches taken in the United States and United Kingdom, public reporting initiatives have lagged in Australia, with challenges being identified in the design and implementation of reporting strategies.2 Specific avenues for improving HAI indicators are now emerging.

The MyHospitals website contains data reported by individual hospitals.1 While all public hospitals submit data, participation by private hospitals is voluntary. Safety and quality indicators include compliance with hand-hygiene practices and rates of Staphylococcus aureus bacteraemia (SAB). Each is compared with a national benchmark — greater than 70% compliance for hand hygiene, and less than 2 days per 10 000 days of patient care for SAB events.

Infection control performance indicators can be broadly categorised as “outcome” or “process” indicators. Outcomes refer to measurable end points, such as hospital length of stay or mortality. SAB events are outcome indicators reported by the MyHospitals website. Currently, all hospitals are compared with a single target rate, and jurisdictional performance is gauged on the aggregate for the relevant state or territory. However, many hospitals (eg, specialist paediatric or cancer hospitals, tertiary referral centres and small rural hospitals) have patient populations that are different from those of most general hospitals. For rational comparisons to be possible, data should be risk-adjusted or stratified to correct for differences in patient casemix across the wide range of health care facilities. This sometimes requires collection of additional data. Determining whether an infection is
(i) present and (ii) health care-associated is not always straightforward, as definitions can be complex and are updated over time.

In contrast to outcome indicators, process indicators encompass a broad range of risk-reduction measures and accepted best practice, such as the use of appropriate antibiotics before surgery and ensuring health care workers’ vaccinations (eg, influenza, hepatitis B) are up to date. Compliance with hand-hygiene practices is a process indicator reported by the MyHospitals website. Process measures generally do not require risk adjustment3 as they are often not patient-specific, but should be applied across the board. However, consistent data collection methods are essential for fair comparison. Process indicators allow unfair comparisons of disparate health care facilities to be more easily avoided and unambiguous targets to be applied.3

The benefits of public reporting in health care include the ability to drive change in practice and reduce risk to admitted patients. Hospital administrators may opt to support and actively resource areas of need, with the aim of maintaining standards and achieving public confidence in their hospital. In the UK, a considerable reduction in
S. aureus infections has been ascribed, in part, to public release of data,4 together with strong and frequent media exposure. The potential pitfalls of public reporting of HAI indicators include the provision of misleading data,5 unfair comparisons between dissimilar health care facilities,6 the application of unfounded target thresholds,7 and an undue focus by health care facilities on a “rate” rather than on prevention of HAIs,7 ultimately diverting a disproportionate amount of infection-control consultant time from prevention to surveillance activities. Expectations must also be realistic. To date, although quality improvement activity has been enhanced at the hospital level,8 improvement in health care performance or any reduction in HAIs have not been demonstrated.9 Further, public reporting has not been shown to influence consumers’ choice of hospital.

The measures that are currently reported have limitations. For example, a hospital may choose to perform regular hand-hygiene surveillance of high-risk wards with rotation of surveillance in other wards, or surveillance of high-risk wards with auditing of all other wards, or surveillance in the intensive care unit with auditing of all other wards.1 Ideally, the process should be uniform across all health care facilities to enable valid comparison. However, as the sizes of health care facilities in Australia vary widely, this is unlikely to be achieved, and stratification of centres would therefore be a sensible way to compare similar data. Another limitation is that non-standardised data are also used to calculate SAB rates. For instance, the denominator (patient-days) includes psychiatry admissions, which are generally associated with a very low risk of developing SAB. Hospitals have large differences in their number of psychiatry beds, so this can affect calculated SAB rates. Also, no adjustment is made for different rates of use of intravenous catheter or haemodialysis access devices between facilities, despite these devices being associated with an increased risk of health care-associated SAB.

A number of factors are necessary for a valid and beneficial strategy for public reporting of infection control indices.3 First, the choice of a reportable outcome or process must be based on burden of illness, preventability and feasibility of monitoring. Second, the target for surveillance and the audience must be defined, together with the intended objectives of reporting.3 Third, valid methods of data collection, analysis and reporting must be applied.10 This includes consistency with widely accepted case definitions and applying appropriate methods for risk adjustment.3 In the absence of methods for hospital-level risk adjustment, the US Centers for Disease Control and Prevention recommends reporting of HAI data according to specific hospital units (eg, intensive care unit, transplant unit, surgical wards) rather than reporting hospital-wide data.11 Hospitals have also previously been stratified according to casemix or size to enable meaningful comparison of SAB rates.12 To reduce the burden of manual data collection, electronic data sources may be considered to optimise case detection.3 Any targets that are set must be: (i) justifiable in terms of available evidence; (ii) reviewed in a timely fashion; and (iii) revised if any improvement in outcome is achieved. For example, SAB rates below the stipulated target (2 days per 10 000 days of patient care) are now reported by many centres,13 and it could be argued that a lower threshold should now be applied. Finally, feedback about the reporting strategy must be sought from stakeholders. Potential incentives or penalties for participating in a system of public reporting must be communicated, ideally with strategies in place for when targets are not met.

Given the limited time frame since the launch of the MyHospitals website, some of these criteria have not yet been addressed. Interestingly, it has been suggested that bloodstream infections associated with central venous catheters, surgical antibiotic prophylaxis, and influenza vaccination rates among health care workers should be the priority and the minimum data that are publicly reported,3,14 rather than the indices selected by the AIHW (SAB and hand hygiene). The scope of the current strategy does not fully represent the Australian health care system, as participation by private facilities is not mandatory. It is assumed that the target audience is the Australian general public. However, it is not clear how data are to be analysed over time or if tests of statistical significance ought to be applied. In small hospitals, a small number of infections may lead to very high rates, interpretable only with the understanding of sample size effects and accompanying confidence intervals. Reporting on the MyHospitals website takes this into consideration by reporting the number of events, rather than a rate of infection, for hospitals with fewer than 5000 days of patient care per year. Quality-assurance measures to ensure submission of valid data from all surveyed health care facilities have not been formally defined. To improve the quality of captured data, implementation guidelines for surveillance of SAB have been released by the Australian Commission on Safety and Quality in Healthcare, and concerted efforts have been made at a national level to train hand-hygiene auditors by standardised methods.13 However, jurisdictions are ultimately responsible for the quality of submitted data. Published threshold rates represent consensus or expert opinion, rather than evidence-based targets for improved quality of care. It is also unclear if public opinion on the relevance of content, terminology, and educational value has or will be canvassed,15 or if the availability of data is likely to or is intended to influence patient perceptions and decisions about the choice of facility before hospital admission.

While it would be optimal to investigate each of these factors, a formal review may not be practical. As a minimum requirement, measures to ensure the validity of data capture, analysis and reporting are paramount. This has also been identified as a priority after a review of reporting by the English National Health Service.16 If methods are not well founded, the value of data will diminish, stakeholders will not support findings, and the resources hospitals need to collate data may be regarded as unjustified. Hospital care and HAIs are complex, and stakeholders with expertise in public health, infectious diseases, infection control, informatics and epidemiology must all be engaged to ensure that valid data are released.

We welcome public reporting of infection control indicators in Australia, and applaud the efforts taken to date to accomplish the release of data and jurisdictional agreement. However, to improve quality, the strategy requires further development. The focus must now be on validating and enhancing reported indices according to local epidemiology, stakeholder opinion and the needs of the Australian public. Other reportable processes (such as rates of influenza vaccination among health care workers) and outcomes (such as rates of central-line-associated bloodstream infections) may be considered as quality measures, but not before the current reporting strategy for hand-hygiene compliance and SAB events is refined.

Risks of complaints and adverse disciplinary findings against international medical graduates in Victoria and Western Australia

To the Editor: In a recent article,1 Elkin and colleagues concluded that, compared with Australian-trained doctors (ATDs), international medical graduates (IMGs) posed a greater risk of attracting complaints to medical boards. However, we think the conclusion is not supported by the data, and might have been compromised by the analytical methods used.

Box 1 of the research article1 shows that IMGs accounted for 37% of the total number of doctors, but were involved in 30% of complaints, suggesting that the risk of complaints among IMGs was lower than among ATDs. Indeed, in a reanalysis of the data, we found that the rate of complaints against IMGs was 25% lower (P < 0.001) than that for ATDs (Box).

The logistic regression used by the authors is suboptimal for count and rate data, which are likely to follow the Poisson distribution. We reanalysed the data using a Poisson regression model assuming similar durations of practice and found that the rate of complaints was highly dependent on the language background of IMGs (Box). For instance, compared with the complaint rate against ATDs, the complaint rate among IMGs whose countries list English as an official language was significantly lower (34%; P < 0.001), but was not lower among IMGs from non-English speaking countries (P = 0.098). Furthermore, among IMGs whose language requirements may have been waived by the medical board, the rate of complaints was substantially lower than among ATDs (46%; P < 0.001).

In summary, our reanalysis of the data suggests that compared with ATDs, IMGs had a lower rate of complaints against them.

Rate ratio of complaints against international medical graduates (IMGs) compared with Australian-trained doctors (ATDs)*

Rate ratio (95% CI)

IMGs v ATDs

0.75 (0.71–0.79)

< 0.001

Analysis by official language

English†

0.66 (0.62–0.71)

< 0.001

Language other than English

0.94 (0.86–1.01)

0.098

Analysis by English requirement

Waived‡

0.54 (0.50–0.59)

< 0.001

Certificate required

0.95 (0.89–1.01)

0.126

* The analysis was based on the Poisson regression model using data presented in Box of the original research article.1 † Countries where English is considered an official language include United Kingdom/Ireland, South Africa, New Zealand, India, Nigeria, Pakistan, the Philippines and Singapore. ‡ Countries for which the requirement for an English language proficiency certificate may be waived include South Africa, New Zealand and United Kingdom/Ireland.

Risks of complaints and adverse disciplinary findings against international medical graduates in Victoria and Western Australia

To the Editor: In the article by Elkin and colleagues,1 the conclusions drawn from the data analysed do not appear to be adequately supported. It should be noted that only 30% of complaints against doctors had a final hearing, and the risk of adverse findings (ie, where the doctor was found to be guilty) was not analysed by country of origin. Although doctors from certain countries attracted higher rates of complaints, it is inappropriate to label them as “high risk” in the absence of data on adverse findings. It is conceivable that the complaints against the so-called low-risk groups (typically, from English-speaking countries other than Australia, such as the United Kingdom, New Zealand and similar countries) were less likely to be simply driven by cultural misunderstanding or racism and might therefore have been based on actual deficiencies in practice. This, in reality, could possibly be driving the significantly higher risk associated with international medical graduates (IMGs) as a whole in terms of the rate of adverse findings. In addition, a larger proportion of complaints attracted by the so-called high-risk groups may eventually be proven to be unsubstantiated. Therefore, this study is unable to prove whether the professional practice of the group labelled as high-risk was concerning at all.

We agree with the authors that more research is needed on underlying causes for complaints against IMGs and Australian medical graduates. Ideally, such a study would use data on complaints adjudicated by board members blinded to the doctor’s country of training. Equally important would be creating more opportunities for acculturation and supervised training for IMGs, especially in the initial period of their practice in Australia.

Risks of complaints and adverse disciplinary findings against international medical graduates in Victoria and Western Australia

To the Editor: While using medical board data for research purposes is very welcome, I have some reservations about ethical aspects of the study by Elkin and colleagues.1 Prospective ethics review of human research is intended to protect participants from foreseeable and avoidable harm. I suggest that, in this instance, avoidable harm has resulted, through generalised adverse and possibly unfair publicity about particular groups of international medical graduates.

Although identifying “complainant factors” as one source of potential bias, the authors interpreted the data as if this potential bias could be ignored. The surprising difference in complaint frequency between Victoria and Western Australia might say something about these complainant factors, but this is left unremarked. In addition, they failed to identify another potential source of bias in the outcome data, namely the potential for medical board adjudicators to assess complaints in a prejudiced way. Without knowledge of the nature of the complaints, the investigative findings and the outcomes that resulted, readers cannot assess the possible existence and impact of such biases.

A claim is made that complaints represent a form of sentinel-event surveillance system that flags problems, which may be true for investigated and substantiated complaints. But this cannot be asserted when, in this study, 70% of complaints were not investigated and of the 1606 that were investigated, 1233 apparently did not result in an adverse finding.

In my view, at the very least, the study should not have been published in the format chosen. Any important potential conclusions relevant to the regulator and the wider medical profession need not have identified specific countries of origin. I feel deeply for the vast majority of doctors in the identified so-called high-risk groups who have never been the subject of a complaint. I hope that the authors, the human research ethics committee involved, the reviewers and your editorial team will, after reflection, act to ensure better participant protection in the future.

Clinical effectiveness research: a critical need for health sector research governance capacity

The barriers to conduct of clinical research will require solutions if we are to implement evidence-based health care reform

Reforms in the funding of health services, such
as “activity-based” funding initiatives, seek to facilitate changes in how health care is delivered, leading to greater efficiency while maintaining effectiveness. However, often these changes in treatment strategies and service provision evolve without evidence demonstrating effectiveness in terms of patient outcomes. The pressures on health care expenditure (currently around 9% of gross domestic product1) make such an approach untenable and unsustainable. The evidence necessary to support these initiatives can only be derived through carefully conducted clinical research. Most readers would immediately think of clinical trials in terms of pharmaceuticals or clinical devices, and this type of research is critically important, although continuing to decline, in Australia.2 Other questions relate to the effectiveness of changes in health practice or policy, usually (but not always) based on sensible ideas that seem self-evident. However, in order to function with an evidence base, these ideas need to be proven to be clinically effective and cost-effective. Such research can be costly, and many of the questions to be addressed are not ones that would be the subject of an industry-sponsored trial. Researchers, clinicians and health administrators are therefore faced with the problem of how best to measure the outcomes of changes to health care strategies, without the necessary resources to ask and answer the question.

The MJA Clinical Trials Research Summit held in Sydney on 18 May 2012 included a working group addressing issues of research governance and ethics. The key discussion outcomes of that group were:

confusion exists regarding the differences between ethics and governance;
variability continues in state and federal legislation and regulations, despite attempts at harmonisation;
processes for improvement at government and institutional levels are underway but are not yet complete or implemented;
hospital boards and chief executive officers need to have incentives to make the infrastructure work;
substantial challenges exist when working with international investigator-initiated trials;
trials involving the private health sector include specific difficulties such as insurance and contracts; and
national accreditation of researchers and training should be considered.

Costs are not the only barrier. Efforts to rationalise health care provision on the basis of evidence provided through the conduct of clinical research are also hampered by existing or perceived obstacles in the form of cumbersome institutional research governance and ethics approval processes. Substantial changes and streamlining of the processes of ethical review are underway across Australia, addressing inconsistencies and inefficiencies of human research ethics committee approval, financial processes, and contractual clinical research governance processes. Nevertheless, the system remains complex, slow and expensive. Unfortunately, the old adage of “good, quick or cheap: pick two” still applies.

Many researchers fail to distinguish between research governance and ethics. Clinical research in Australia is governed by the National Health and Medical Research Council (NHMRC) National statement on ethical conduct in human research3 and the Australian code for the responsible conduct of research.4 Research governance can “be understood as comprising distinct elements ranging
from the consideration of budgets and insurance, to the management and conduct of scientific and ethics review”.5 Research governance thus includes oversight of all processes of ethics review, but also includes responsibilities of both investigators and institutions for the quality and safety of their research.3

The Harmonisation of Multi-centre Ethical Review (HoMER) initiative by the NHMRC is a significant step forward, enabling a single ethics review process that has been adapted for several states. This process, if used effectively, should reduce the resources required to obtain ethics approval for multicentre research, but it has also created some challenges in ensuring that research governance obligations are maintained within various health service jurisdictions.6 Currently, no incentives or requirements exist for health services or hospital chief executive officers to ensure that appropriate infrastructure is in place and working. Similarly, a different set of challenges arises when considering performing research in the private sector, where insurance and contractual issues may differ substantially from those in the public sector.

Much of the non-industry-sponsored clinical research performed in Australia is investigator-initiated research, supported by funding organisations such as the NHMRC, state governments, and other non-government organisations such as Cancer Council Australia, the National Heart Foundation of Australia and cooperative clinical trial groups. At present, investigator-initiated trials require comparable levels of research governance and are certainly subject to the same requirements for good clinical practice as industry-sponsored trials. The research questions addressed by these studies are based on clinical imperatives, a broad understanding of the underlying science, and a necessary ability to work on a shoestring — the latter being the main point of distinction from industry-sponsored trials. Current models of competitive research grant funding do not recognise the complexities, duration, costs and distribution of costs across the length of a clinical trial, especially when considering late clinical outcomes that are often the most clinically relevant ones. As an example, an NHMRC project grant can be funded for at most 5 years and therefore necessitates a focus on end points occurring within end-point time frames. The clinical questions that we and the community recognise as important might not be able to be answered with such designs. The resources required to meet these requirements continue to escalate and we currently run the risk that these trials will soon be untenable in Australia. Anecdotally, many academic clinical research units are already questioning what level of involvement they should have in such relatively underresourced trials or if they should be involved at all, for the most part purely for financial reasons.

Within the current Australian health care environment, clinical research is being conducted in the face of significant headwinds. These inefficiencies arise from resource costs due to complex governance arrangements combined with those of research conduct (Box). Processes to be considered that will improve clinical research capacity might include:

continued adoption of electronic health records that span clinical, investigative (ie, pathology and radiology) and therapeutic information (eg, the Australian Orthopaedic Association National Joint Replacement Registry);
data-linkage techniques to obtain clinical outcomes
(eg, hospital readmission data, Medicare Benefits Schedule and Pharmaceutical Benefits Scheme use data, the National Death Index);
better integration of research into routine clinical practice;
national accreditation of investigators;
standardised good clinical practice training;
increased profile for research participation at the clinician–patient level, enabling the conduct of studies that are more representative of a wide spectrum of patients;
development of a clinically relevant strategic research agenda led by collaborations between clinicians, researchers and health policy decisionmakers;
a culture shift where lawyers and hospitals communicate and quantify the risk or research appropriately.

Research developed through partnerships between health policymakers and health service providers should lead to outcomes that are more immediately relevant and translatable to the care we provide, the outcomes we achieve and the costs incurred by the health system. Reinvestment of financial and efficiency gains realised from initial research outcomes back into the next relevant translational research question provides a model for a sustainable health system that evolves with the support of a robust clinical research-driven evidence base. These financial windfalls currently go back into government coffers and ideally should be seen as a potential funding stream to support future clinical research.

As the demands on our health system continue to mount, the need for clinical effectiveness research to build a robust evidence base upon which to reform care has become even more acute. It will be critical to align the clinical and policy research agenda while strengthening the governance structures that facilitate the conduct of research within the clinical space if we are to develop “an agile, responsive and self-improving health system for future generations”.7

Key points

Barriers to clinical research include:

regulatory complexity
inflexibility of ethical review and oversight
funding models that are not designed to support clinical trials
lack of incentive for engagement of health services in research support.

Solutions may include:

different funding models, including support for longer time frames
simplification of ethical and governance processes recognising the different goals of industry- versus investigator-initiated research
better involvement by health services in supporting research
return of savings from clinical research to support further research
clinical research key performance indicators for health service administrators.