more_vert
THE Medical Board of Australia has released a research report on revalidation as part of a “conversation” on its introduction to Australia.
It is hard to dispute the general idea that doctors, like any other professionals, should be up to date and fit to practice.
However, the precise purpose and preferred approach to revalidation remains unclear. Is revalidation simply a formative process to support ongoing learning and improvement, or does it also seek to identify and intervene with poorly performing doctors?
And if screening doctors to identify those who are unfit to practise is one of the goals of revalidation (as many patients would reasonably expect), does such an approach stack up against the evaluation of public health screening programs?
Here I outline four areas for consideration.
First, an effective screening program should target an important problem with a well understood natural history.
There is little question that doctors whose performance is impaired due to health, conduct or performance concerns pose a risk to the public. Current best estimates are that, at any given time, 1%‒2% of doctors are impaired in their ability to practice, and fewer than 5% account for around 50% of patient complaints.
The natural history of unsafe practice varies — some doctors practise safely for many years before a health condition or other life stressor makes performance dip, while others lag behind their peers from the start.
A more sophisticated understanding of the epidemiology of doctors who have health, conduct or performance issues may help us to identify early warning signs of deteriorating performance.
Second, screening tools should be capable of differentiating between high and low risk individuals to identify genuine concerns, while minimising the resource and personal costs associated with false positives.
All screening tests— including commonly used approaches to revalidation — are imperfect. Self-assessment relies on doctors’ limited ability to recognise their own deficiencies, written exams bear little resemblance to clinical practice, complaints underrepresent certain voices, and practice audits may not probe beyond “surface compliance”.
The recent report on revalidation provides a helpful overview of the strengths and limitations of each approach. Importantly, the report suggests that a targeted approach, based on risk factors such as age, practice type or complaint history, may improve the performance of certain revalidation tools compared with a one-size-fits-all approach.
Third, screening programs are of little benefit if effective remediation for identified problems is not available.
When dealing with doctors in difficulty, the long-term objective is to support them back into safe practice, not to “weed them out” (although in rare cases this may be the only realistic solution).
Much has been written on the importance of health programs for impaired doctors. However, there is little hard data on the comparative effectiveness of these programs. Even less is known about the most effective interventions for doctors with performance or conduct concerns.
This dearth of evidence is worrying given the potential risks to the public of an ineffective approach to remediation.
Finally, the screening program as a whole should be acceptable, cost-effective and sustainable.
The costs of revalidation are both direct (producing assessment tools, administering the program and following up on concerns) and indirect (distraction from clinical care, anxiety and inconvenience).
At a broader social level, the time and resources devoted to revalidation are then lost to other aspects of health care.
On the other hand, a program that ensured all doctors were up to date and fit to practice would carry great benefits for patients, health systems and the profession as a whole. Understanding these trade-offs will help ensure the appropriate and sustainable allocation of resources.
As doctors we have an obligation to ensure that the trust and confidence the community has in our profession is well founded.
Revalidation offers a beguiling option for achieving that goal. However, history is pock-marked with examples of public health screening programs that were introduced with more enthusiasm than evidence.
To avoid a similar fate for revalidation in Australia, we need a transparent debate regarding the intended aims, costs and benefits of revalidation, an ongoing investment in research to improve the targeting and performance of revalidation tools, and a commitment to using evidence-based interventions to support struggling doctors back into safe practice.
Dr Marie Bismark is a public health physician and health lawyer at the School of Population and Global Health, University of Melbourne. Her research focuses on the role of clinical governance, regulation and patient complaints in improving the quality and safety of health care. On Twitter: @mbismark
Acknowledgements: Thanks to Margaret McCartney @mgtmccartney who inspired this article with her tweet of 10 June 2014: “revalidation is a screening test of unproven efficacy and unquantified harms”
Loading ...
Marie Bismark’s article ‘Guaging Revalidation’ opens an interesting debate. The Australian Medical Board ‘Research Report’ is verbose, badly referenced and avoids professional competence, which is pivotal to this issue. Solutions to medical regulation are available and must cover all circumstances. Factors contributing to healthcare ‘incompetence’ in two cases are illustrative.
Use of American College of Cardiologists (ACC) guidelines in hospitals managing patients with STEMI and non-STEMI was shown to reduce mortality in 2006. A 10% fall in compliance was associated with a 10% rise in mortality. A 2015 BMJ article showed differences in compliance with STEMI and non-STEMI guidelines (European Society of Cardiology & ACC) between Sweden and the UK. The variation accounted for >3000 deaths if best practice had been achieved in all UK hospitals. Despite revalidation no practitioners registration status will change. No hospital will be held to account for the difference in guideline adherence adversley affecting patient survival. Despite the mangement contribution to the development of ‘underperforming hospitals’ there will be no ‘revalidation for managers’.
In 1998 three UK doctors were found guilty of serious professional misconduct relating to the deaths of <20 children in a cardiac surgery centre. 171 children over 12 years may have survived had they been operated on elsewhere.
Revalidation may not have the impact on improved outcomes the public, DoH or profession expects. The profession knows high quality healthcare delivery is a team activity. All healthcare professionals, including managers, must accept responsibility for achieving optimal outcomes.
A fascinating paper from Michigan by Birkmeyer et al (N Engl J Med 2013; 369:1434-1442October 10, 2013) showed that clumsier surgeons had worse results (intuitive, but rarely proven!), and chillingly, that experience did not correlate with skill.
With any psychomotor skill there will be individual variation, and the talents of Mozart or daVinci were evident as young children, long before formal training. Comparing the stellar and public careers of Cate Blanchett and Arnold Schwarzenegger, each has made many successful films, but it would be hard to say that either has improved in acting skills over decades.
Birkmeyer’s large study shows that clumsy surgeons kill more patients – yet I am unaware of any revalidation/ retraining which proves a quantifiable patient benefit by improving an individual’s skills. And Birkmeyer’s surgeons were all in a US board-certified role, showing no benefit to their revalidation model.
Given that we have a Medical Board which has been remarkably inactive at dealing with highly public allegations about how senior surgeons treat their trainees and got away with it, I am unconvinced that the Board can deal with a more complex issue like technical underperformance.
Let’s also note that the Medical Board has a significant self-interest here. As it stands, the Board’s main function is to maintain a database of doctors and to issue renewals. We all know how competently they have done that in the past. One could argue that outsourcing this function to Computershare or similar would be cheaper and more efficient. The other main function is investigation of complaints. If the Colleges were to take over this function (and they are perhaps better equipped to investigate in their own field), then there would be little for the Board to do and we could seriously contemplate simply abolishing it. On the other hand, if the Board can become the owner of revalidation, then its future and the jobs of its members are secure.
Advocates of revalidation are strong on rhetoric and soft on science. Motherhood statements like ‘increasing public trust’ are not backed up with meaningful numbers. What is needed is to clearly define set goals, then a prospective RCT to assess the validity of the intervention, and then obtain cost benefit analysis and number needed to treat.
For example, half the medical population can be randomised to revalidation, and the other half not. Then hard variables can be assessed at the end of a 3 year period. This will show whether revalidation has any patient benefit. Patient variables would be specific to each specialty. Cardiologists can be assessed for death rate and re-infarction rate, vascular or bowel surgeons can be assessed for anastomotic leakage, endocrinologists for mean glycated haemoglobin and hypoglycaemic episode frequency, nephrologists for mean arterial blood pressure, and orthopaedic surgeons for fracture nonunion rate or prosthetic dislocation.
Doctor variables could include time and dollar cost spent on learning activities in general and specifically geared towards re-validation, time away from practice, and given that there will be those who fail revalidation and are subject to significant stress, suicide rates or substance abuse rates. Could any advocates suggest variables they would measure?
Having recently turned 60 I now work part time. If revalidation looks like being costly time consuming or stressful I will ask myself if it is worth it. My years of experience will then be lost. Is that a good outcome for the community? I suggest there needs to be serious consideration of the pluses and minuses of such a process. Just like surgery medication psychotherapy Etc there is a risk of untoward side effects. There needs to be care that the cure is not worse than the disease (or ask how much disease there is in the first place)
The whole idea is impractical. Would an orthopaedic surgeon who only does lower limb work (or even just knee surgery) have to be revalidated in general orthopaedics? What about an academic physician who has confined his/her practice to cerebrovascular disease, or to hypertension? These are not specialties recognised by AHPRA. As medical practitioners age, they tend to focus their practice more and more on their particular areas of clinical interest and skill, attend conferences on these topics and narrow their expertise. This does not make them unfit to practise.
The idea of defining every medical practitioner over the age of 60 years as being at high risk to the public is ridiculous, insulting and probably discriminatory. I turned 60 earlier this year, and find that as an “empty nester”, I have more time for attending conferences, keeping up with the literature, etc, than when my children were at school. Older practitioners sit on hospital and College committees, write papers, review grants, supervise postgraduate examinations and so on. How old is Dr Flynn, the head of AHPRA, and what revalidation process does she have in mind for herself when she turns 60?
The medical profession is being singled out for special bureaucratic attention yet again, and it is crucial that we fight these proposals. Other professions can have a great impact on people’s lives if a mistake is made – politicians and judges for a start. Yet they are not subject to revalidation. The medical profession in the USA is having a terrible time with revalidation, with practitioners typically having to take several weeks off work and pay thousands of dollars to prepare for their recertification examinations. Let’s not allow this to happen in Australia.
Is the Medical Board competent to evaluate and revalidate cardiothoracic surgeons, neuro-oncologists, occupational health specialists? What about GPs – the Medical Board would need to demonstrate that it was more competent to evaluate GPs than, say, the College of GPs. In the absence of such a demonstration then revalidation of all of these doctors is more properly done by the relevant College. It’s hard to see how the Medical Board could be more competent than the Colleges in matters of professional standards. The Board’s push into this territory should be strongly resisted until and unless the Board can show that it brings something unique and useful to the table.
Second all of the above. The Med Board have given us no evidence that there is a serious problem of public health due to inadequate practitioners, no evidence that there is a sensitive and specific test and no evidence that there is a cost-effective intervention. Are these three not the basic criteria for a public health programme?
Could we begin by asking Dr Flynn and the Med Board to publish a paper showing the definitions of inadequate standards and the incidence and prevalence of these in Australia for some recent year, say 2014? Then and only then could we start to have a discussion of what could/should be done.
Where is the evidence that practising doctors are so dangerous? In my experience, most problems occur as a result of system weaknesses -poor economic viability of longer GP consults and continuity of care via provision of after hours care, and crowding and strains on public system etc l taught briefly at tertiary level (where l was dismayed at superficiality) and for several years in the public system (where lack of basic knowledge impressed); l think attention would more profitably directed at this undergraduate level. Will it be only GPs subjected to revalidation, or will this extend to all under the AHPRA umbrella ie specialists, and allied health practitioners?
Apparently there is evidence that older doctors (>60 yrs) are at greater risk of being poorly performing. This clearly does not take into account the increasing number of mature age students currently in medical training. Perhaps it should be based on number of years in practice as opposed to actual age of the doctor.
The critical point is that “… the time and resources devoted to revalidation are then lost … to [clinical] … care.” We are seeing this phenomenon throughout health care. Let our professional associations and Colleges be quite blunt with bureaucracy: unless a method of revalidation can be demonstraed to be cost effective, the profession will not co-operate. Too much money is wasted on giving politicians and bureaucrats a warm, fuzzy feeling without significant benefit to the society.
We do need to be very careful about the drivers for this and about what is introduced.
It must also be acknowledged that one of the drivers is a bureaucratic need to be seen to “do something” , in other words to protect the positons of institutions such as AHPRA, which has a distinctly patchy performance record.
The comments about methodological validity are also critical – we all know doctors to whom we would never send anybody who sail easily through examinations, which have little validity in predicting long term clinical competence. (not bad at predicting exam success – surprise!)
The first step must be to prove that we have a problem worth addressing, then to prove that the solution will address this. All systems can be gamed – Harold Shipman would have passed most practice audits. The truly bad will find a way to game the system, the disorganised and incompetent should be detectable by means currently available.
The knee jerk response of requiring examinations, as in the USA, is no solution, as discussed above. It is all too easy a solution for politicians and bureaucrats to impose – easily done and generates an income stream , while creating the illusion of having done something.
CME based systems have the advantage of raising the standard of all, good and bad, if properly designed and enforced. Their disadvantage is that they advantage people like me who practice in tertiary/academic areas, as much of our normal work can be made to count (not necessarily a bad thing ) while the solo/rural-remote GP or specialist, has to jump through addtional hoops at some expense and inconvenience. Surely this can be fixed.
well said. just because there is a problem doesn’t mean there is a solution that provides a better outcome, after taking into account costs, both opportunity and real.
Some good insights here thanks Marie. Australia must have the debate, and we should remember that we can learn from the experiences of other countries. For example, there is sound evidence that reliable multi-source feedback (MSF) that is appropriately implemented is able to provide helpful insights for both highly and poorly performing doctors. Patient feedback helps identify both strenghts and weaknesses with communication and interpersonal skills. Colleague feedback focuses more on the doctor’s professionalism. Experience shows that such feedback, both from patients and colleagues, is often welcomed by doctors as being constructive in their on-going professional development, as well as providing the public with a sense of confidence that the doctor is ‘fit to practise’.