more_vert
ON Saturday, 20 November 2021, the Royal Australian and New Zealand College of Psychiatrists (RANZCP) experienced a devastating examination failure.
Across Australia and New Zealand, 248 senior psychiatry trainees and specialist international medical graduates (SIMGs) were sitting at their home computers, sweating into their suits but ready to attempt an audio-visual version of the Objective Structured Clinical Exam (OSCE). Elsewhere, actors and volunteer psychiatrist examiners were ready and waiting to begin.
Many candidates, especially those in New South Wales who had their July 2021 OSCE cancelled at the last minute due to the lack of a COVID-19 outbreak contingency plan, had been waiting since early 2020 for an opportunity to sit the exam and complete their training.
Then, disaster struck.
After a week of fraught technical practice sessions, which had already distressed and distracted the candidates in the lead-up to the exam, the platform failed spectacularly. A small portion of the morning session’s candidates were able to partially complete the examination. The afternoon candidates received urgent emails from the College advising their session would still go ahead, but about an hour into the scheduled registration time, the College made the decision to abort.
To the candidates’ dismay, the College had seemingly failed to develop a contingency plan, and at time of writing they were scrambling to find a remedy. Scores of letters from trainees, psychiatrists and various committees – many of them shared on social media — have been pouring into the President’s inbox. Candidates are advocating for alternative assessment pathways, noting the abundance of pre-existing workplace-based assessments and centrally administered examination results from which to draw.
A binational survey of candidates was rapidly deployed by the New South Wales Association of Psychiatry Trainees (NAPT) and yielded alarming results. Of the 206 respondents, 53% said they had already completed their training time or would do so within 3 months. For 88% of respondents, this was not the first time the College had either cancelled an exam during the pandemic, or told them they were ineligible to sit on account of limited spots and not being deemed a “priority” candidate.
RANZCP has struggled to run its OSCE throughout the COVID-19 pandemic. According to College reports (available only to RANZCP members), only 207 registrars have had the opportunity to sit the OSCE during 2020 and 2021. By comparison, 680 registrars sat the OSCE during 2018 and 2019.
Most concerningly of all, the free text responses to NAPT’s survey ran to 25 printed pages and contained a litany of horrific stories about the impact of the exam failure on the candidates and their families, after months or years of study.
“I feel trapped and unsupported,” wrote one candidate.
“I’m a single mother. My child needs me,” wrote another.
SIMGs spoke of the added stress of not being eligible for permanent residency until they attain Fellowship, and how as a condition of their SIMG status they were prohibited from working part-time or leaving the country to visit loved ones overseas.
Relationship stress was commonly mentioned, as was the deferral of on-call shifts until after the examination date, meaning it would be nigh on impossible to prepare for any reschedule.
“Let me finish my training. Please,” begged one registrar.
What went wrong
The exact technical failures are yet to be revealed; however, other colleges, including the Royal Australian College of General Practitioners and the Royal College of Psychiatrists in the UK, have successfully delivered similar clinical exams during the pandemic. Ultimately, the problem appears to relate to poor project management and a failure to prepare a contingency plan.
The first sign of trouble was in early September 2021, when candidates were informed the OSCE would be postponed from October to November on account of technical difficulties. NAPT wrote to the College on 31 October with a further request that contingency plans be developed.
In the week before the exam, it became clear to candidates that the postponement had not been particularly fruitful. In the belief that participating in testing would help familiarise them with the platform, candidates logged on to a series of technical run-throughs which proved alarming. There were reports from those familiar with information technology and project management that basic load and user acceptance testing appeared not to have been performed. Some candidates took time away from study to propose back-up plans, including a simplified process using only the Zoom platform, with slightly longer breaks between stations, and with invigilators sharing OSCE stems via Zoom’s screen-sharing function.
Hierarchy and culture
In my first year of psychiatry training, I wrote in InSight+ about the harms to trainees and patients of excessive hierarchy in medicine. This latest examination failure has occurred in this milieu.
The RANZCP has a two-tier membership structure. Fellows have voting rights and seven of them are elected to form the College’s Board of Directors. The Board is responsible for appointing — and, when necessary, firing — the Chief Executive Officer and ensuring the College performs its core functions, which include graduating the next generation of psychiatrists and promoting mental health.
Trainees and SIMGs, however, are classed as “associate” members. This category is a subclass, devoid of voting rights. It is a case of taxation without representation. According to the College’s latest annual review, there were 1822 associate members out of 7023 total members; that is, 26% of the College membership. Yet trainees have no vote, and no seat on the Board – unlike our physician and emergency medicine peers.
Collectively, the membership fees of associate members contribute about $2.4–$3.4 million of the College’s $42 million annual operating revenue. In addition, trainees are charged joining ($702) and examination fees ($425–$2658 for each of six assessments). Given there were 248 candidates for last Saturday’s exam, that single OSCE provided the College with $659 000.
The RANZCP does have a Trainee Representative Committee (TRC), but members are forced to sign a “Deed of Undertaking in Relation to Confidential Information and Conflict of Interest” (and here). TRC members commonly refer to the Deed as a “non-disclosure agreement”, and it has the effect of inhibiting their ability to communicate with the broader trainee cohort. The co-Chairs of the TRC have since announced their resignation, effective from 7 December.
The feeling among trainees, especially after this OSCE exam failure, is one of disenfranchisement.
Where to from here
Acutely, the College needs to find an acceptable means of allowing candidates to progress in their training and start serving the community as psychiatrists. We are cautiously optimistic that this will occur, with statements from the College that an alternative assessment pathway is being developed. It is pleasing that they have listened to candidates and their many supporters that any attempt to repeat the OSCE would not be viable.
The bigger issue is one of rebuilding the relationship between the College and trainees. However, the Board could take two immediate decisions that would help enormously.
The first is to exercise a provision within section 9.1 of the Constitution to appoint two trainees or SIMGs to the Board.
The second is to tear up the TRC’s “non-disclosure agreements” and commit to meaningful and transparent engagement with all trainees – a shift from paternalism to inclusion: nothing about us, without us.
From there, relevant staff and office bearers will need to take accountability for their persistent failure to deliver examinations, which may include terminations and resignations. Trainees should be reimbursed not only their OSCE fees but also, in some instances, specific costs incurred, such as headsets, larger monitors and increased bandwidth they were advised to purchase.
Constitutional change needs to be pursued to give trainees and SIMGs voting rights and a proportional quota on the Board, and the College should finally commit to the implementation of all recommendations of the 2019 Australian Council for Educational Research review of the RANZCP’s examinations.
The alternative is for the College to continue “business as usual”. The risk of this, however, is a long term rupture in the trust of hundreds of soon-to-be Fellows in the institution itself. Given the reliance of all specialist medical colleges on their members to write exams and policy and govern the organisation, choosing this path would be short-sighted and self-defeating.
In psychiatry, we speak of recovery-oriented and trauma-informed practice, both of which involve genuine collaboration and empathy. Well, here is the College leadership’s opportunity to walk the talk.
Benjamin Veness is the Advocacy Officer for the Victorian Association of Psychiatry Trainees. He was not a candidate in last Saturday’s OSCE. Twitter @venessb
The statements or opinions expressed in this article reflect the views of the authors and do not represent the official policy of the AMA, the MJA or InSight+ unless so stated.
Loading ...
There could be another way of assessing a medical specialist confidence and performance since OSCE exam is too stressful and some what discriminatory – every individual is different personally.
Would say an individual experienced specialist mentor in a public hospital and a private practice mentor- can do both make an assessment of a clinical trainee after some years of practice under them.
OSCE is not appropriate or likely obsolete now as part of a trainee assessment since can be mentally traumatic to people and the modern science and technology so complex now that can be very socially and psychologically draining to students/trainee- “Times Of Chronic Depression, Depression, Suicidality And Drugs Abuse”.
Educators need more Innovations, Innovations to improve for the better to all learning, not to harm people.
We learn from the old guards but these age of Computers, we should make our lives easier, less stress and joy in learning new ways of thinking.
As a Fellow of the college, it is disrespectful to trainees that a platform to conduct the OSCE was not developed prior to confirming the OSCE date. We pay considerable fees to ensure that these assessments are conducted fairly and delivered to the highest standards!
Local OSCEs with independent examiners must be a priority to maintain reliability and validity of fellows practicing at a reasonable standard. This is the only independent assessment that can truly assess a trainees ability to communicate with patients, so it is essential. There should be less emphasis on non-essential aspects of practicing as a psychiatrist (scholarly projects, writing essays).
I am a trainee of the RANZCP. I have attempted many times to sit the exams, the MCQ’s; CEQ and MEQ’s; and OSCE’s and have failed to pass these exams. Like most candidates that are in the same position as myself. I find it difficult to have to justify my failures to the RANZCP. In my attempt to continue on this pathway in becoming a psychiatrist, it has become increasingly there are more problems with the RANZCP than the trainees.
There is a blatant culture of ignoring the needs and the quality of trainees attempting to get through these hideous culturally blind assessments that not only impact on the self esteem of the candidates but erodes the confidence of many. I am of Australian Aboriginal heritage and very proud of my culture and my achievements throughout my several postings and career. I continue to stand on my feet and fight than crawl on my knees to address the cultural biases of a one size fits all. The RANZP has not only failed First Nations trainees, but also the First Nations people in this wonderful country.
I believe the rhetorical and hypocritical position must be over hauled and those who are setting the examinations of the RANZCP must stand accountable and as well show transparency. Until then, there will no confidence in the RANZCP, except for those Psychiatrist’s who belong to the majority of those who really show empathy, compassion towards the trainees, and provide their support on the ground and in the workplace.
I’m a RANZCP trainee. I just want to highlight the issue is at a larger scale. The whole college examination needs a reform. In fact the essat-style exam is holding back more trainees than other exams. It’s unique to RANZCP. Our college emphasises on being able to write a critical essay is a must for being a competent psychiatrist. The irony is they don’t have critical thinking when it comes to their training program and exams. The president talks about the importance of being respectful to each other. I agree but how much respect and support is shown by the college to trainees who are going through enormous stress because of the exams process and the low pass rates. Respect is mutual and not just by the way we talk, it should be in action. The feedback from them are a few numbers and not that helpful.
I am almost a consultant, and the repeated disasters with exams (not to mention the limited sittings, excessive examinations compared to our colleagues, and general lack of movement on the part of the college have made me seriously consider jumping ship into another speciality. I do not feel like a valued member of RANZCP. I do not feel valued as a mental health professional. We hear about a deficit of psychiatrists in Australia, yet our college is more inclined to keep their club small than to meet the needs of the public and produce quality psychiatrists. I am no longer proud to call myself a psychiatry registrar.
College has adopted the culture of “apology”
RANZCOG has successfully runs several AV OSCE examinations, like the Anaesthetists, and I suspect other Colleges.
The combined Medical Colleges Committee needs to get all different specialties out of their respective silos and work as a team to make sure that the provision of health services in this country improves across the board. We are all in this together, and I believe rowing the boat in the same direction, but we are going in circles and not talking to each other nor presenting a united front to Government or the public. The expense of funning these exams is not covered by the fees charged, and is above the cost of face to face exams. They are heavily supported by Fellows annual subscriptions. The divisions between various specialties has been encouraged and supported by politicians who have worked on the divide and conquer strategy for years, very successfully I might add.
the college seems more interested in making money than developing a pool a caring clinicians
what a total **** show
This is the time and if not so the qualification and profession risk losing face…..the leadership style is outdated and risks further ruin to an organization with noble ideals…..the college needs to be modernized and the current crop of leaders are not the ones to do so…..the failure of the AV OSCE is just a symptom of a leadership, structural and philosophical problem!
Australian and New Zealand College of Anaesthetists and within the College, Faculty of Pain Medicine have now conducted several exams remotely and successfully. The RANZCP need to consult urgently with this College and others who have successfully ran their fellowship exams. It’s unconscionable that nothing is done to get this latest group of candidates through the examination process.
An absolute lack of transparency and accountability on the part of RANZCP as trainees continue to lose trust and hope under an authoritative system.
The culture within the RANZCP has to change.
I was one of the Examiners. Can you believe that at the start of the weekend software training session when I
rang someone ( not a Psychiatrist ) to notify her of my inability
to access the Examiner Registration session, I was admonished and told that I was the only person who could
not navigate that and that ” everybody else has been able to get in.” Further assistance proved fruitless.
The whole weekend training was a washout !
The ? RANZCP IT person, David, stepped in and was most helpful, kind and reassuring.
I may not be a Tech person but am a Senior psychiatrist, partially retired but involved in other areas that require
some competence in software programme usage and fast execution. As I told David, I am not a complete idiot!
The RANZCP hierarchy is basically or used to be a Boys Club. that used to wield a lot of power( and still does? )
within and outside the RANZCP. I was at the receiving end once when I stood up for a cohort of patients ( and won ).
I was never a rebel at any other time.
Let me not get into the details of this. I want an enjoyable and peaceful retirement !
The college continues to look at ‘supporting’ victims but not looking within self. Hope the reaction include trainees seeking legal redress and seek heavy compensation for the trauma caused
Repeated exam failures immensely impact self-confidence and esteem.
The OSCE exam is subjective though expressed as “objective”. The individual feedbacks after exams are absolutely vague and useless. Though the passing rate has improved in the last few years, if one compares with other faculties, RANZCP still has a very low rate of passing exams. Why?? RANZCP hasn’t made effort to improve the passing rate in the last decade. On the other hand, there are still few trainees who managed to pass every assessments and exams without much hassle.
Salute to those who passed all the exams in one attempt.
I’ve been an RANZCP Fellow for over 40 years, and have served on a branch committee. I have become increasingly irritated by the increased bureaucracy and complexity in College administration since around 2000 when a small group took over the committee in one state and then federally and changed the set up of administration.
An opportunity to rebuild trust. A laughable attempt at reframing.
The spectacular failure of the November AVOSCE and tone-deafness of the RANZCP leadership was not an isolated event, but merely symptomatic of a larger problem.
As one of my psychiatrist peers stated several years ago, “everyone hates the RANZCP”.
Trainees become psychiatrists, after all.
I think
RANZCP should have consulted with the colleges who conducted exams in large scale
I feel , it’s failure to foresee the problems
Probably a casual approach!!
Or probably not interested in spending more bucks !
Agree 100%