RANZCP’s exam failure a chance to rebuild trust

ON Saturday, 20 November 2021, the Royal Australian and New Zealand College of Psychiatrists (RANZCP) experienced a devastating examination failure.

Across Australia and New Zealand, 248 senior psychiatry trainees and specialist international medical graduates (SIMGs) were sitting at their home computers, sweating into their suits but ready to attempt an audio-visual version of the Objective Structured Clinical Exam (OSCE). Elsewhere, actors and volunteer psychiatrist examiners were ready and waiting to begin.

Many candidates, especially those in New South Wales who had their July 2021 OSCE cancelled at the last minute due to the lack of a COVID-19 outbreak contingency plan, had been waiting since early 2020 for an opportunity to sit the exam and complete their training.

Then, disaster struck.

After a week of fraught technical practice sessions, which had already distressed and distracted the candidates in the lead-up to the exam, the platform failed spectacularly. A small portion of the morning session’s candidates were able to partially complete the examination. The afternoon candidates received urgent emails from the College advising their session would still go ahead, but about an hour into the scheduled registration time, the College made the decision to abort.

To the candidates’ dismay, the College had seemingly failed to develop a contingency plan, and at time of writing they were scrambling to find a remedy. Scores of letters from trainees, psychiatrists and various committees – many of them shared on social media -- have been pouring into the President’s inbox. Candidates are advocating for alternative assessment pathways, noting the abundance of pre-existing workplace-based assessments and centrally administered examination results from which to draw.

A binational survey of candidates was rapidly deployed by the New South Wales Association of Psychiatry Trainees (NAPT) and yielded alarming results. Of the 206 respondents, 53% said they had already completed their training time or would do so within 3 months. For 88% of respondents, this was not the first time the College had either cancelled an exam during the pandemic, or told them they were ineligible to sit on account of limited spots and not being deemed a “priority” candidate.

RANZCP has struggled to run its OSCE throughout the COVID-19 pandemic. According to College reports (available only to RANZCP members), only 207 registrars have had the opportunity to sit the OSCE during 2020 and 2021. By comparison, 680 registrars sat the OSCE during 2018 and 2019.

Most concerningly of all, the free text responses to NAPT’s survey ran to 25 printed pages and contained a litany of horrific stories about the impact of the exam failure on the candidates and their families, after months or years of study.

“I feel trapped and unsupported,” wrote one candidate.

“I’m a single mother. My child needs me,” wrote another.

SIMGs spoke of the added stress of not being eligible for permanent residency until they attain Fellowship, and how as a condition of their SIMG status they were prohibited from working part-time or leaving the country to visit loved ones overseas.

Relationship stress was commonly mentioned, as was the deferral of on-call shifts until after the examination date, meaning it would be nigh on impossible to prepare for any reschedule.

“Let me finish my training. Please,” begged one registrar.

What went wrong

The exact technical failures are yet to be revealed; however, other colleges, including the Royal Australian College of General Practitioners and the Royal College of Psychiatrists in the UK, have successfully delivered similar clinical exams during the pandemic. Ultimately, the problem appears to relate to poor project management and a failure to prepare a contingency plan.

The first sign of trouble was in early September 2021, when candidates were informed the OSCE would be postponed from October to November on account of technical difficulties. NAPT wrote to the College on 31 October with a further request that contingency plans be developed.

In the week before the exam, it became clear to candidates that the postponement had not been particularly fruitful. In the belief that participating in testing would help familiarise them with the platform, candidates logged on to a series of technical run-throughs which proved alarming. There were reports from those familiar with information technology and project management that basic load and user acceptance testing appeared not to have been performed. Some candidates took time away from study to propose back-up plans, including a simplified process using only the Zoom platform, with slightly longer breaks between stations, and with invigilators sharing OSCE stems via Zoom’s screen-sharing function.

Hierarchy and culture

In my first year of psychiatry training, I wrote in InSight+ about the harms to trainees and patients of excessive hierarchy in medicine. This latest examination failure has occurred in this milieu.

The RANZCP has a two-tier membership structure. Fellows have voting rights and seven of them are elected to form the College’s Board of Directors. The Board is responsible for appointing — and, when necessary, firing — the Chief Executive Officer and ensuring the College performs its core functions, which include graduating the next generation of psychiatrists and promoting mental health.

Trainees and SIMGs, however, are classed as “associate” members. This category is a subclass, devoid of voting rights. It is a case of taxation without representation. According to the College’s latest annual review, there were 1822 associate members out of 7023 total members; that is, 26% of the College membership. Yet trainees have no vote, and no seat on the Board – unlike our physician and emergency medicine peers.

Collectively, the membership fees of associate members contribute about $2.4–$3.4 million of the College’s $42 million annual operating revenue. In addition, trainees are charged joining ($702) and examination fees ($425–$2658 for each of six assessments). Given there were 248 candidates for last Saturday’s exam, that single OSCE provided the College with $659 000.

The RANZCP does have a Trainee Representative Committee (TRC), but members are forced to sign a “Deed of Undertaking in Relation to Confidential Information and Conflict of Interest” (and here). TRC members commonly refer to the Deed as a “non-disclosure agreement”, and it has the effect of inhibiting their ability to communicate with the broader trainee cohort. The co-Chairs of the TRC have since announced their resignation, effective from 7 December.

The feeling among trainees, especially after this OSCE exam failure, is one of disenfranchisement.

Where to from here

Acutely, the College needs to find an acceptable means of allowing candidates to progress in their training and start serving the community as psychiatrists. We are cautiously optimistic that this will occur, with statements from the College that an alternative assessment pathway is being developed. It is pleasing that they have listened to candidates and their many supporters that any attempt to repeat the OSCE would not be viable.

The bigger issue is one of rebuilding the relationship between the College and trainees. However, the Board could take two immediate decisions that would help enormously.

The first is to exercise a provision within section 9.1 of the Constitution to appoint two trainees or SIMGs to the Board.

The second is to tear up the TRC’s “non-disclosure agreements” and commit to meaningful and transparent engagement with all trainees – a shift from paternalism to inclusion: nothing about us, without us.

From there, relevant staff and office bearers will need to take accountability for their persistent failure to deliver examinations, which may include terminations and resignations. Trainees should be reimbursed not only their OSCE fees but also, in some instances, specific costs incurred, such as headsets, larger monitors and increased bandwidth they were advised to purchase.

Constitutional change needs to be pursued to give trainees and SIMGs voting rights and a proportional quota on the Board, and the College should finally commit to the implementation of all recommendations of the 2019 Australian Council for Educational Research review of the RANZCP’s examinations.

The alternative is for the College to continue “business as usual”. The risk of this, however, is a long term rupture in the trust of hundreds of soon-to-be Fellows in the institution itself. Given the reliance of all specialist medical colleges on their members to write exams and policy and govern the organisation, choosing this path would be short-sighted and self-defeating.

In psychiatry, we speak of recovery-oriented and trauma-informed practice, both of which involve genuine collaboration and empathy. Well, here is the College leadership’s opportunity to walk the talk.

Benjamin Veness is the Advocacy Officer for the Victorian Association of Psychiatry Trainees. He was not a candidate in last Saturday’s OSCE. Twitter @venessb

 

 

The statements or opinions expressed in this article reflect the views of the authors and do not represent the official policy of the AMA, the MJA or InSight+ unless so stated.