more_vert
PRIVATE coaching is unlikely to boost prospective medical students’ chances of being accepted into medical programs, say researchers who warn that students may be wasting their time and money on coaching.
In a short research report published this week in the MJA, researchers evaluated the results of applicants to Western Sydney University in 2014. (1)
They found no statistical difference between coached (n = 122) and non-coached (n = 222) interviewees on any of the three Undergraduate Medical and Health Sciences Admissions Test (UMAT) scores or Multiple Mini Interview scores.
Although the coached group had higher academic university entrance ranking (ATAR) scores, they were no more likely to be accepted into the medical program than students who had not been coached.
“Applicants who are likely to be shortlisted for interviews and yet feel the need for UMAT coaching may be less confident in their own ability or more susceptible to industry advertising — but the time and money spent on commercial coaching appears to be misdirected”, the researchers wrote.
Professor Richard Murray, dean of medicine and dentistry at James Cook University, said the findings provided some reassurance that those who wanted to “play by the rules” would not be disadvantaged.
“Debunking the myth that expensive private coaching will help [prospective students] on an admission test is important for the integrity of the selection process and for the wallets of prospective students and parents”, said Professor Murray, who is also vice-president of Medical Deans Australia and New Zealand.
“The industry being created around trying to gain entry [to medical programs] is not only inherently unfair — because it attempts to tip the scales towards those with resources — but it also doesn’t work.”
Lead author of the MJA article Associate Professor Barbara Griffin, director of organisational psychology at Macquarie University, said the coaching industry had made strong claims about its outcomes, but these findings added to evidence that coaching was largely ineffective.
“Universities would now be fairly comfortable that students are not being advantaged by doing coaching”, Professor Griffin told MJA InSight.
She said evidence of poorer academic performance among medical students who had been coached for UMAT compared with those who had not been coached indicated that any increase in scores due to coaching did not represent true ability, but “test wiseness”. This had, in part, prompted Western Sydney University to devise a metric to reduce the proportion of coached students being put forward for interviews.
Professor Griffin said this had seen the proportion of coached students being invited to interview at the university drop from 51.4% in 2008 to 35.5% in 2014.
She said students who performed well across all areas of UMAT had an equal chance of being selected for interview, whether or not they had been coached. “Research suggests, over a number of studies, that if someone does very well across all aspects of the UMAT then their performance is most unlikely to be due to coaching.”
James Lawler, president of the Australian Medical Students’ Association, said the research sent a clear message to prospective students that there was “little to no” benefit in seeking out commercial coaching.
“The lack of evidence for the effectiveness of these coaching courses means that the only winners are the business owners who are selling them”, he said.
Mr Lawler said coaching towards a test discounted other important attributes required in medical practice, such as emotional intelligence.
“With the vast number of students trying to gain entry to medicine, UMAT has a place in screening applicants”, Mr Lawler said. “But universities should focus more on their own interview processes and other procedures to find the best candidates for their degree than on UMAT.”
Professor Murray said the findings had a broader significance as medical schools developed innovative ways to select students who more closely reflected the community.
“If we have an admission process that overwhelmingly favours the sons and daughters of an affluent elite who are predominantly based in large cities — as we have done in the past — then we should not be surprised that the choices graduates make are not necessarily aligned to priority community needs”, he said.
“This is publicly funded medical education and it has to be administered in ways that are fair and transparent. It isn’t only about narrow forms of scholastic achievement or performance on tests; it has to be a broader metric.”
(Photo: Syda Productions / shutterstock)
Loading ...
Regarding interview training, if it does not work or unfair, why do universities train their graduates for job interviews?
They benefit by training graduates for job interviews so they can claim that higher proprtion of their graduates are employed.
It seems ‘unfairness’ only applies when universities are not benefiting from the training/coaching.
Richard Murray says: “The industry being created around trying to gain entry [to medical programs] is not only inherently unfair — because it attempts to tip the scales towards those with resources — but it also doesn’t work.”
This is contradictory to his claim that UMAT coaching does not work. If it does not work, why would it be unfair??
The research was funded by ACER, so there is fundamental conflict of interest. Its in the ACER and Universities’ interest to claim coaching (for UMAT/GAMSAT) does not work.
The analysis described does not prove that UMAT coaching has no effect on the probability of gaining medical school admission. More importantly, there is no evidence that selection by UMAT score results in improved medical school outcomes, so UMAT testing is of no proven benefit and is possibily discriminatory. UMAT is a game played by universities, but according to Richard Murray, all is OK provided that we all “play by the rules” (of the game).
Agree with most of the comments coming in. In many ways it is irrelevant whether coaching helps or not. The more important issue is the best, fairest & most efficient selection process. Certainly the current process fails at least on the latter 2 counts & I also doubt whether it is the best way of identifying those who might go on to become good doctors.
My daughter, almost 15 years ago, was thinking of studying Medicine & I think she would have made a stellar doctor. She easily exceeded the cut off VCE score for potential selection, performs well in interviews & has the empathic, caring, listening skills & manual dexterity. She didn’t get coaching & scored above the 90th percentile in logic & empathy but scored very low in the bottom 20th percentile in spatial recognition. The geniuses who designed UMAT average out the scores & then put applicants in a pecking order to qualify for an interview. Because of her low score in spatial recognition, she was 1 or 2 points below the cut off mark & therefore didn’t qualify for an interview at least at her chosen University. She may or may not have qualified for an interview at one of the other Universities, but she did not put others on her application list. The end result was that she was excluded from the interview & hence from being selected because she scored badly in spatial recognition. Since when does spatial recognition be the sole determinant of whether one should be selected to study Medicine or not? Truth is stranger than fiction but this was the case. When I phoned the University, they assured me they had the skill &wisdom in best deciding selection methods & after the interview, how clever they were at deciding who was best to study Medicine!
The question here should not be whether UMAT coaching is effective, or beneficial, or necessary.
Rather, the question should be whether UMAT itself is an effective and beneficial tool to select the next generation of doctors; and whether it is necessary. As someone who has been through the undergraduate medical selection process, my opinion is that the UMAT is an unnecessary, ineffective test, that is full of disadvantages.
A multiple choice test for logic, reasoning and so-called ’empathy/social skills/understanding’ that is section 2 is ludicrous. it is nothing more than a money-making scheme from ACER. Where are the studies showing that students who do well at the UMAT go on to be better, or even good doctors? In my anecdotal experience, students who have entered medicine with a relatively low ATAR and stellar UMAT score are overrepresented in dropping out of medicine or achieving poor results/failing years in the course. What about the fact that post graduate entry students never have to do well or even sit the UMAT- the GAMSAT is a different test of knowledge and reasoning. If the UMAT really was a good indicator, then it would suggest that post graduate medical students (many of whom may not have made it into undergraduate courses due to UMAT) are inferior, which is absolutely not the case. The UMAT has created a toxic culture, where high school students hoping to study medicine who are already working hard academically and extracurricularly, have another weight put on their shoulders- another unnecessary weight.
My daughter was rejected for undergrad medicine on the basis of her UMAT (no coaching). She studied Psychology and then was admitted to a Post Grad medical course (public university). She has passed her specialist Physician exams in her second year after internship, has been offered a fellowship in neurology and will be a first class caring Physician. Is she smart? – yes. Has she worked hard? – of course. Has the UMAT been a good measure – or any measure at all? – of course not. And that is precisely the reason the universities that utilise the UMAT will NOT release or allow any research into its use.
From my experience in both supervising both medical students and gp registrars I doubt the selection processses for both the undergraduate and postgraduate medical courses in Australia produce any better doctors than 30 years ago. However a whole industry has developed around coaching for entrance to medicine especially for those whose parents can afford it (whatever the motive). And by way for those who think rural kids have an advantage try leaving home at 18, go to a totally unfamiliar city, try to make friends with the ex private school city kids in your year , wash your own clothes, cook your own food and clean your own low quality rental accommodation, live on the poverty line and study.
Some facts 1. Rural students quota gives them more advantage over metro students 2. The sheer fact that the students coming out of the system today is of lesser quality than 1988 graduates is not because selection process but current teaching process. 3. For the same symptoms 10 tutors come up with 11 different diagnosis without proper analysis is not uncommon-no wonder students are confused 4. Fact is that these highly motivated students learn themselves and university plays, very minimal role on their development and quality, is the reason for poorer quality than that was in 1988. 5. Education is simply has become a business and to minimise cost they have reduced 6 years course to 5 yr course, now 5th year student are asked to teach in tutorial in many universities simply to reduce cost of teaching 6. Big prof would not leave their practice to teach-simply because not cost effective. List is long we all know how education system has been compromised with business model. So all deans and heads of medical colleges wake up and just fight for quality not to crucify commercial coaching. Because our education system does not protect quality therefore conscientious parents and students find other option for development.
Agree with Dr Paul Lonsdale: I’m concerned that the UMAT process may be professional engineering in the wrong direction(s). I don’t know enough about what they are looking for in the selection process in order to comment from an informed perspective, but am interested to know exactly what types of doctors they are trying to turn out.
Do they just want practising clinicians who conform to the current norms of the profession, except are also interested in rural practice? The obvious answer would be “yes”, but in my experience the current make-up of the profession often compromises patient satisfaction, innovation in favour of conformity, and cost-effectiveness from a Medicare perspective. Therefore, is it a worthy goal to perpetuate the current professional make-up? Will academic medicine / research be neglected by the UMAT graduates? Will it breed innovative clinicians?
Perhaps the most important “acid test” for the current selection process would be to put today’s respected and prominent clinicians through the selection process, alongside other prospectives. How would they do? Would they get into medicine? I don’t know if that has been done already, but would be very interested to see if we might lose some important figures because the selectors’ “crystal ball” has been poorly-engineered.
As has been said, it was not randomised, it was by definition self selected. The biggest question is whether those who narrowly miss out on consideration due to a lacklustre UMAT but have stellar academic records should be lost to the world of medicine.
I would like to share an observation as someone who has completed the medical admission progress in the recent years. The individuals convening UMAT preparation courses strongly advise their clients not to disclose their degree of preparation in questionaries after the test. Therefore, the advantage gained by undertaking a preparation course is likely to be significantly higher than it appears in the literature.
What I’m looking forward to is an assessment of whether the selection processes that have been employed over the last decade and the new medical courses have been successful in producing an improved cohort of doctors compared with my graduating class of 1988. From my vantage point as a tutor in the public hospital system and the private hospital system….I have not seen any difference. If anything, I have been a little disappointed by the groups coming through……
I totally agree that there is very minimal effect of coaching on interview outcome. That is best achieved by lifelong training and exposure to public speaking, lateral thinking ability, social empathy and judging ethical and moral dilemmas. However these can be enhanced in those who lacked the opportunities to be able to enjoy such environments for development.
In relation to ACER exam, I disagree with researchers strongly. Coaching certainly improves logical thinking ability and ability to solve non-verbal problems.
While only interview is not a solution, combination of ATAR, UMAT and Interview is a good approach. Many Medical appointments are favoured to students who have parents/relatives high up in medical fields.
Education is lifelong process and while crash UMAT coaching can help exposure to developing some important skills, it is not a perfect ticket for success. Why attack commercial coaching?
Many students who have done well did not depend on school alone went for extra tutoring.
So yes UMAT and coaching give some opportunities to disadvantaged student (even it is a short term coaching). At the end of the day most of the affluent families also avail themselves of these facilities.
I believe UMAT to be a major disadvantage for rural, Indigenous and lower socioeconomic prospective medical school applicants.
Millions of dollars are spent annually to entice and retain rural medical practitioners. There is good evidence that medical students from rural locations are far more likely to return to rural locations to practise medicine when they graduate. UMAT potentially creates a hurdle for these rural students when applying for medical school.
Currently the UMAT exam can only be sat on one day in specific locations. In Western Australia this is Perth. This makes it an extremely expensive logistical exercise for a student in the north west to attend. There is the travel time, travel cost, exam cost and potentially UMAT coaching course cost. This all for 17-year-old Year 12 students. This is potentially unsurmaountable for many rural and Indigenous students
In NSW, UMAT can be sat in Sydney, Newcastle, Dubbo or Armidale. In 2015, UMAT was scheduled in the middle of HSC trial exams for all Catholic secondary schools making it virtually impossible for a candidate to sit the exam unless they resided in one of these centres.
The situation is almost impossible for truly remote or Islander students.
I believe the current UMAT arrangements are discriminatory against rural, Indigenous and lower socioeconomic candidates and this situation must be corrected.
Dr Paul O’Brien MBBS, FRACGP
It was not randomised…