GPs need more evidence that digital mental health services work before prescribing them

In a recent press release, the Health Minister Mark Butler claimed that new digital services are the first step towards a better, fairer mental health system. The assertion that “help is just a click away” is attractive, but not necessarily accurate.

To use digital mental health services, consumers need the resources to access and understand them. Many digital mental health services require a high degree of literacy in English, technology and health to interpret and follow the content. In this systematic review of barriers to adoption of digital mental health services, the authors cite a series of structural, individual and professional barriers to the use of digital services. Consumers may not have access to the technology required, they may lack cognitive resources to make sense of complex concepts when they are unwell, they may find the resources inappropriate to their own cultural context, or they simply may not like them or find them helpful. “Digital poverty” is a form of exclusion, restricting access to resources others take for granted, including health care. People who lack the skills, resources or capacity to engage with digital health resources may well be the people with the greatest mental health needs.

Help may be “just a click away” for some, but is clearly not available to all.

Are digital mental health services effective?

The evidence for digital mental health services is often presented as overwhelmingly positive. One systematic review panel explored studies around mobile phone apps and concluded “we failed to find convincing evidence in support of any mobile phone-based intervention on any outcome”. There are a series of challenges in implementing the findings of these studies into clinical care, particularly in General Practice. These include methodological limitations, conflicts of interest and limited reporting of negative outcomes. However, the most important limitation of these studies is the representativeness of the population sample. The diversity of the study samples is rarely reported, but is likely to be narrower than the communities in which they will be implemented.

Drop-out rates with smartphone apps and other digital tools are high, so it is difficult to rely on the findings.

The evidence behind the National Digital Mental Health Framework

The National Digital Mental Health Framework claims that “Digital mental health services can improve access barriers for traditionally underserved cohorts by overcoming geographical and socio-economic barriers. They also provide consumers with the ability to exercise greater choice and control over when and where treatment will take place, presenting as a valuable option for consumers who are reluctant to use face-to-face services.” It supports this claim with only one reference, a surprisingly outdated paper looking at evidence from 1998 to 2009.

Later, the Framework references the National Safety and Quality Digital Mental Health Standards (2020) by the Australian Commission on Safety and Quality in Health Care. The Commission also references only one paper to support their claim that “there is growing evidence regarding the important role digital mental health services can play in the delivery of services to consumers, carers and families.” The paper cited is a more modern one (2016), but one with significant flaws. To illustrate the challenges of interpreting the digital mental health evidence, I have summarised the findings of the cited paper below.

The MindSpot study: an evaluation

The paper examined the MindSpot Clinic (an online digital mental health service). Here’s the attrition of their convenience sample of people visiting their website in the 30 months of the study. One interesting observation in this study is that 76% of participants who completed assessment were thought to be unsuitable for the MindSpot intervention.

 The team describes that “large clinical effects were found from assessment to follow-up on all outcome measures.” On this basis, they claim that their service “has considerable value as a complement to existing services, and is proving particularly important for improving access for people not using existing services.”

The 99.8% of people who visited the site and did not engage remain unresearched. We do not know whether they were harmed or whether they recovered more quickly or more effectively than the cohort who completed the study. Using a paper like this to claim that digital mental health services have “considerable value as a complement to existing services, and is proving particularly important for improving access for people not using existing services” seems an inflation of their research outcomes.

Next steps

It is clear that digital mental health services play an important role in future health service design. However, if GPs are to prescribe them, we need to have more than persuasive tactics at our disposal.

The evidence-based medicine triad [AH3] outlines the three conditions under which GPs should prescribe an intervention:

• We need sufficient research evidence.
• We need to form a professional opinion that the intervention is appropriate in a particular clinical context.
• The patient needs to decide that they are willing to accept this form of care.

In the case of digital mental health interventions, I need three conditions to overcome my reluctance to prescribe. I need independent research, and Therapeutic Goods Administration endorsement through application of an appropriate digital mental health standard. I need to know enough about the demographics of the cohort studied to be able to decide whether the research applies to my patient and their situation, and the patient needs to decide that the intervention is right for them. Without this fundamental information, we cannot form a valid professional opinion about the appropriate use of any app, helpline, artificial intelligence program or digital course.

Early in my career, I was trained to analyse the science beneath any drug or device, and avoid the influence of pharmaceutical representatives with vested interests persuading me to prescribe their intervention of choice. I am unlikely to change my professional values and acquiesce to the marketing of new interventions now, no matter how hard funders and managers attempt to drive me to adopt these models of care. I am satisfied that this is the unavoidable consequence of exercising appropriate clinical judgement.

It is my duty to form my own professional judgements and my patients rely on my professional integrity to reject marketing pressure. If anyone misinterprets GP conduct as reluctance, rather than conscientiousness, they are underestimating our commitment to good practice.

We have seen marketing behaviour before with a variety of drugs and devices. We have not forgotten the marketing behind “non-addictive” opiates, or the damage caused by regulated medical devices. We are understandably wary when claims of efficacy outstrip evidence. Our patients deserve better.

Dr Louise Stone is a Canberra GP with clinical, research, teaching and policy expertise in mental health. She is an associate professor in the social foundations of medicine group, Australian National University Medical School.

Read part 1: Are GPs 'reluctant' to prescribe digital mental health services?

Read Part 2: Are digital mental health services cost-effective and safe?