Mind Launches Inquiry into Google’s Risky AI Mental Health Advice

A year-long investigation into the impact of artificial intelligence on mental health has been initiated by the mental health charity, Mind. This follows a report by The Guardian, which revealed that Google’s AI-generated summaries, known as AI Overviews, provide potentially harmful advice to the public. These summaries are viewed by approximately 2 billion users each month, raising significant concerns about their accuracy and the risks they pose to individuals seeking mental health information.

Rosie Weatherley, the information content manager at Mind, expressed serious concerns regarding the AI Overviews. She pointed out that Google has historically designed its search engine to prioritize credible health content. “Over three decades, Google designed and delivered a search engine where credible and accessible health content could rise to the top of the results,” Weatherley stated. She emphasized that while online searches for health information were not flawless, users generally found reliable sources that addressed their queries.

The introduction of AI Overviews has altered this landscape. According to Weatherley, these summaries replace rich, nuanced information with simplified, clinical-sounding responses that can mislead users. “It’s a very seductive swap, but not a responsible one,” she asserted. Many users end their search after encountering these AI-generated summaries, often leaving them with incomplete or misleading information.

In an effort to illustrate the potential dangers, Weatherley and her team conducted a brief experiment. They searched for common queries related to mental health, expecting to spend about 20 minutes on the task. Surprisingly, Weatherley found alarming results in under two minutes. The AI Overviews suggested that starvation could be healthy, claimed that mental health issues stem from chemical imbalances in the brain, and even stated that 60% of benefit claims for mental health conditions are fraudulent. None of these statements are accurate or supported by credible research.

“AI Overviews are flattening information about highly sensitive and nuanced areas into neat answers,” Weatherley explained. This approach removes essential context, potentially making harmful misinformation appear plausible. The risks are particularly pronounced for individuals who may already be experiencing distress, as they might rely on inaccurate information for guidance.

Weatherley criticized Google’s approach to addressing these issues. She stated that the tech giant should allocate more resources to ensure accurate information is available, rather than reactively adjusting AI Overviews only after problems are flagged by users or journalists. This “whack-a-mole” strategy, according to her, falls short of what is needed given the scale and resources of a multi-billion-dollar company.

To combat harmful search results, Google has made efforts to limit access to dangerous content, such as methods of self-harm. Nevertheless, Weatherley warned that users searching for mental health information may still encounter misleading content presented as factual and authoritative. “The AI Overview haphazardly collaged various contradictory signposts in long lists,” she noted.

Weatherley recognizes the potential benefits of AI in improving lives but remains cautious. “Right now, the risks are really worrying,” she stated. Users in distress deserve access to care that is empathetic, informative, and nuanced, rather than relying on AI-generated summaries that may lead to harmful misconceptions. As the investigation by Mind progresses, the emphasis will remain on ensuring that accurate, sensitive, and responsible mental health information is readily available to those who need it most.