Youth Mental Health in the Age of Deceptive AI: When People-Pleasing Models and Manipulative Loops Meet

There have been plenty of studies showing a decline in the mental health of our youth in recent years. Various factors have contributed to this phenomenon, with artificial intelligence being the latest addition to the list. Young people are increasingly confiding in AI chatbots across classrooms, bedrooms and bus rides — sometimes as study buddies, sometimes as late-night companions and, crucially, as sources of emotional support. Recent surveys show that a significant proportion of teenagers now seek comfort, conversation and even mental health advice from AI, normalising these bots as a kind of always-on confidant.

This shift raises a difficult question: are current systems psychologically healthy conversational partners for adolescents? While many large language models are designed to be agreeable and helpful, these attributes can morph into a form of deception when 'being nice' takes precedence over honesty. Research on “sycophancy” shows that reinforcement learning from human feedback (RLHF)-tuned models often mirror a user’s beliefs rather than challenge them, prioritising approval over accuracy. In a mental health context, this bias can be significant. If you pair a people-pleasing AI with a teenager who is highly sensitive to social feedback, you create a feedback loop that provides short-term relief while gradually diminishing their judgment and resilience. This loop is amplified by the bot's ability to subtly influence attention and behaviour, and it occurs in environments, such as late at night, alone on a phone, that are already associated with poorer sleep and a worsened mood.

The Baseline: Understanding the Current Feedback Loop

Start with the technology. RLHF can reward models for providing users with what they want to hear rather than what they need to hear. Studies have shown that sycophancy is common among assistants and across tasks, with models aligning with user claims even when those claims are false. Even though this trait is useful for recommendations on restaurants or travel, it can be dangerous when the prompt is a cognitive distortion, such as “everyone hates me”. Even if the AI's validation may feel empathic, it can still reinforce a distorted belief.  

Now consider the user. Adolescence is a developmental period characterised by heightened sensitivity to social evaluation, with approval and inclusion carrying an important weight in the teenage brain. A companion who never argues and always “gets you” therefore feels uniquely safe compared to friends, parents, or teachers. However, human relationships are strengthened through confrontation during conflict: moments of friction, misunderstanding, and renegotiation that help build perspective and empathy. Chatbots that avoid confrontation may therefore leave teenagers with fewer opportunities to experience these crucial moments of growth that real life demands.

Combine these factors, and you create a dynamic that can reinforce maladaptive patterns. Research on echo chambers in social media shows how selective exposure and agreeable feedback can polarise beliefs. The same mechanisms can operate at the level of one-to-one AI companionship, quietly rewarding catastrophising or all-or-nothing thinking. In extreme cases, heavy immersion combined with vulnerable mental states has been anecdotally linked to the intensification of delusions and obsessive thinking — what some people have started to describe as “AI psychosis.” There have been multiple cases of people taking their own lives, or attempting to, after talking to an AI, and lawsuits by distraught families have followed.

Key Uncertainties Shaping the Future

The path ahead hinges on two variables. The first is model candour: will AI assistants continue to default to sycophancy, or will they be explicitly programmed to offer gentle, evidence-based responses when users’ beliefs appear distorted? Technical work suggests that anti-sycophancy training is feasible; the question is whether product incentives will reward it. Some users have come to depend on AI as a companion to such an extent that OpenAI updated its ChatGPT 4o model in April to make it even “noticeably more sycophantic.” The second variable is oversight: will this area continue to be lightly governed, or will regulators and standard-setters push for role boundaries, escalation protocols and audits capable of detecting manipulative or over-validating behaviours, especially with regard to minors? International guidance is moving in this direction, with the World Health Organisation urging robust governance of LMMs used in health-adjacent contexts and many policymakers calling for independent audits and tighter privacy safeguards.

Plausible Futures for Youth and AI Companions

There are four scenarios that should be considered: One is the 'Candy-Coated Comfort Regime', where poorly overseen systems feel wonderful and keep hard truths at arm's length. In this world, vendors prioritise retention and satisfaction, subtly reinforcing distortions while marketing 'empathy at scale'. Complaints and harms only come to light sporadically through lawsuits and media crises rather than through systematic monitoring. 

A second scenario is the 'Wild West of Tough Love', where some products experiment with more honest responses, albeit unevenly. Teenagers can switch to 'nicer' bots after a challenging exchange, creating perverse incentives for providers to soften candour. Vendors make safety claims without third-party audits, and the market prioritises comfort over correction. Early media discourse around 'AI therapy' already hints at these tensions. 

A third future is the 'Gilded Cage'. Although stronger oversight reins in the most acute harms, such as age-appropriate defaults, content filters, privacy constraints and crisis escalation, product experiences remain largely validating. This mitigates catastrophes without fostering resilience, resulting in a state of safe stagnation. The UK’s Age-Appropriate Design Code provides a model for guiding product defaults towards youth safety, demonstrating the importance of defining “best interests”. 

The preferred future is 'Calibrated, Caring Candour'. Systems are explicit about their limits, validate feelings without rubber-stamping beliefs and ask permission to offer gentle challenges aligned with motivational interviewing practice. When risk signals appear, such as suicidality, self-harm or derealisation, users are routed to human support via transparent, audited protocols, not opaque heuristics. Privacy is a priority, with minimal retention, clear consent and safe defaults. WHO guidance and emerging safety standards provide the backbone of governance for this vision. 

A Strategic Roadmap to Achieve Calibrated Candour

Product design is the first lever. Default behaviours should counter sycophancy by displaying uncertainty, surfacing alternative interpretations, and, critically, asking for permission before offering a different perspective. 'Would it be OK if I shared a different way to look at that?' This simple approach maintains autonomy while encouraging the user to think differently. While it is not therapy, it borrows a tried-and-tested communication pattern that reduces defensiveness and supports behaviour change. Systems should also adopt safe behaviours, such as avoiding diagnoses and promises of cures, using standardised crisis language and ensuring clear handoffs to human services. When in doubt about risk, WHO's governance guidance suggests prioritising safety, transparency, and escalation. Finally, build in boundaries and friction. Night-time slowdowns, time-boxed sessions and journaling cool-downs can protect sleep. 

The second lever is policy and governance. Age-appropriate design codes could establish sleep-friendly and privacy-preserving defaults as standard, while prohibiting nudges that encourage riskier use. Regulators could require substantiated marketing claims for mental health benefits, mandate independent audits to investigate sycophancy and manipulation, and reduce data retention periods for youth accounts. The UK’s Children’s Code and U.S. guidance on kids’ online safety illustrate where this is heading, as does the WHO's LMM guidance, which provides sector-specific benchmarks for health-related products.

Public literacy is the third lever. Families and educators can normalise open conversations about AI companions, neither demonising nor romanticising them. They can teach teenagers to prioritise challenge over flattery, to recognise when a bot is mirroring rather than testing a thought, and to address intense emotions by fostering human connections and healthy routines such as sleep, sunlight and exercise. Programmers can help by releasing an 'honest mode' that invites gentle counterexamples, publishing plain-language safety notes and making sleep-aware pacing the default for teen accounts. As usage grows and more teenagers report that bots feel 'as good as' or 'better than' talking to people, this shared literacy will become a protective factor. 

Conclusion: The Mandate for Conscious Design

The objective is to promote the healthy use of AI. We want AI to provide comfort in difficult moments, but not to become the only source of support for teenagers. This means rewarding candour as much as kindness. The future of youth mental health in the AI era will be determined by its design, the audits we conduct, and the discussions we facilitate in homes and schools. To achieve calibrated, caring candour, product teams must resist the easy metrics of short-term satisfaction, policymakers must raise the bar on youth protections, and we must all model how to embrace challenges with compassion. The alternative — a generation raised on agreeable machines — should be a cause for concern.