Anthropic has launched a $5 million grant programme to support independent research examining how artificial intelligence systems affect users’ wellbeing, as the company seeks to encourage the development of open-source evaluations and benchmarks for AI interactions.
The programme will provide selected researchers with direct funding, access to Anthropic’s AI models and technical support to develop open-source tools that can help the wider AI industry assess the impact of AI systems on users’ wellbeing.
According to Anthropic, grantees will operate independently and publish their research as open-source projects, allowing developers and researchers across the industry to use the resulting evaluations.
The initiative comes as AI systems increasingly become part of how people work, learn and solve problems, while also being used as conversational partners and sources of emotional support. Anthropic said the industry still lacks clear standards for evaluating how AI models should behave in sensitive interactions, including when users seek companionship from AI or turn to models while experiencing a mental health crisis.
Anthropic said evaluating AI’s impact on wellbeing is more difficult than assessing individual model responses for accuracy or appropriateness. The potential risks may only become apparent over the course of a conversation, particularly when a user’s circumstances or emotional state changes.
For example, a user experiencing distress may not initially disclose thoughts of self-harm, meaning that the need for a more cautious response could emerge only after a longer interaction. Similarly, advice that appears appropriate in isolation may become unsuitable when considered alongside a user’s previous conversations or circumstances.
The company cited weight-loss conversations as an example. General advice about balanced diets and exercise may be appropriate for many users, but could be potentially harmful for someone who has demonstrated a history of disordered eating.
Anthropic said it is already developing safeguards designed to identify sensitive conversations and help ensure its Claude AI assistant responds appropriately. The company also conducts and publishes research examining how people use Claude for personal guidance, support and other sensitive interactions.
However, Anthropic acknowledged that these considerations are nuanced and that approaches to protecting user wellbeing will need to evolve alongside AI models and their applications.
Through the new grant programme, Anthropic aims to broaden research into AI and wellbeing by bringing in expertise from clinicians, psychologists, research methodologists and other specialists.
The company said independent evaluations and benchmarks could help establish more consistent ways of measuring AI’s effects on users and provide tools that developers across the industry can use to assess model behaviour.
By making the resulting projects open source and allowing grantees to work independently, Anthropic aims to support a broader research ecosystem around AI wellbeing rather than relying solely on evaluations developed internally.
The initiative reflects growing attention within the AI industry to the potential social and psychological effects of increasingly capable conversational AI systems and the need for robust, independent methods to evaluate their impact on users.
