Top scientific minds gathered for the first time in Singapore to advance AI that is trustworthy, reliable and secure

01. As part of Singapore AI Research Week, the “Singapore Conference on AI: International Scientific Exchange on AI Safety” (SCAI: ISE) took place on 26th April 2025, organised on the sidelines of the International Conference on Learning Representations (ICLR) 2025 held in Singapore this year for the first time. Luminaries in the field of AI safety research from 11 countries (Australia, Canada, Chile, China, France, Japan, Korea, Netherlands, UK, US and Singapore), including academics, industry players, government representatives, think tanks and policy makers, gathered for the conference to discuss the importance of AI safety research and the need for an international consensus, resulting in the publication of “The Singapore Consensus on Global AI Safety Research Priorities” (The Singapore Consensus).

2. Singapore has always embraced new technologies by being prepared. In the area of AI, Singapore has consistently taken a globally collaborative approach in understanding how to make AI trustworthy, reliable and secure. With the advancement of AI models and its increasing use in different sectors, there is a greater need to develop common evaluation benchmarks, methodologies and tools to manage the inherent risks that such models may bring. The SCAI:ISE is the latest milestone step to do just that by convening over 100 participants (such as Yoshua Bengio and Max Tegmark) from 11 countries in the field of AI safety research to identify, prioritise and arrive at a consensus on what needs to be researched on. Through SCAI:ISE, the “The Singapore Consensus” identified the following 3 broad areas of AI safety research priorities.

• Risk Assessment: The primary goal of risk assessment is to understand the severity and likelihood of a potential harm, which can then help prioritise risks and determine whether action needs to be taken. The research areas in this category involve developing methods to measure the impact of AI systems for both current and future AI, enhancing metrology to ensure that these measurements are precise and repeatable, and building enablers for thirdparty audits to support independent validation of these risk assessments.
• Development: AI systems that are trustworthy, reliable and secure by design give people the confidence to embrace and adopt AI innovation. Following a classic safety engineering framework, the research areas in this category involves specifying the desired behaviour, designing a system that meets the specification, and verifying that the AI system meet its specification.
• Control: In engineering, “control” usually refers to the process of managing a system’s behaviour to achieve a desired outcome, even when faced with disturbances or uncertainties, and often in a feedback loop. The research areas in this category involve developing monitoring and intervention mechanisms for AI systems, extending monitoring mechanisms to the broader AI ecosystem to which the AI system belongs, and societal resilience research to strengthen societal infrastructure (e,g, economic, security) against AI-enabled disruption and misuse.

3. Singapore remains committed to a scientifically driven and evidence-based approach to AI governance. This is essential to building a trustworthy, reliable and secure AI ecosystem where there are sufficient guardrails to protect people, while providing maximal space for innovation. In her opening remarks, Minister for Digital Development and Information, Mrs Josephine Teo, shared that it is important to build stronger pathways between the research world and policy making world to translate AI safety research into real effective policies to govern AI well. Through this scientific exchange, “The Singapore Consensus” aims to bridge discussions between AI scientists and AI policy makers, as part of a continuous dialogue to help governments strike the delicate balance of safety and innovation based on scientific evidence. To influence policy-making, the “The Singapore Consensus” will be presented to digital ministers attending the ministerial roundtable at the upcoming Asia Tech x Singapore Summit (ATxSummit) from 28-29 May 2025. The end-goal is to create a virtuous cycle of trust and, more importantly, ensure that AI is harnessed for the public good.

4. Internationally, this scientific exchange is the latest in a series of milestone steps in building a trusted AI ecosystem. The AI Verify Foundation (AIVF) previously launched two toolkits, the AI Verify Testing Toolkit and Project Moonshot, to test traditional and Gen AI models respectively to help developers, compliance teams, and AI system owners manage deployment risks. Singapore’s IMDA along with AIVF also released the Model AI Governance Framework for Gen AI in 2024 that provided a systematic and balanced approach to address Gen AI concerns while facilitating innovation. In November and December 2024, IMDA, in partnership with Humane Intelligence, conducted the world’s first multicultural and multilingual red-teaming exercise and published the “Singapore AI Safety Red-Teaming Challenge Evaluation Report” on it. AIVF also launched the “Global AI Assurance Pilot” earlier this year to help codify emerging norms and best practices around technical testing of Gen AI applications.