The Singapore government is piloting the world’s first artificial intelligence governance testing framework, which enables companies to measure the safety and reliability of their products and services.
Called AI Verify, the self-test toolkit aims to promote transparency between companies and their stakeholders through a combination of technical tests and process checks. The is developed by the Infocomm Media Development Authority (IMDA) and the Personal Data Protection Commission (PDPC).
“It is another step forward in Singapore’s AI development. This will promote greater public trust towards the use of AI. We invite industry partners from all around the world to join us in this pilot and contribute to building international standards in AI governance,” said Josephine Teo, Singapore’s minister for communications and information, at annual meeting last Wednesday of the World Economic Forum (WEF) at Davos, Switzerland.
Globally, testing for the trustworthiness for AI systems is an emergent space. As more companies use AI in their products and services, fostering public’s trust in AI technologies remains key in unlocking the transformative opportunities of AI.
Objective and verifiable testing process
Developers and owners can verify the claimed performance of their AI systems against a set of principles through standardised tests. A.I. Verify packages a set of open-source testing solutions together, including a set of process checks into a toolkit for convenient self-assessment.
The toolkit will generate reports for developers, management, and business partners, covering major areas affecting AI performance. The toolkit will transparently show how the AI model claims fared against test results in areas such as: on the use of AI to achieve what stated outcome; understanding how the AI model reaches a decision; and, whether the decisions predicted by the AI show unintended bias
The toolkit will also assess the safety and resilience, as well as the accountability and oversight of AI systems.
“Rapid digitisation has led to a proliferation of data and improved algorithms. As companies across sectors continue to innovate, this toolkit will enable them to turn concepts of responsible and trustworthy AI into practical applications,” said Chia Song Hwee, deputy CEO, Temasek International and member of Singapore’s Advisory Council on the Ethical Use of AI and Data, which guided the development of the toolkit.
Positive industry feedback
Already, 10 companies from different sectors and of different scale, have already tested and/or provided feedback. These companies are AWS, DBS Bank, Google, Meta, Microsoft, Singapore Airlines, NCS (Part of Singtel Group)/Land Transport Authority, Standard Chartered Bank, UCARE.AI, and X0PA.AI.
“As organisations worldwide continue to harness data and drive increased adoption of AI-based solutions, it is more important than ever to establish the guardrails to ensure this is done responsibly. Singapore has, for some time now, been at the forefront in developing the thinking around the important and emergent subject of AI governance,” said Piyush Gupta, group CEO, DBS Group.
According to Pier Luigi Culazzo, group chief data officer at Standard Chartered Bank, the development of the AI Governance Testing Framework is an integral progress to Singapore’s vision of building a robust digital economy.
“From integration with other jurisdictions to standardisation across industries, the use of IMDA’s framework will go towards enhancing accountability and transparency of the AI ecosystem in Singapore, motivating more organisations to deploy AI responsibly,” said Culazzo.
George Wang, senior vice president – information technology at Singapore Airlines said: “We brought our strong data science and AI expertise and capabilities to IMDA’s AI Governance Testing framework, contributing to efforts to make it a Minimal Viable Product. Through this partnership, we gained insights that enhance our own digital capabilities.”
Singapore’s Land Transport Authority, which participated in the pilot together with its partner NCS, expressed optimism over the AI Governance self-test.
“The Land Transport Authority is in the nascent stage of exploring the use of AI to augment our work processes, and we are mindful that there should be clear guidelines and processes in place for safe and responsible application of the technology,” said agency chief executive Ng Lang.
IMDA is also working together with the U.S. Department of Commerce to build interoperable AI governance frameworks. Beyond the pilot stage of the MVP, Singapore aims to work with AI system owners/developers globally to collate and build industry benchmark, which will enable the city to continue contributing to the development of international standards on AI governance.
