Require AI systems above a capability threshold to be interpretable

ai-ethics transparency ai-risk ai-governance ai-regulation ai-safety ai-policy

Share this with your representatives, journalists and friends

Arguments for

Accountability in consequential use

Where AI affects fundamental rights, public services, hiring, credit, or other consequential areas, proponents say black-box systems are not acceptable. Interpretable outputs make decisions comprehensible and contestable, help trace responsibility across the AI lifecycle, and preserve human accountability and redress.

AI [must] be transparent in automated decision-making, to ensure “the chain of accountability tracing responsibility from developers to deployers at every AI lifecycle stage.”

Archbishop Ettore Balestrero · Jul 7, 2026 · full quote
Where AI affects fundamental rights or the delivery of public services, meaningful human oversight, transparency, and effective mechanisms for accountability and redress must remain central.

Government of Lesotho · Jul 6, 2026 · full quote
We don't treat AI as a decision-maker. With structure, AI creates an explainable signal that teams can trust. [...] If AI can't explain itself, it doesn't belong in hiring.

Meredith Johnson · Apr 17, 2026 · full quote
The national AI policy focuses on the creation of explainable AI systems that deliver clear, interpretable outputs. This is important for trust and acceptance by users, especially in high-risk applications such as credit scoring. By making decisions by AI systems comprehensible a…

Government of South Africa · Apr 10, 2026 · full quote
For systemically important infrastructures, black-box decision-making is not an option. Accountability must always remain with humans.

Klaus Löber · Mar 3, 2026 · full quote

Control of powerful systems

As AI becomes more capable or agentic, supporters argue that understandability must keep pace with capability. They present interpretability as a safety tool for monitoring reasoning, spotting deception or goal misalignment, understanding failures, and avoiding deployment of systems whose internal operation is poorly understood.

Model assessments should include rigorous scientific evaluations of capabilities in cybersecurity, biological threats and other high-risk domains. Specific agentic AI tests could look for attempts to bypass safety guardrails or signs of deception, and ensure best practices, such …

Demis Hassabis · Jul 14, 2026 · full quote
We don’t need perfect understanding. But we do need to ensure that understandability keeps pace with capability, and to give high priority to building insight into how these systems work, how they fail, and how they can be aligned with human needs and flourishing.

Eric Horvitz · Jun 4, 2026 · full quote
I am very concerned about deploying such systems without a better handle on interpretability. These systems will be absolutely central to the economy, technology, and national security, and will be capable of so much autonomy that I consider it basically unacceptable for humanity…

Dario Amodei · 2025 · full quote
“If we pursue [our current approach], then we will eventually lose control over the machines. But, we can take a different route that actually leads to AI systems that are beneficial to humans,” said Russell. “We could, in fact, have a better civilization.” “You should not deplo…

Stuart J. Russell · 2023 · full quote
Today, we find that models’ reasoning is generally interpretable and easy to monitor. However, in the future, monitorability may break down for a variety of reasons.

OpenAI · Mar 5, 2026 · full quote

Auditability and oversight

Supporters frame interpretability as part of a governance program in which systems are inspectable, auditable, documented, and independently tested throughout their lifecycle. On this view, clear logic and traceability let organizations justify actions to auditors, boards, customers, and regulators instead of merely asserting trust.

We are deploying AI we can't explain, defending AI we can't inspect, and trusting AI we can't audit. That is not a governance program. That is a liability surface.

Eric Zielinski · Jul 14, 2026 · full quote
Both must rest on independent third-party oversight, trust verified by third parties, not asserted by the developers who profit from them.

Sam Gregory · Jul 7, 2026 · full quote
Trust must be demonstrated through practical verifiability, transparent governance, data policies, and independent testing of high-risk applications throughout their life cycle.

Government of Latvia · Jul 7, 2026 · full quote
Organisations should require interpretable outputs and documented logic.

Institute of Chartered Accountants in England and Wales · Jul 3, 2026 · full quote
When AI influences analysis that informs internal or external communications, the RBA expects clear disclosure and the maintenance of documentation that supports explainability and traceability.

Reserve Bank of Australia · May 12, 2026 · full quote
Every action and every outcome needs to be clarified and justified to a board, an auditor, or a customer.

Rodrigo Coelho · Aug 6, 2026 · full quote

Accessible explanations build trust

Another line of support emphasizes disclosure and accessible explanation for users, affected people, and organizations using AI. People should know when AI is being used and be able to understand how and why it contributes to an outcome, which supporters tie to trust, acceptance, and responsible use.

There should be transparency and responsible disclosure so people can understand when they are being significantly impacted by AI and can find out when an AI system is engaging with them.

Net Zero Economy Authority · Jul 10, 2026 · full quote
You should know that an automated system is being used and understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automated systems should provide generally accessible plain language documentation including clear descriptions of…

White House Office of Science and Technology Policy (OSTP) · 2022 · full quote
Finally, explainability, one of the thorniest and most technically complicated issues we face, but perhaps the most important of all. Explainability is about transparency. When you ask an AI system a question and it gives you an answer, perhaps the answer you weren’t expecting, y…

Chuck Schumer · Jun 21, 2023 · full quote
Private actors, for their part, need a transparent framework and information for their use of AI.

Government of France · Jul 7, 2026 · full quote
Explainable and observable AI are key parts of our Responsible AI principles at Snowflake, and it needs to be a governing principle for any organisation that hopes to reap the benefits of AI. Compliance, legal, and business teams need to be able to verify AI behavior just as rigo…

James Hall · Jun 18, 2026 · full quote
In ISO, transparency means you need to be explainable. Your model needs to be explainable.

Syed Ahmed · Jul 7, 2026 · full quote

Arguments against

Premature technical mandate

Critics argue that explainability and related concepts are still aspirational or not yet mature enough to be defined and imposed broadly. In this view, turning them into policy requirements too early would add unnecessary burdens.

When addressing definitions, C_TEC urges NIST to avoid defining premature requirements relating to aspirational standards for AI that either have not matured or are not developed yet, despite ongoing efforts by the private sector and academia to develop them. These include concep…

U.S. Chamber of Commerce's Technology Engagement Center · Sep 17, 2021 · full quote

Limits of explainability

Some opponents contend that advanced AI systems may be unable to accurately explain certain decisions, making a universal requirement unrealistic. They also question demanding a level of internal explanation from AI that people themselves often cannot provide.

[...] advanced AIs would not be able to accurately explain some of their decisions [...]

Roman V. Yampolskiy · Jun 20, 2019 · full quote
GH: I’m an expert on trying to get the technology to work, not an expert on social policy. One place where I do have technical expertise that’s relevant is [whether] regulators should insist that you can explain how your AI system works. I think that would be a complete disaster.…

Geoffrey Hinton · Dec 12, 2018 · full quote

Costs and misdirected focus

Another objection is that forcing human-understandable explanations adds an extra constraint that could affect decision quality. From this perspective, the better policy question is which decisions actually require explanations, rather than imposing a blanket explicability rule on AI.

Forcing an AI to produce a human-understandable explanation is an additional constraint, and it could affect the quality of its decisions.

Bruce Schneier · 2021 · full quote
[...] principles requiring that AI be explicable are misguided. We should be deciding which decisions require explanations. Automation is still an option;

Scott Robbins · 2019 · full quote

Middle ground

Proportional interpretability

Some voices support stronger interpretability but not a demand for perfect understanding. They favor safe boundaries and proportional requirements, asking that understandability keep pace with capability and that explanations be provided where technically feasible or wherever possible.

We don’t need perfect understanding. But we do need to ensure that understandability keeps pace with capability, and to give high priority to building insight into how these systems work, how they fail, and how they can be aligned with human needs and flourishing.

Eric Horvitz · Jun 4, 2026 · full quote
Understandability is fundamental to building trust and should be a core design feature, not an afterthought. Though AI systems are probabilistic, they must have clear explanations and disclosures to help users and regulators understand how the system works, what it means for the …

Ministry of Electronics and Information Technology · Feb 15, 2026 · full quote
Wherever possible, provide reasonably interpretable and explainable AI systems and models to accountable people within the organisation or downstream deployers to enable them to meet their own regulatory obligations.

National AI Centre · May 1, 2026 · full quote
We have a very natural instinct to say: we want as much interpretability as possible. One of the sci-fi worry cases is that agents start speaking in languages to each other that we don't understand. [...] I bet there are ways to define boundaries that are safe.

Reid Hoffman · Jan 5, 2026 · full quote

Insights

Supporters rarely treat interpretability as a standalone feature; they usually pair it with auditability, traceability, documentation, human oversight, or redress.
Many supportive quotes focus especially on high-risk or consequential contexts, including public services, hiring, credit, security, and agentic systems.
Supportive voices differ on strictness: some call for human-legible reasoning, while others explicitly say perfect understanding is unnecessary or that explanations should be provided where technically feasible.
Critical quotes split between feasibility and policy-design concerns: some say the standards are not mature, while others warn that forced explanations can be unrealistic or can degrade quality.
The supporting case is voiced by governments, regulators, companies, researchers, and civil-society organizations, while the skeptical quotes mainly target overbroad or technically rigid requirements.

Across the quotes, interpretability is most often defended as a governance and safety requirement for high-risk or increasingly capable AI, especially where people need to contest outcomes or institutions need to audit behavior. The main resistance centers on whether a blanket requirement is technically mature, universally achievable, or compatible with decision quality. Several nuanced voices support bounded, capability-linked interpretability rather than perfect transparency.

Cast your vote:

Results (83 votes):

Total (83 votes)

For 72 (87%) Abstain 4 (5%) Against 7 (8%)

For (70)

National Cyber Security Centre (Ireland)
Ireland’s national authority for cyber-security guidance and incident response.

I am this person! Add a sourced quote Report comment

votes For and says:

Is the provided explainability proportionate to the system’s risk classification, and does it provide sufficient technical detail for the cyber security and operational teams to understand, challenge, and respond to unexpected or malicious outputs?
AI Verified source (Jul 31, 2026)

added 20h ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Hiskias Dingeto

I am this person! Add a sourced quote Report comment

votes For and says:

For interpretability, high reconstruction does not certify individual claims. For AI safety, RECAP makes designated internal content independently checkable against probes rather than asserted by prose a model can game: an independent probe scores th...
more AI Verified source (Jul 22, 2026)

added 9h ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Eric Zielinski

I am this person! Add a sourced quote Report comment

votes For and says:

We are deploying AI we can't explain, defending AI we can't inspect, and trusting AI we can't audit. That is not a governance program. That is a liability surface.
AI Verified source (Jul 14, 2026)

added 25d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Demis Hassabis
Nobel laureate, AI Researcher and CEO of DeepMind

I am this person! Add a sourced quote Report comment

votes For and says:

Model assessments should include rigorous scientific evaluations of capabilities in cybersecurity, biological threats and other high-risk domains. Specific agentic AI tests could look for attempts to bypass safety guardrails or signs of deception, an...
more AI Verified source (Jul 14, 2026)

added 27d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Net Zero Economy Authority
Australian Government authority supporting the net-zero economic transition.

I am this person! Add a sourced quote Report comment

votes For and says:

There should be transparency and responsible disclosure so people can understand when they are being significantly impacted by AI and can find out when an AI system is engaging with them.
AI Verified source (Jul 10, 2026)

added 19d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Archbishop Ettore Balestrero
Apostolic Nuncio and Permanent Observer of the Holy See to the United Nations and other international organizations in Geneva.

I am this person! Add a sourced quote Report comment

votes For and says:

AI [must] be transparent in automated decision-making, to ensure “the chain of accountability tracing responsibility from developers to deployers at every AI lifecycle stage.”
AI Verified source (Jul 7, 2026)

added 25d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Sam Gregory
Executive Director, WITNESS

I am this person! Add a sourced quote Report comment

votes For and says:

Both must rest on independent third-party oversight, trust verified by third parties, not asserted by the developers who profit from them.
AI Verified source (Jul 7, 2026)

added 26d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Universal Postal Union
United Nations specialized agency for the postal sector, representing postal governance across 192 member states.

I am this person! Add a sourced quote Report comment

votes For and says:

Principles such as transparency actually require further deep dive to ensure that we have a common level of understanding as to how it might actually apply in a particular sector.
AI Verified source (Jul 7, 2026)

added 26d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Government of France
National government of France, represented at the UN Global Dialogue on AI Governance.

I am this person! Add a sourced quote Report comment

votes For and says:

Private actors, for their part, need a transparent framework and information for their use of AI.
AI Verified source (Jul 7, 2026)

added 26d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Government of Latvia
National government of Latvia, speaking through its delegation at the UN Global Dialogue on AI Governance in July 2026.

I am this person! Add a sourced quote Report comment

votes For and says:

Trust must be demonstrated through practical verifiability, transparent governance, data policies, and independent testing of high-risk applications throughout their life cycle.
AI Verified source (Jul 7, 2026)

added 26d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Eugênio Garcia
Brazilian delegate speaking at the UN Global Dialogue on Artificial Intelligence Governance in July 2026.

I am this person! Add a sourced quote Report comment

votes For and says:

Opacity cannot be accepted as an inevitable feature of innovation.
AI Verified source (Jul 7, 2026)

added 26d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Paula Bragantes Zamora
Costa Rican diplomat who co-chaired the UN Global Dialogue on Artificial Intelligence Governance in July 2026.

I am this person! Add a sourced quote Report comment

votes For and says:

Second, develop a shared minimum baseline for interpretability that builds on existing global frameworks and reinforces trust and openness.
AI Verified source (Jul 7, 2026)

added 27d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Syed Ahmed
Global systems integrator executive at Infosys, speaking on AI-governance interoperability and standards.

I am this person! Add a sourced quote Report comment

votes For and says:

In ISO, transparency means you need to be explainable. Your model needs to be explainable.
AI Verified source (Jul 7, 2026)

added 29d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Government of Bangladesh
National government of Bangladesh, speaking through its delegation at the UN Global Dialogue on AI Governance in July 2026.

I am this person! Add a sourced quote Report comment

votes For and says:

Today’s AI systems are increasingly powerful yet opaque. Its decision-making process is uninterpretable.
AI Verified source (Jul 7, 2026)

added 29d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Amal El-Fallah Segrouchni
Morocco’s Minister Delegate in charge of Digital Transition and Administrative Reform.

I am this person! Add a sourced quote Report comment

votes For and says:

We need to be able to open up the black boxes of AI.
AI Verified source (Jul 7, 2026)

added 1mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Mena El Asady
Computer science professor and member of the UN Independent International Scientific Panel on AI, presenting its Working Group 1 findings in July 2026.

I am this person! Add a sourced quote Report comment

votes For and says:

We have to independently work on interpretability and reliable auditing methods because they are a critical bottleneck and they remain immediate concerns for the scientific community.
AI Verified source (Jul 6, 2026)

added 14h ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Government of Lesotho
National government of Lesotho, speaking through its deputy prime minister at the UN Global Dialogue on AI Governance in July 2026.

I am this person! Add a sourced quote Report comment

votes For and says:

Where AI affects fundamental rights or the delivery of public services, meaningful human oversight, transparency, and effective mechanisms for accountability and redress must remain central.
AI Verified source (Jul 6, 2026)

added 19d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Institute of Chartered Accountants in England and Wales
Professional membership body for chartered accountants in England and Wales, publishing guidance on AI governance and responsible adoption.

I am this person! Add a sourced quote Report comment

votes For and says:

Organisations should require interpretable outputs and documented logic.
AI Verified source (Jul 3, 2026)

added 25d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Future of Life Institute
Nonprofit on existential risks

I am this person! Add a sourced quote Report comment

votes For and says:

Leading AI companies should: Ensure Human-Legible Reasoning - AI models should reason in ways that are accessible and understandable to humans. Developers should avoid opaque reasoning methods.
AI Verified source (Jul 2026)

added 27d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
James Hall
VP and Country Manager UK&I at Snowflake, writing on transparent and explainable AI governance.

I am this person! Add a sourced quote Report comment

votes For and says:

Explainable and observable AI are key parts of our Responsible AI principles at Snowflake, and it needs to be a governing principle for any organisation that hopes to reap the benefits of AI. Compliance, legal, and business teams need to be able to v...
more AI Verified source (Jun 18, 2026)

added 3d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Guide Labs Team
AI research team at Guide Labs, publishing research on interpretable language models.

I am this person! Add a sourced quote Report comment

votes For and says:

Post-hoc explanations have no guarantee of being faithful, because the model was never trained to make them valid. The alternative is to make interpretability part of the training contract.
AI Verified source (Jun 11, 2026)

added 3d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Eric Horvitz
Microsoft Chief Scientist

I am this person! Add a sourced quote Report comment

votes For and says:

We don’t need perfect understanding. But we do need to ensure that understandability keeps pace with capability, and to give high priority to building insight into how these systems work, how they fail, and how they can be aligned with human needs an...
more AI Verified source (Jun 4, 2026)

added 14d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Defense Advanced Research Projects Agency (DARPA)
United States Department of Defense research and development agency.

I am this person! Add a sourced quote Report comment

votes For and says:

The behavior, decisions, and impacts of AI systems must be understandable to humans, with an objective to move beyond explanations in routine settings toward operational interpretability.
AI Verified source (Jun 1, 2026)

added 26d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Reserve Bank of Australia
Australia's central bank.

I am this person! Add a sourced quote Report comment

votes For and says:

When AI influences analysis that informs internal or external communications, the RBA expects clear disclosure and the maintenance of documentation that supports explainability and traceability.
AI Verified source (May 12, 2026)

added 19d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Tshilidzi Marwala
Rector of the United Nations University.

I am this person! Add a sourced quote Report comment

votes For and says:

The challenge ahead is therefore not merely regulatory. In a deeper sense, it is about governing rationality itself, ensuring that as these systems grow more capable, they remain accountable, interpretable and anchored in values that cannot be reduce...
more AI Verified source (May 8, 2026)

added 23d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
U.S. Department of Energy
U.S. federal department responsible for national energy policy, national laboratories, and nuclear security.

I am this person! Add a sourced quote Report comment

votes For and says:

The assessment must address the following: [...] Explainability requirements.
AI Verified source (May 8, 2026)

added 25d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Federation of American Scientists
US science policy nonprofit

I am this person! Add a sourced quote Report comment

votes For and says:

Support research into interpretability, robustness, control, and the preservation of monitorable AI reasoning.
AI Verified source (May 2026)

added 23d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
National AI Centre
Australian Government initiative supporting safe and responsible AI adoption.

I am this person! Add a sourced quote Report comment

votes For and says:

Wherever possible, provide reasonably interpretable and explainable AI systems and models to accountable people within the organisation or downstream deployers to enable them to meet their own regulatory obligations.
AI Verified source (May 1, 2026)

added 26d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Department of the Prime Minister and Cabinet
Australian Government department responsible for advising the Prime Minister and Cabinet.

I am this person! Add a sourced quote Report comment

votes For and says:

We use AI models that are interpretable, fair, and free from bias, with clear explanations available for decisions made using AI.
AI Verified source (May 1, 2026)

added 26d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Rohan Sharma
Global Head, AI & Analytics

I am this person! Add a sourced quote Report comment

votes For and says:

If leadership cannot clearly trace the decision back to defined objectives and human intent, the system should not be operating. Governance begins where explainability fails.
AI Verified source (Apr 28, 2026)

added 13d ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate