Require AI systems above a capability threshold to be interpretable

Key insights

Supportive quotes come from a mix of government bodies, researchers, civil society groups, and companies, showing that interpretability is being framed as both a technical and governance issue.
Many supportive quotes treat interpretability less as a user-facing feature than as oversight infrastructure tied to auditability, documentation, traceability, and independent testing.
A recurring fault line is whether opacity is an unavoidable property of advanced AI or a design choice that can be changed through model architecture, training, and governance.
Several supportive quotes reserve the strongest demands for high-risk, high-capability, or otherwise consequential systems rather than describing one identical standard for every use of AI.
Both supportive and skeptical quotes distinguish between nominal explanations and meaningful understanding, though they differ on whether reliable interpretability is achievable enough to require.

Arguments for

Oversight and accountability

Proponents argue that interpretable systems are necessary for governance because humans need to inspect, document, audit, and assign responsibility for AI behavior and impacts across the lifecycle. Without that visibility, they say, organizations are left trusting systems they cannot meaningfully oversee.

We are deploying AI we can't explain, defending AI we can't inspect, and trusting AI we can't audit. That is not a governance program. That is a liability surface.

Eric Zielinski · Jul 14, 2026 · full quote
AI [must] be transparent in automated decision-making, to ensure “the chain of accountability tracing responsibility from developers to deployers at every AI lifecycle stage.”

Archbishop Ettore Balestrero · Jul 7, 2026 · full quote
Trust must be demonstrated through practical verifiability, transparent governance, data policies, and independent testing of high-risk applications throughout their life cycle.

Government of Latvia · Jul 7, 2026 · full quote
The behavior, decisions, and impacts of AI systems must be understandable to humans, with an objective to move beyond explanations in routine settings toward operational interpretability.

Defense Advanced Research Projects Agency (DARPA) · Jun 1, 2026 · full quote
When AI influences analysis that informs internal or external communications, the RBA expects clear disclosure and the maintenance of documentation that supports explainability and traceability.

Reserve Bank of Australia · May 12, 2026 · full quote

Safety for powerful systems

For increasingly capable systems, supporters frame interpretability as a safety control that helps detect failures, deception, unsafe actions, and misalignment. They argue that oversight must not fall behind capability growth, especially as systems become more autonomous and harder to monitor.

A model that explains itself less isn't more aligned, it's just harder to catch when it goes wrong.

Oleksandr Yaremchuk · Sep 5, 2026 · full quote
Alignment auditing is starting to get really hard and we're going to need new techniques (e.g. interpretability-based) if we want to keep up.

Evan Hubinger · Sep 1, 2026 · full quote
Model assessments should include rigorous scientific evaluations of capabilities in cybersecurity, biological threats and other high-risk domains. Specific agentic AI tests could look for attempts to bypass safety guardrails or signs of deception, and ensure best practices, such …

Demis Hassabis · Jul 14, 2026 · full quote
Today, we find that models’ reasoning is generally interpretable and easy to monitor. However, in the future, monitorability may break down for a variety of reasons.

OpenAI · Mar 5, 2026 · full quote
I am very concerned about deploying such systems without a better handle on interpretability. These systems will be absolutely central to the economy, technology, and national security, and will be capable of so much autonomy that I consider it basically unacceptable for humanity…

Dario Amodei · 2025 · full quote
We’re trying to build safe, interpretable, steerable AI systems. These things are getting more and more powerful, and because they’re more powerful, they almost necessarily get more dangerous unless you put guardrails around them to make them safe intentionally.

Scott White · Jan 17, 2026 · full quote

Contestability and trust

Many quotes focus on people affected by AI-assisted decisions, arguing that they should be able to understand why an outcome occurred, challenge it, and receive genuine human oversight. This case is made especially strongly for public services, hiring, credit-related uses, and other consequential settings.

Governments should provide clear, simple explanations for how an AI system reaches an outcome.

Australia’s Data and Digital Ministers · Sep 4, 2026 · full quote
Where AI affects fundamental rights or the delivery of public services, meaningful human oversight, transparency, and effective mechanisms for accountability and redress must remain central.

Government of Lesotho · Jul 6, 2026 · full quote
We don't treat AI as a decision-maker. With structure, AI creates an explainable signal that teams can trust. [...] If AI can't explain itself, it doesn't belong in hiring.

Meredith Johnson · Apr 17, 2026 · full quote
Every score should come with an explanation that relates to specific behavioural evidence. This transforms oversight from rubber-stamping into a genuine review.

Buddhi Jayatilleke · Apr 15, 2026 · full quote
The national AI policy focuses on the creation of explainable AI systems that deliver clear, interpretable outputs. This is important for trust and acceptance by users, especially in high-risk applications such as credit scoring. By making decisions by AI systems comprehensible a…

Government of South Africa · Apr 10, 2026 · full quote
You should know that an automated system is being used and understand how and why it contributes to outcomes that impact you. Designers, developers, and deployers of automated systems should provide generally accessible plain language documentation including clear descriptions of…

White House Office of Science and Technology Policy (OSTP) · 2022 · full quote

Design for interpretability

Another supportive argument rejects the idea that opacity is simply the inevitable price of innovation. It holds that interpretability can be designed into models, training, and documentation, and that requirements would push developers toward more faithful, human-legible systems instead of relying on post-hoc explanations whose validity is uncertain.

Post-hoc explanations have no guarantee of being faithful, because the model was never trained to make them valid. The alternative is to make interpretability part of the training contract.

Guide Labs Team · Jun 11, 2026 · full quote
Opacity is often presented as inevitable. In practice, explainability is frequently a design and governance choice. Interpretable models, explainability techniques and documentation standards already exist and are used in safety-critical domains. Accepting opacity by default unde…

Niusha Shafiabady · Feb 25, 2026 · full quote
Leading AI companies should: Ensure Human-Legible Reasoning - AI models should reason in ways that are accessible and understandable to humans. Developers should avoid opaque reasoning methods.

Future of Life Institute · Jul 2026 · full quote
Opacity cannot be accepted as an inevitable feature of innovation.

Eugênio Garcia · Jul 7, 2026 · full quote
In ISO, transparency means you need to be explainable. Your model needs to be explainable.

Syed Ahmed · Jul 7, 2026 · full quote

Arguments against

Immature standards

Some opponents argue that explainability is still an aspirational area whose methods and standards have not yet matured enough for regulation. On this view, imposing a legal requirement now would create unnecessary burdens before the underlying concepts are sufficiently developed.

When addressing definitions, C_TEC urges NIST to avoid defining premature requirements relating to aspirational standards for AI that either have not matured or are not developed yet, despite ongoing efforts by the private sector and academia to develop them. These include concep…

U.S. Chamber of Commerce's Technology Engagement Center · Sep 17, 2021 · full quote

Inherent opacity

Another critique holds that some advanced systems may simply be unable to accurately explain certain decisions. From this perspective, a blanket interpretability requirement asks AI to do something that even people often cannot do when making complex judgments.

[...] advanced AIs would not be able to accurately explain some of their decisions [...]

Roman V. Yampolskiy · Jun 20, 2019 · full quote
GH: I’m an expert on trying to get the technology to work, not an expert on social policy. One place where I do have technical expertise that’s relevant is [whether] regulators should insist that you can explain how your AI system works. I think that would be a complete disaster.…

Geoffrey Hinton · Dec 12, 2018 · full quote

Decision-specific explanations

Critics also argue that policymakers should decide which decisions need explanations rather than require explicability from every powerful system. They warn that forcing human-understandable explanations is an extra constraint that could reduce decision quality.

Forcing an AI to produce a human-understandable explanation is an additional constraint, and it could affect the quality of its decisions.

Bruce Schneier · 2021 · full quote
[...] principles requiring that AI be explicable are misguided. We should be deciding which decisions require explanations. Automation is still an option;

Scott Robbins · 2019 · full quote

Middle ground

Risk-proportionate standard

Several quotes favor stronger interpretability expectations but frame them as calibrated rather than absolute. In this view, understandability should keep pace with capability, be proportionate to risk, and be provided wherever possible or to the extent technically feasible, with safe boundaries defined in practice.

Is the provided explainability proportionate to the system’s risk classification, and does it provide sufficient technical detail for the cyber security and operational teams to understand, challenge, and respond to unexpected or malicious outputs?

National Cyber Security Centre (Ireland) · Jul 31, 2026 · full quote
We don’t need perfect understanding. But we do need to ensure that understandability keeps pace with capability, and to give high priority to building insight into how these systems work, how they fail, and how they can be aligned with human needs and flourishing.

Eric Horvitz · Jun 4, 2026 · full quote
Wherever possible, provide reasonably interpretable and explainable AI systems and models to accountable people within the organisation or downstream deployers to enable them to meet their own regulatory obligations.

National AI Centre · May 1, 2026 · full quote
Understandability is fundamental to building trust and should be a core design feature, not an afterthought. Though AI systems are probabilistic, they must have clear explanations and disclosures to help users and regulators understand how the system works, what it means for the …

Ministry of Electronics and Information Technology · Feb 15, 2026 · full quote
We have a very natural instinct to say: we want as much interpretability as possible. One of the sci-fi worry cases is that agents start speaking in languages to each other that we don't understand. [...] I bet there are ways to define boundaries that are safe.

Reid Hoffman · Jan 5, 2026 · full quote

Validated understanding

Some supportive voices caution that simply calling a system explainable is not enough. They argue that explanations should be faithful, independently checkable, and tested with the people meant to rely on them, because current approaches can fall short of their stated goals.

We’ve become increasingly concerned that AI systems that claim to be interpretable and trustworthy to humans — termed ‘explainable AI’ — are not actually validated with the humans they’re intended to be understood by.

Ashley Suh · Jan 6, 2026 · full quote
When you design something that's meant to be interpreted, understood, and trusted by a real person, you ought to test whether it'll work as you intend with that person.

Hosea Siu · Jan 6, 2026 · full quote
The explanations of AI systems that we can currently offer typically fall short of the stated goals (including mine for appropriate reliance and system improvement), so it seems that if we truly have a strong requirement that systems are explainable then we simply cannot currentl…

Stefan Buijsman · Feb 27, 2026 · full quote
Post-hoc explanations have no guarantee of being faithful, because the model was never trained to make them valid. The alternative is to make interpretability part of the training contract.

Guide Labs Team · Jun 11, 2026 · full quote
For interpretability, high reconstruction does not certify individual claims. For AI safety, RECAP makes designated internal content independently checkable against probes rather than asserted by prose a model can game: an independent probe scores the verbalizer's true claims abo…

Hiskias Dingeto · Jul 22, 2026 · full quote

The debate shows broad support for more understandable AI, especially where systems are highly capable or affect consequential decisions, but not a single shared view of what a binding interpretability requirement should demand. Supporters present interpretability as a precondition for safety, oversight, and contestability, while critics question whether current methods are mature enough and whether some systems can realistically meet such a mandate without tradeoffs. A central unresolved issue is how to set a threshold and standard that are strict enough to be meaningful yet flexible enough to reflect technical limits and differing uses.

Cast Your Vote:

Results (93 votes):

Total (93 votes)

For 82 (88%) Abstain 4 (4%) Against 7 (8%)

New Top

For (80)

Jason McEwen
Interim Chief Scientist and Mission Director for Fundamental Research in AI at the Alan Turing Institute; Professor at University College London.

I am this person! Add a sourced quote Report comment

votes For and says:

The scientific work now is to ensure reasoning of powerful AI models remains observable as they become more capable, or to build the tools to read it where it is not, and to verify that complete AI systems, not just the models inside them, can be rel...
more AI Verified source (Sep 23, 2026)

added 1mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Oleksandr Yaremchuk
Co-Founder and CTO at Manifold Security.

I am this person! Add a sourced quote Report comment

votes For and says:

A model that explains itself less isn't more aligned, it's just harder to catch when it goes wrong.
AI Verified source (Sep 5, 2026)

added 1mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Sydney Von Arx
AI safety researcher and founder of the nonprofit Nightingale.

I am this person! Add a sourced quote Report comment

votes For and says:

This is an even bigger deal than Hugging Face.
AI Verified source (Sep 4, 2026)

added 1mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Australia’s Data and Digital Ministers
Australian federal, state, and territory ministers responsible for data and digital policy.

I am this person! Add a sourced quote Report comment

votes For and says:

Governments should provide clear, simple explanations for how an AI system reaches an outcome.
AI Verified source (Sep 4, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Evan Hubinger
Research scientist at Anthropic working on model organisms of misalignment; former Research Fellow at MIRI.

I am this person! Add a sourced quote Report comment

votes For and says:

Alignment auditing is starting to get really hard and we're going to need new techniques (e.g. interpretability-based) if we want to keep up.
AI Verified source (Sep 1, 2026)

added 1mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
National Cyber Security Centre (Ireland)
Ireland’s national authority for cyber-security guidance and incident response.

I am this person! Add a sourced quote Report comment

votes For and says:

Is the provided explainability proportionate to the system’s risk classification, and does it provide sufficient technical detail for the cyber security and operational teams to understand, challenge, and respond to unexpected or malicious outputs?
AI Verified source (Jul 31, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Hiskias Dingeto

I am this person! Add a sourced quote Report comment

votes For and says:

For interpretability, high reconstruction does not certify individual claims. For AI safety, RECAP makes designated internal content independently checkable against probes rather than asserted by prose a model can game: an independent probe scores th...
more AI Verified source (Jul 22, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Andrea Ferrario
Author of a 2026 arXiv paper on auditable trustworthiness in AI lifecycle governance.

I am this person! Add a sourced quote Report comment

votes For and says:

Using decision trees as an interpretable proof-of-concept model class, the methodology yields explicit trustworthiness plateaus, readable level transitions, and two simple lifecycle diagnostics: boundary margins and profile drift.
AI Verified source (Jul 17, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Eric Zielinski

I am this person! Add a sourced quote Report comment

votes For and says:

We are deploying AI we can't explain, defending AI we can't inspect, and trusting AI we can't audit. That is not a governance program. That is a liability surface.
AI Verified source (Jul 14, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Demis Hassabis
Nobel laureate, AI Researcher and CEO of DeepMind

I am this person! Add a sourced quote Report comment

votes For and says:

Model assessments should include rigorous scientific evaluations of capabilities in cybersecurity, biological threats and other high-risk domains. Specific agentic AI tests could look for attempts to bypass safety guardrails or signs of deception, an...
more AI Verified source (Jul 14, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Net Zero Economy Authority
Australian Government authority supporting the net-zero economic transition.

I am this person! Add a sourced quote Report comment

votes For and says:

There should be transparency and responsible disclosure so people can understand when they are being significantly impacted by AI and can find out when an AI system is engaging with them.
AI Verified source (Jul 10, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Archbishop Ettore Balestrero
Apostolic Nuncio and Permanent Observer of the Holy See to the United Nations and other international organizations in Geneva.

I am this person! Add a sourced quote Report comment

votes For and says:

AI [must] be transparent in automated decision-making, to ensure “the chain of accountability tracing responsibility from developers to deployers at every AI lifecycle stage.”
AI Verified source (Jul 7, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Sam Gregory
Executive Director, WITNESS

I am this person! Add a sourced quote Report comment

votes For and says:

Both must rest on independent third-party oversight, trust verified by third parties, not asserted by the developers who profit from them.
AI Verified source (Jul 7, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Universal Postal Union
United Nations specialized agency for the postal sector, representing postal governance across 192 member states.

I am this person! Add a sourced quote Report comment

votes For and says:

Principles such as transparency actually require further deep dive to ensure that we have a common level of understanding as to how it might actually apply in a particular sector.
AI Verified source (Jul 7, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Government of France
National government of France, represented at the UN Global Dialogue on AI Governance.

I am this person! Add a sourced quote Report comment

votes For and says:

Private actors, for their part, need a transparent framework and information for their use of AI.
AI Verified source (Jul 7, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Government of Latvia
National government of Latvia, speaking through its delegation at the UN Global Dialogue on AI Governance in July 2026.

I am this person! Add a sourced quote Report comment

votes For and says:

Trust must be demonstrated through practical verifiability, transparent governance, data policies, and independent testing of high-risk applications throughout their life cycle.
AI Verified source (Jul 7, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Eugênio Garcia
Brazilian delegate speaking at the UN Global Dialogue on Artificial Intelligence Governance in July 2026.

I am this person! Add a sourced quote Report comment

votes For and says:

Opacity cannot be accepted as an inevitable feature of innovation.
AI Verified source (Jul 7, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Paula Bragantes Zamora
Costa Rican diplomat who co-chaired the UN Global Dialogue on Artificial Intelligence Governance in July 2026.

I am this person! Add a sourced quote Report comment

votes For and says:

Second, develop a shared minimum baseline for interpretability that builds on existing global frameworks and reinforces trust and openness.
AI Verified source (Jul 7, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Syed Ahmed
Global systems integrator executive at Infosys, speaking on AI-governance interoperability and standards.

I am this person! Add a sourced quote Report comment

votes For and says:

In ISO, transparency means you need to be explainable. Your model needs to be explainable.
AI Verified source (Jul 7, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Government of Bangladesh
National government of Bangladesh, speaking through its delegation at the UN Global Dialogue on AI Governance in July 2026.

I am this person! Add a sourced quote Report comment

votes For and says:

Today’s AI systems are increasingly powerful yet opaque. Its decision-making process is uninterpretable.
AI Verified source (Jul 7, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Amal El-Fallah Segrouchni
Morocco’s Minister Delegate in charge of Digital Transition and Administrative Reform.

I am this person! Add a sourced quote Report comment

votes For and says:

We need to be able to open up the black boxes of AI.
AI Verified source (Jul 7, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Mena El Asady
Computer science professor and member of the UN Independent International Scientific Panel on AI, presenting its Working Group 1 findings in July 2026.

I am this person! Add a sourced quote Report comment

votes For and says:

We have to independently work on interpretability and reliable auditing methods because they are a critical bottleneck and they remain immediate concerns for the scientific community.
AI Verified source (Jul 6, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Government of Lesotho
National government of Lesotho, speaking through its deputy prime minister at the UN Global Dialogue on AI Governance in July 2026.

I am this person! Add a sourced quote Report comment

votes For and says:

Where AI affects fundamental rights or the delivery of public services, meaningful human oversight, transparency, and effective mechanisms for accountability and redress must remain central.
AI Verified source (Jul 6, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Institute of Chartered Accountants in England and Wales
Professional membership body for chartered accountants in England and Wales, publishing guidance on AI governance and responsible adoption.

I am this person! Add a sourced quote Report comment

votes For and says:

Organisations should require interpretable outputs and documented logic.
AI Verified source (Jul 3, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Future of Life Institute
Nonprofit on existential risks

I am this person! Add a sourced quote Report comment

votes For and says:

Leading AI companies should: Ensure Human-Legible Reasoning - AI models should reason in ways that are accessible and understandable to humans. Developers should avoid opaque reasoning methods.
AI Verified source (Jul 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
James Hall
VP and Country Manager UK&I at Snowflake, writing on transparent and explainable AI governance.

I am this person! Add a sourced quote Report comment

votes For and says:

Explainable and observable AI are key parts of our Responsible AI principles at Snowflake, and it needs to be a governing principle for any organisation that hopes to reap the benefits of AI. Compliance, legal, and business teams need to be able to v...
more AI Verified source (Jun 18, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Guide Labs Team
AI research team at Guide Labs, publishing research on interpretable language models.

I am this person! Add a sourced quote Report comment

votes For and says:

Post-hoc explanations have no guarantee of being faithful, because the model was never trained to make them valid. The alternative is to make interpretability part of the training contract.
AI Verified source (Jun 11, 2026)

added 2mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Thomas Holland
Policy analyst at the Nakada Foundation to Save Humanity, writing on AI governance and superintelligence risk.

I am this person! Add a sourced quote Report comment

votes For and says:

This is why Anthropic has made interpretability research a stated organizational priority, and why the Foundation's governance proposals include interpretability verification requirements as a condition of frontier AI deployment.
AI Verified source (Jun 9, 2026)

added 1mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Eric Horvitz
Microsoft Chief Scientist

I am this person! Add a sourced quote Report comment

votes For and says:

We don’t need perfect understanding. But we do need to ensure that understandability keeps pace with capability, and to give high priority to building insight into how these systems work, how they fail, and how they can be aligned with human needs an...
more AI Verified source (Jun 4, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate
Defense Advanced Research Projects Agency (DARPA)
United States Department of Defense research and development agency.

I am this person! Add a sourced quote Report comment

votes For and says:

The behavior, decisions, and impacts of AI systems must be understandable to humans, with an objective to move beyond explanations in routine settings toward operational interpretability.
AI Verified source (Jun 1, 2026)

added 3mo ago

Choose a list of delegates

to vote as the majority of them.

Unless you vote directly.

Delegate