Who Should Decide Whether AI Is Safe?
Sam Altman says people outside the AI labs need a real say. Four questions tell you whether that promise becomes real oversight.
Deciding whether AI is safe currently happens inside the companies building it. This week, one of them proposed changing that.
Sam Altman said that people outside the AI labs should have a real say in how the technology develops, and a clear way to judge whether it is happening safely.
He was introducing an OpenAI proposal for shared technical standards, independent assessment and clearer reporting when advanced systems fail. The company frames these as standards rather than licences, coordinated through national safety institutes and led by the United States. Altman added that standards should keep power from concentrating, by making sure new companies and open-model developers can still compete.
That sounds like a policy discussion. It is also a practical question about trust.
Who regulates AI safety right now?
Mostly the labs themselves. Companies run their own tests and publish their own results. Governments are still writing the rules, and the independent evaluators who do exist see what each company chooses to show them.
Most of us will never inspect a model or test a safety claim, which makes the real question a simple one: does the oversight give outsiders enough access and authority to find something the company would rather keep quiet?
Plenty of people are ready for that question. In a Reuters/Ipsos poll of 1,277 US adults taken over four days ending September 21, 73% said AI companies have yet to go far enough to prevent serious harm to society. The same share put safe, responsible development ahead of staying ahead of other countries. More than half said slowing development down would be a good thing.
A company inviting outside scrutiny is a real step. The next step is the machinery that makes scrutiny work.
How do you judge an AI safety standard?
Use The Oversight Test. Four questions separate the appearance of accountability from the conditions that create it:
- Who chooses the outside voices? Public input means something when the people invited reach beyond the familiar technical experts. Workers, teachers, researchers, regulators and the communities living with these systems each notice different risks.
- What can independent evaluators see? A review is as independent as its access. Evaluators need enough information to challenge a company's conclusions, rather than confirm a prepared demonstration.
- What happens after a failure? Incidents described in a consistent, shared format let other companies and countries compare evidence, spot patterns and improve their own safeguards. Failures shared openly teach the whole field.
- Who can afford to comply? Strong standards protect the public. Expensive ones can also work as a moat that only today's largest labs can cross. A credible framework measures risk while leaving room for smaller and open-model developers, which is the concentration-of-power problem Altman named himself.
These four questions work for any safety framework, from any lab or government, long after this week's news.
The Focused Human Lens
Attention costs energy, and there is only so much of it in a week. Following every technical argument and weighing competing corporate assurances would consume all of yours.
Good standards do that work for you. They compress the complexity into evidence you can check: who tested the system, what they found, what failed and what changed afterward. That is what makes the four questions worth your attention, and the daily back-and-forth safe to skip. Your attention settles on the one thing that decides the rest, which is whether outsiders can see enough to tell you the truth. The same habit works at your own scale, when you check a single AI answer before acting on it.
What this makes possible
You get to hold an informed opinion about AI safety without becoming an AI researcher. When the next framework, pledge or set of standards arrives, you have four questions ready, and the answers tell you how much weight the promise carries.
Try it this week. Take one AI safety announcement you see in the news and ask the fourth question: who can afford to comply? With AI safety standards, the answer usually says a great deal about who the standard protects.
Wondering where your attention is going in an ordinary week? The two-minute attention quiz names the drain costing you the most right now, and gives you one thing to try about it.
A. Karacay is the author of The Focused Human. The completed Focused Human podcast series is on YouTube. The Weekly Attention Reset Protocol is free: a simple weekly practice for reclaiming coherence, fifteen minutes on Sunday and five minutes a day.
Stay curious!
