When I started building AI workers for Wheeler Intelligent Systems, I expected the hard part to be getting them to do useful work. Could an AI worker understand a job? Could it use the right tools? Could it complete work without me directing every step? Could I give it a goal and get back something I could actually use?
Those are still important questions. But after spending time actually building AI workers, I have discovered another problem that may be even more important: What should the AI worker not be allowed to do?
That question sounds simple. It isn’t. As AI gets better, I am becoming less concerned about whether it can perform a task and more concerned about whether I can define and enforce the boundaries around that task. That has changed the way I think about AI agents, autonomous systems, and what I call a Managed AI Workforce.
I Started With Capability
Wheeler Intelligent Systems is an experiment as much as it is a company. I am trying to understand what happens when a very small company has access to a large amount of AI capacity. Instead of automatically asking, “Who should I hire to do this?” I can ask a different question: “Could an AI worker do this?”
Software development has been one of my laboratories. I can define a business problem, give an AI coding worker a job, and have it produce software. Other AI systems can help research the problem, examine the code, run tests, and look for weaknesses.
That is remarkable. Work that once might have required a small development team can increasingly be attempted by one person directing AI. At first, this makes capability seem like the big breakthrough.
Then you start operating the system, and another problem appears.
Being Able to Do Something Is Not the Same as Being Allowed to Do It
Imagine I have an AI worker that can write software. That does not mean I want it to merge its own code. I may want it to create a proposed change, test that change, and prepare the evidence for someone else to review.
Likewise, an AI accounting worker might be capable of creating a payment. That does not mean it should be able to approve the payment. A sales AI might be capable of changing a customer’s price. That does not mean it has the authority to do so.
This distinction is basic management. We have dealt with it for years with people and software. An employee may know how to issue a check but not have the authority to sign one. A manager may be able to view payroll information but not change the payroll system. An accounting clerk may enter a vendor invoice but not approve the payment to that vendor.
We call many of these controls separation of duties. AI doesn’t make these ideas obsolete. It makes them more important because an AI worker can sometimes move much faster than a person.
“Don’t Do That” Isn’t Much of a Control
This is one of the more important lessons I have learned. You can tell an AI not to access something, not to change something, not to take an action without approval, or not to continue if a certain condition occurs. Those instructions matter, but instructions alone are not enough.
If the system is important, I want the boundary to exist outside the AI’s willingness to follow an instruction. Think about a company credit card. You can tell an employee, “Don’t spend more than $1,000.” That is a policy. But you can also give the card a $1,000 spending limit. That is a control.
The best business systems often use both. AI needs the same thinking. I don’t merely want an AI worker that has been asked not to exceed its authority. I want a system designed so exceeding that authority becomes difficult or impossible.
That difference has consumed far more of my attention than I expected.
Then I Learned That Controls Need to Be Tested Too
It isn’t enough to build a control and say, “That looks good.” You have to try to break it.
What happens if the expected information is missing? What happens if two instructions conflict? What happens if a worker that should be stopped tries to continue? What happens if another part of the system gives the AI information that makes the work look authorized when it isn’t? What happens if the AI finds another path to accomplish the same thing?
This starts looking less like ordinary prompting and more like engineering, internal controls, security, and management combined.
I have spent much of my career around accounting systems, ERP systems, manufacturing systems, and business controls. The technology is different, but the management problem feels surprisingly familiar: Do not assume a control works because you wrote it down. Test it.
I Don’t Want the Builder Grading Its Own Work
Another lesson from the Wheeler IS experiment has been the value of independent evaluation. One AI worker can build something. A separate evaluator can examine it. I increasingly like that separation.
Suppose an AI developer creates a software feature and then reports, “I tested my work. Everything looks good.” Maybe it does. But that is not enough evidence for me. The builder knows how it approached the problem. It knows the assumptions it made. It knows the tests it created.
That creates a risk. The builder may test whether the system works the way the builder expects it to work. I want another evaluator asking, “How could this fail in a way the builder didn’t expect?”
That is a different job. The evaluator should challenge assumptions, look for paths the builder missed, test the boundaries, and produce evidence. This doesn’t mean the evaluator is always right either. It means we have created another layer of challenge.
Businesses already understand this idea. The person preparing a bank reconciliation should not be the only person deciding whether unexplained differences are acceptable. The salesperson proposing a large discount may need someone else to approve it. The developer who writes software is not necessarily the person who should approve it for production.
AI workers should not get a free pass from controls that we already know are valuable.
I Started Thinking About AI Authority Like a Credit Limit
This led me to another idea: AI autonomy should be earned.
When I create a new AI worker, why would I immediately give it broad authority? I wouldn’t do that with a new employee.
I might start an AI worker in observation mode. It can watch what happens, find exceptions, and report them. If it performs well, perhaps the next step is drafting. It can prepare the response but not send it, prepare the transaction but not post it, or prepare the recommendation but not execute it.
Then perhaps it moves to recommendation. It can tell me what it thinks we should do and provide the evidence supporting that recommendation. Eventually, for some jobs, it may earn limited authority to act.
That authority might grow as performance improves. It might also shrink if performance gets worse. Certain activities may always require human approval regardless of performance.
I have started thinking about this almost like a credit limit. A new worker starts small. Reliable performance may justify a larger limit. Poor performance may reduce it. The goal isn’t maximum autonomy. The goal is the right amount of autonomy for the evidence we have.
The Human Role Doesn’t Disappear
This experiment has also changed how I think about the human side of AI. There is a popular idea that as AI becomes more autonomous, people simply disappear from the process. I don’t think that describes what I am seeing.
The human role moves. Instead of performing every step, I spend more time deciding what the worker should accomplish, what information it should trust, what tools it should have, what authority it should receive, how its performance will be measured, what happens when it fails, who or what independently checks its work, and when its authority should increase.
Those are management questions. In some ways, better AI makes management more important rather than less important.
If an AI worker can perform ten times as much work, a poorly designed process can now produce ten times as many problems. Speed magnifies good systems. It can also magnify bad ones.
This Changed How I Think About the Product
When I first thought about a Managed AI Workforce, it was easy to focus on the workers: a research agent, a sales agent, an accounting agent, a customer-service agent, or a software-development agent.
But I increasingly think the AI worker itself may be only part of the product. The management system around the worker may be just as important.
Every serious AI role needs a purpose, approved knowledge, tools, authority, limits, tests, performance measures, escalation rules, and a human owner. It also needs a way to stop.
That last part deserves more attention than it usually receives. If an AI worker starts behaving incorrectly, can I stop it? Can I remove its access? Can I determine what it already did? Can I identify which information it used? Can I correct the problem before it starts working again?
These aren’t exciting demonstration features, but they are the kinds of questions businesses eventually have to answer.
The Better AI Gets, the More This Matters
AI models are improving quickly. That means many tasks that seem difficult today will probably become easier: better reasoning, better coding, better research, better use of business systems, and better ability to work across applications.
That’s good news. But every increase in capability makes the authority question more important.
A chatbot that can only answer questions has a limited blast radius. An AI worker that can access company systems, change records, communicate with customers, create software, move money, place orders, or direct other AI workers is something different.
The important question is no longer simply, “Can the AI do it?”
It becomes, “Should this AI be allowed to do it, under what conditions, and how do we know the boundary will hold?”
That is where I think much of the real work of building AI-native companies will happen.
I’m Building the Management System, Not Just the Worker
I still want capable AI workers. I want them to research, build, monitor, find opportunities I missed, and complete useful work while I am doing something else. That is part of the promise of an AI-native company.
But I no longer think capability alone is enough.
The worker needs a job. The job needs boundaries. The boundaries need controls. The controls need tests. The tests need independent challenge. Performance needs measurement. Greater autonomy needs to be earned.
I started this experiment thinking the hard question would be whether AI could do the work.
I’m increasingly convinced that’s the easier question.
The harder question is whether we can build organizations that know when AI should work, what authority it should have, how we know when it fails, and when it has earned the right to do more.
That’s the part I’m working on now.