AIAgentsSafetyAutomation

Give an AI agent a budget, not a blank check

People often talk about AI agents as if the hard part is making them capable.

The harder part is making them limited.

A helpful assistant is useful because it can draft, sort, summarize, and prepare work. A dangerous one is useful for the same reasons, except it can also send a message you did not mean to send, change a record you still needed, or publish something that looked finished but was not checked.

That is why the most useful question is not "How much can this assistant do?" It is "What is this assistant allowed to do, and what still needs a person?"

A budget is a boundary with a purpose

A budget is not a punishment. It is a way to match the tool to the job.

Think of it like this: if you were asking a junior person to handle a task, you would not hand them every key, every account, and every publish button and hope for the best. You would give them enough to do the work, and a clear rule for when to stop and ask.

AI agents deserve the same kind of framing. The goal is not to make the assistant weak. The goal is to make its authority explicit.

The first rule: separate preparation from execution

One of the most useful boundaries is the line between preparing an action and executing it.

An assistant can prepare a draft email, a proposed edit, a suggested publish, or a likely next step. That is often where it adds the most value.

But the moment the action becomes consequential—sending a message to a client, changing a shared document, spending money, publishing something publicly—the assistant should not act as if "looks good" means "go ahead."

The better pattern is:

• The assistant proposes.
• A person reviews.
• The person approves or changes the proposal.
• Only then does the action happen.

This is not a fancy workflow. It is just the difference between a tool that helps and a tool that decides.

Define the smallest useful set of permissions

When people set up an agent, they often start by asking what the agent can do. A better starting point is what the agent should be allowed to do for this specific task.

A practical permission set usually includes:

• what the assistant may read,
• what it may draft,
• what it may propose,
• what it may not do on its own,
• what requires explicit approval,
• what must never be automated.

That last one matters more than it sounds. "No automation" is sometimes the right answer for sensitive actions, even if the assistant is otherwise helpful.

A small, specific permission set is usually safer than a broad one with a vague promise to "be careful."

Give the agent a visible stop button

Every useful agent should have a way to stop it before it becomes a problem.

That could mean:

• a review step before any send,
• a human confirmation before any publish,
• a dry-run mode that shows what would happen without doing it,
• a manual override that pauses or cancels the next step.

The point is not to assume the assistant will fail. The point is to make failure less expensive when it happens.

If you cannot tell whether the assistant is about to do something consequential, you do not have a usable agent yet. You have a black box with access.

Use the simplest approval that matches the risk

Not every action needs the same level of review.

A low-risk action might only need a quick check. A high-risk action might need a person to approve the exact change, not just the general idea.

A useful habit is to sort actions into three buckets:

• **Prep only:** the assistant can draft or organize, but nothing leaves the workspace without a person.
• **Review then act:** the assistant prepares a change, and a person approves the exact action.
• **Manual only:** the assistant can suggest, but the person does the action themselves.

This keeps the workflow practical. If every action required the same heavy process, people would stop using the assistant. If every action required no process, the assistant would eventually do something you would not have approved.

Keep a record of what the assistant did

A budget is easier to trust when you can see what happened.

That means keeping a short log of:

• what the assistant was allowed to do,
• what it proposed,
• what was approved,
• what actually happened,
• what needed to be changed afterward.

This is not about surveillance for its own sake. It is about making the assistant's work reviewable. If something goes wrong, the record helps you understand whether the issue was the tool, the permission, the prompt, or the review step.

Start small and expand only after proof

The safest way to use an agent is to start with a narrow, low-risk task and expand only after it has proved useful.

For example:

• let the assistant draft, but not send,
• let it suggest edits, but not apply them,
• let it prepare a publish, but not publish,
• let it gather options, but not make the final choice.

Once the assistant has shown that it can handle a narrow task well, you can decide whether a wider permission makes sense. The point is to earn broader trust through small, observable wins, not to start with broad trust and hope.

Keep the human label honest

One reason agents feel risky is that the word "agent" can make the tool sound more decisive than it is.

A better frame is this: the assistant is a worker with a limited role. It can do parts of the job. It can prepare. It can suggest. But unless you have decided otherwise on purpose, it should not be the final authority on anything that matters.

NIST's AI RMF is intended for organizations designing, deploying, using, and evaluating AI systems, and its current site notes that the framework is being revised.[1] Its generative-AI profile recommends context-sensitive risk tiers, thresholds, testing, and processes to halt systems that pose unacceptable risk.[2] That is a useful reminder that risk is not a fixed property of the tool alone. It also depends on what the tool is allowed to touch, where it is used, and what happens after it acts.

Try this next

For the next assistant workflow you set up, define the budget before the tool is used:

• what it may read,
• what it may draft,
• what it may propose,
• what requires approval,
• what it may never do on its own.

Then test it on one small task. If it works, keep the boundary and consider a slightly wider one. If it fails, tighten the boundary rather than blaming the idea of assistance.

The goal is not to build an assistant that does everything. The goal is to build one that does the right things, in the right way, with a person still in control of what matters.

Sources

[1] https://www.nist.gov/itl/ai-risk-management-framework — NIST AI Risk Management Framework

[2] https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf — NIST Generative AI Profile