An automation with write access to your systems can do damage faster than a person can. That is not an argument against building one — it is an argument for settling a specific set of questions before it goes live.
This is the checklist we work through on every build. Our AI agent service ships with most of it as standard, but you should ask any vendor the same questions.
Data handling
1. Is your data used to train models?
It should not be. Enterprise API endpoints from the major providers do not train on customer data by default — consumer tiers may. Get the specific endpoint and tier in writing.
2. What actually leaves your network?
Map it. Often far less needs to reach a model than people assume. A classifier deciding which department an email belongs to needs the subject and body — not the attached customer database.
3. Is sensitive data redacted before processing?
Card numbers, national identifiers and health data can frequently be stripped or tokenised before a model sees them, with no loss of function. Ask whether this is done.
4. Where is data at rest, and for how long?
Logs, queues and caches all hold copies. Set a retention period and make sure it is actually enforced rather than aspirational.
Access and permissions
5. Is access scoped to the minimum?
The commonest and most dangerous shortcut is giving an automation admin credentials because it is quicker than configuring a role. Scope it to the specific objects and actions the job needs. An invoice-processing automation has no business being able to delete customers.
6. Does it have its own identity?
Never run automation under a staff member's account. It muddies every audit trail and it keeps working after that person leaves — or breaks unexpectedly when their access is revoked.
7. How are credentials stored and rotated?
In a secrets manager, not in the workflow definition, not in a config file in a repository. Ask how rotation works before you need it.
Behaviour controls
8. What requires human approval?
Anything irreversible or externally visible: sending to a customer, moving money, deleting records, changing pricing. Approval gates are the single most effective control and they cost very little.
9. What happens when it is not confident?
It should escalate, not guess. Confidence thresholds with a human queue behind them. Ask to see how the threshold is set and who monitors the queue.
10. Are there rate and volume limits?
A bug or a prompt injection that causes an automation to act ten thousand times is a much worse day than one that acts once. Cap the blast radius.
Visibility
11. Is every action logged and attributable?
What it did, when, on what input, and why. Without this you cannot investigate an incident or satisfy an auditor. It must be built in from the start.
12. Who gets alerted when something fails?
A named person, through a channel they actually read. The worst automation failure is the silent one that runs wrong for three weeks.
The threat specific to AI systems
Prompt injection deserves its own mention. If your automation reads untrusted content — customer emails, uploaded documents, web pages — that content can contain instructions aimed at the model.
Mitigations: treat retrieved content as data rather than instructions, keep tool permissions tight so a hijacked agent cannot do much, require approval for consequential actions, and test adversarially before launch. Ask any vendor whether they have considered this. It is a fast way to find out how current their thinking is.
Questions to put to a vendor directly
- Which model provider and endpoint tier, and does it train on our data?
- Will you sign our NDA, and a BAA or DPA if we need one?
- What permissions does this need, and what is the least it could work with?
- Show me the audit log format.
- What happens if this automation misbehaves at 2am?
- Who holds the credentials after handover?
Frequently asked questions
Is AI automation safe for regulated industries?
It can be, with the controls above and honest scoping. Healthcare, legal and financial work all have specific obligations that change the design — raise them on the first call. Any vendor who waves this away is one to avoid.
Can we keep everything on our own infrastructure?
Largely, yes. Self-hosted automation plus a model endpoint with a no-training guarantee covers most requirements. Fully air-gapped needs open-weight models running locally, which is possible but a different scope of project.
Does more security mean a slower build?
Marginally. Scoped permissions, approval gates and audit logging add days, not weeks — and retrofitting them costs far more than building them in. They are standard on our builds rather than an upgrade.
What is the most commonly skipped control?
Least-privilege access, by a distance. Giving the automation admin rights is quicker and nobody notices until something goes wrong. It is also the one that turns a small incident into a large one.
Building something that touches sensitive data? Tell us your constraints on the first call and we will scope around them — or tell you honestly if we are not the right fit. See how we build agents or message us on WhatsApp.
