AI agents at work: the database wipe and the permissions problem

In April, a small software company that serves car rental businesses watched an AI agent delete its production database, and the backups with it, in about nine seconds. The data was restored. The lesson didn't go away. As AI tools move from answering questions to taking actions inside your accounts, the most important question stops being "how smart is it?" and becomes "what is it allowed to touch?"

What happened

According to ABC7 News, PocketOS founder Jeremy Crane said a Cursor AI coding agent was doing a routine job in a test environment when it ran into a credential mismatch. On its own, it deleted a storage volume at the company's hosting provider, Railway. That wiped the live database and the volume's backups, including about three months of reservations and customer signups. Crane said the agent later acknowledged it had guessed the deletion would only affect test data and hadn't checked. Railway restored the data about 30 minutes after Crane reached its CEO, who described the agent as a "rogue customer AI" holding a fully permissioned access token. Railway has since added delays on backup deletion and patched the old endpoint the agent used.

This isn't a one-off. The OECD's AI Incidents Monitor logged an entry in March on businesses whose AI agents, including OpenClaw, deleted user data and made unauthorized purchases. The cited reports, mostly from Taiwan, include unauthorized card charges and deleted emails. The monitor attributes the incidents to excessive permissions.

Meanwhile, use is climbing fast. In June, a Stack Overflow survey of 1,100 developers and tech professionals, reported by IT Brief, found workplace AI agent use had risen to 59%, from 31% a year earlier, with daily use up from 14% to 37%. Most respondents are cautious, though: 63% rarely or never let agents finish a task without a human stepping in, and 60% block agents from making unapproved system changes. That sample is technical people, who tend to be early adopters and also the most aware of the risks.

Why it matters for your business

Agents are arriving in ordinary tools. You don't need to hire a developer to run an agent anymore. Email assistants, bookkeeping add-ons, browser helpers and scheduling tools increasingly offer to do the task for you. Each one asks for access to something.

The failure wasn't intelligence, it was access. In the PocketOS case, both the founder and the hosting company pointed at permissions and missing safeguards. The agent did something a human with the same keys could also have done by mistake. The difference is speed and the absence of a pause to double-check.

Small businesses often run with master keys. In many small firms, one admin login does everything, and backups live in the same place as the data. That's the setup that turns an agent's mistake into a disaster.

Purchases and deletions are the danger zone. Reading and drafting are low risk. Spending money, deleting records and sending messages on your behalf are not.

The other side deserves a fair hearing. Agents are improving quickly, and the people using them every day report real time savings. Refusing them outright means doing by hand what competitors automate. The answer isn't no. It's narrower access.

What we'd do

• Give each agent its own login with the least access it needs. Never hand an AI tool your owner or admin account.
• Separate read from write. Let agents read and draft freely. Require a human click for anything that deletes, pays or sends.
• Keep backups somewhere else. Store at least one backup the agent can't reach, and test that you can restore it.
• Turn on spending limits. Use a separate card with a low limit for any tool that can buy things.
• Keep a log. Choose tools that record what the agent did, so you can see and undo it.
• Start in a sandbox. Test new agents on copies of your data before you point them at the real thing.

We like AI agents, and we think most businesses will use them within a few years. But the PocketOS story is a cheap lesson you get to learn from someone else. Treat an AI agent like a new hire on day one: helpful, fast, and not yet trusted with the keys to the safe. Set the permissions before you need them, because by the time you notice, it may have taken nine seconds.