Make it Impossible for AI to Delete Your Database Backups

^{This article is part of the "move fast and brace things" series.}

A few months ago, this article on X went viral: An AI Agent Just Destroyed Our Production Data. It Confessed in Writing.

These two paragraphs were the money shot:

Yesterday afternoon, an AI coding agent — Cursor running Anthropic's flagship Claude Opus 4.6 — deleted our production database and all volume-level backups in a single API call to Railway, our infrastructure provider.

It took 9 seconds.

Now, as someone who is rewriting our nearly 30-year-old Microsoft Access tax collection program as a web app...using Claude Opus...and hosting it on Railway...this got my attention.

I vowed to make sure this could never happen to me.

Write-Only Backups that Can't Be Deleted

In researching my options, I came across this nifty little feature in AWS Storage: Object Lock. More specifically, compliance mode:

In compliance mode, a protected object version can't be overwritten or deleted by any user, including the root user in your AWS account. When an object is locked in compliance mode, its retention mode can't be changed, and its retention period can't be shortened. Compliance mode helps ensure that an object version can't be overwritten or deleted for the duration of the retention period.

It is a Write-Once, Read-Many model. Backups go in. You retrieve them as needed. But you can never force-delete them–even with your own root user (i.e., superadmin) account.

Managing the Write-only Backup Lifecycle

You do have the option of keeping the data forever, but that's probably a bad idea. While AWS storage is relatively cheap, it will add up if you keep saving backups that you can never delete.

You manage your backup lifecycles by setting retention periods on individual files when you save them or at the AWS "bucket" (i.e., folder) level.

A common practice is some version of the GFS model (Grandfather, Father, Son):

• A daily backup that you keep for 2 weeks
• A weekly backup that you keep for 5 weeks
• A monthly backup that you keep for 13 months
• An annual backup that you keep for 7 years

You get the idea.

Public Key Encryption for Added Security

Amazon Web Services stores files in their S3 cloud storage platform encrypted at rest (server-side) by default.

That's all well and good, but I'll still be doing my own encryption client-side, thank you very much.

For client-side encryption, I use the AGE library:

age is a simple, modern and secure file encryption tool, format, and Go library.

It features small explicit keys, post-quantum support, no config options, and UNIX-style composability.

AGE stands for Actually Good Encryption. It's similar to PGP ("Pretty Good Privacy"), but it simplifies your life by choosing sensible and secure defaults.

To use it, you start by creating a public-private key pair. The public key is used to encrypt your backup file. The private key is used to decrypt it. The nice part about this arrangement is that you can safely deploy the public key to whatever production server is generating the backups; it's not sensitive on its own. The private key is sensitive. Store it in a secure password vault like Keepass (my choice for all of my personal passwords) and/or Bitwarden (where we store secrets that need to be shared among members of our team).

Least Privileged Access for Backup Writers and Your Customers

Don't get lazy with your security hygiene when it comes to AWS access.

AWS has comprehensive support for proper access control via their Identity and Access Management platform.

Root User

When you first set up your AWS account, you need to create a root user. You should rarely use this account. This is essentially a superadmin that can do everything, including deleting the account itself (the only way to delete those otherwise "undelete-able" backups we created via Compliance Lock above).

NEVER give an AI agent access to this account!!!!

Day-to-day Admin

The first account you set up after creating an AWS account should be a "day-to-day admin" account. If I'm logged into the AWS web console, this is typically the account I use.

Fit-for-purpose Accounts

This next category is a list of accounts that you should set up assuming that they will one day be driven by an enthusiastically helpful AI agent that will do everything it can to deliver on whatever is asked of it.

If you keep that framing in mind, it will help you remember to keep the actual permissions you grant it as minimal as possible.

• Backup Writer: an account with write-only permission to a specific customer's S3 storage bucket
• Backup Reader: an account with read-only permission to a specific customer's S3 storage bucket

These accounts should generally be separated by both function and client. So you would have AWS IAM accounts like:

• acme-backup-writer
• acme-backup-reader
• zenith-backup-writer
• zenith-backup-reader

Backup Writer

Do NOT give the credentials for the Backup Writer account to your client; they don't need it. You will want to secure those credentials somewhere on the production server in such a way that they can be retrieved when the script's scheduled task runs, but not in plain text if you can help it. The Windows Credential Manager is a good option on Windows.

Backup Reader

These are the credentials you give to your client (plus a copy for yourself) so that they can have access to their own data. I suppose you could hold your client's data hostage, but that just gives all software providers a bad name. So don't do that.

One of the nice things about keeping these users tightly scoped is that if any one of them has their password or account compromised, the blast radius is very small.

Don't Forget to Do Restores!!!

If you (and/or your client) are not regularly restoring your backups, then you don't have backups–you have hopes and dreams.

And if the first time you find out your backups aren't working is while reacting to a real emergency, then all you have is a nightmare.

^{*All text in this article written by a 100%-certified free-range human.}