Bean & Bark: the bad Friday deploy
Friday, 4:44pm: revision 4 ships the new wholesale tier pricing. Friday, 4:52pm: Nadia's phone lights up — every order total on the storefront has doubled. The deploy itself was flawless; the code is wrong. This is the chapter the ledger has been preparing you for: the worst-day path, done calmly, in two commands — and the one thing a rollback will not fix for you.
- Read the ledger under pressure with
kanject aws deployments list - Roll a stage back with
rollback --to previous-stable— and know the four forms--toaccepts - Say exactly what a rollback changes on AWS (one thing) and what it never touches
- Mark a revision
brokenso no future rollback lands on it
Read the ledger first
Resist the urge to hotfix. First question on a bad deploy is never "what's wrong with the code?" — it's "what was the last version that worked?" The ledger has been answering that all week:
kanject aws deployments list --env dev --limit 5── Deployments stage dev ────────────────────────────────────────────────────────────╭──────────┬────────┬──────────────────────────────┬─────────┬──────────┬────────────╮│ Revision │ Target │ Deployed │ Commit │ Status │ By │├──────────┼────────┼──────────────────────────────┼─────────┼──────────┼────────────┤│ 4 │ Lambda │ 2026-07-10T16:44:08.1172304Z │ d8e91f0 │ ✓ stable │ you@laptop ││ 3 │ Lambda │ 2026-07-08T09:12:51.6620915Z │ 2b41c77 │ ✓ stable │ you@laptop ││ 2 │ Lambda │ 2026-07-07T11:37:26.0493318Z │ 6c03aa1 │ ✓ stable │ you@laptop ││ 1 │ Lambda │ 2026-07-06T09:26:14.5821930Z │ 9f3c1ab │ ✓ stable │ you@laptop │╰──────────┴────────┴──────────────────────────────┴─────────┴──────────┴────────────╯There's the week in four rows, timestamps in UTC as recorded: Monday's first deploy, Tuesday's config wiring, Wednesday's connection-string fix, and tonight's revision 4. Revision 3 ran clean for two days — that's the target.
Un-ship it
kanject aws rollback --env dev --to previous-stable --yesBefore you run it — a guess. How long does this take? There's no build step in the answer: rollback doesn't rebuild anything. Every deployed revision is an immutable, already-published Lambda version sitting in AWS, with the environment it was deployed with baked in. A rollback checks that version 3 still exists, then flips the live alias from version 4 to version 3. Seconds, not minutes — which is exactly what you want at 4:53pm on a Friday.
── Rollback plan ───────────────────────────────────────────────────────────────╭────────┬─────────────────────────────────────────────────────────────────╮│ Stack │ dev-beanandbark-orders ││ Target │ Lambda ││ From │ 4 ││ To │ 3 (2b41c77) ││ Method │ target-native promotion (drift-free; infra ownership preserved) │╰────────┴─────────────────────────────────────────────────────────────────╯ ✓ Live Lambda revision is now 3.--to takes four forms: previous (the newest revision other than the live one, whatever its status), previous-stable (the newest one not marked broken — the one to reach for by default), a commit SHA or its prefix like 2b41c77, or a Lambda version number. In a script or CI it must be explicit — a non-interactive run refuses to guess, and won't change anything without --yes. And the ledger stays honest: the rollback appends a new entry for revision 3, with a (rollback) suffix in its By column; revision 4 is not erased, because history that can be rewritten isn't history.
Leave a warning for your future self
Revision 4 still says ✓ stable in the ledger — and some future 4:52pm, someone running rollback --to previous-stable could land right back on it. Fix the record while the context is fresh:
kanject aws deployments mark 4 --status broken --env dev ✓ Marked revision 4 as broken.From now on previous-stable skips revision 4 for good. The status column is the institutional memory of which deploys betrayed you — two keystrokes now, one avoided incident later. (The mark rewrites revision 4's ledger entry, so its row moves to the top of deployments list, which orders rows by when each entry was last written. Its Deployed time doesn't change.)
Rehearse it while nothing is on fire
The reason tonight took two commands and no adrenaline is that you've done it before. Make that true: after a healthy deploy to dev, spend sixty seconds flipping back and forward once. Use previous-stable here too — after Monday's fix ships as revision 5, plain --to previous would pick broken revision 4, and with --yes it wouldn't even ask. --dry-run prints the plan without changing anything.
# The 60-second drill — on dev, after a healthy deploy (say revision 5):# back to the last good revision, then forward to the one you leftkanject aws rollback --env dev --to previous-stable --yeskanject aws rollback --env dev --to 5 --yes- Start every incident at the ledger:
deployments listshows what shipped, when, by whom, and what's still trusted. rollback --to previous-stableflips thelivealias to an already-published version — no rebuild, seconds to run — and appends a new ledger entry with a(rollback)suffix.- A rollback changes only the alias: the version brings its own code and env, but database rows, migrations, S3 objects, and queue contracts stay as the bad deploy left them.
deployments mark <rev> --status brokenmakesprevious-stableskip that revision. Mark while the context is fresh — plainpreviousignores the mark.
broken, where does rollback --to previous-stable land?