Bean & Bark: bulk work past the preview cap

View .md

In change data without losing it the preview saved you from a wrong WHERE, and the scoped cleanup touched 214 rows. A year later the loyalty-programme import has multiplied the table, and the same — correct — statement now matches tens of thousands. The preview that protected you refuses to even try. That refusal is not a dead end; it's a fork: stream it as a background job, or take the question to the fleet.

You'll learn
  • Understand the interactive preview cap — why a bulk write refuses instead of silently previewing 40,000 rows
  • Run a set-based write as a background job: no preview, the same per-item WHERE re-check, tracked from the footer
  • Know when a BULK SELECT fleet read (Professional) beats scanning through the app's own connection

The preview has a ceiling — on purpose

You already know the ritual: run the scoped statement, read the matched count, run again to apply. But the preview drains every matched row into the grid so you can look at the blast radius, and a human can't meaningfully review 40,000 rows — so the interactive path caps the preview (1,000 items by default; Settings → Querying → Max preview items). Past the cap it refuses outright rather than showing you a set too big to check.

sql
-- same cleanup as last year. the table is not the same table.UPDATE "BeanAndBark"SET Status = 'retired'WHERE begins_with(sk, 'SUB#') AND Status = 'dormant'
Try it · the cleanup at scaleSample
Executes asBulk UPDATE · preview · BeanAndBark
Scan FilterExpression: begins_with(sk, 'SUB#') AND Status = 'dormant' → stopped past 1,000 matched · nothing written
The WHERE is right this time — the SET is scoped to subscriptions. The size is the problem, not the predicate.
The refusal toast carries the way out: a "Run as job" button, right where the refusal lands.
Bulk write won't run — Matched more than 1,000 items — interactive bulk previews are capped for safety. Narrow the WHERE, raise Settings → Querying → Max preview items, or run as a background job. [Run as job]

Three tools, one decision rule. Under the cap, the preview-first ritual stays the default — you look, then you write. Past the cap and still a write, the background job keeps the safety story without the preview: every item is written through an atomic WHERE re-check, so the guarantee moves from "you saw the set" to "each row still matched at the moment it was written". And when the job is really a question — export every dormant record, count them across the whole table — a fleet read does the scanning on workers in your own account instead of through the connection your grid is using.

Recap
  • The interactive preview is capped (1,000 items by default) because a set too big to review isn't a preview — past it, the studio refuses and offers the job path instead.
  • A background job streams the write with no cap and no preview, but keeps the per-item guard: the WHERE is re-checked atomically on every single write, and the footer chip + jobs panel track the run.
  • A BULK SELECT (Professional) runs the scan on a Glue worker fleet in your own account and delivers results to S3 — for table-scale questions that shouldn't ride your interactive connection.
  • Fleet BULK UPDATE / BULK DELETE aren't available yet — today, set-based writes run interactively or as a streaming background job.
Scale check2
Your bulk UPDATE matches 40,000 rows and the preview refuses. What does "Run as job" trade away, and what does it keep?
Nadia wants a CSV of every dormant record for the auditors — the table is millions of items. Best tool?
Try it yourself 2
1 Raise the cap — or don't
The refusal names three ways forward. When is raising Settings → Querying → Max preview items the right one, and when is it the wrong one?
The cap exists so a preview stays reviewable. Ask what number a human can actually check.
Show solution
Raise it when your legitimate working sets sit just past the default — say 1,500 rows you genuinely eyeball. Don't raise it to swallow a 40,000-row set: nobody reviews 40,000 rows, so you'd keep the ceremony of a preview while losing its meaning. Past human scale, take the job path — its per-write guard is honest about what's being checked.
2 Guard-rail the encore
Write the two statements you'd run, in order, to retire this year's dormant subscriptions at scale.
Chapter three's seatbelt, then the scoped write — the size changes the how (job, not preview), never the order.
Show solution
Backup first, then the scoped UPDATE via Run as job. The WHERE is already pinned to subscriptions with the sk prefix; the job re-checks it on every row it writes.
sql
BACKUP TABLE "BeanAndBark" AS 'beanandbark-pre-cleanup-2026'UPDATE "BeanAndBark"SET Status = 'retired'WHERE begins_with(sk, 'SUB#') AND Status = 'dormant'
Was this page helpful?