Most API tutorials assume something will call you back. You send a request, a webhook fires, your handler reacts. Then you move to bulk work — ten thousand product photos to cut out, four hundred hours of audio to transcribe, a stack of PDFs to convert — and the doorbell never rings. The job runs somewhere you cannot see, on hardware that may not have woken up yet, and the only signal you get is the one you go looking for.
That is not a downgrade. Pull-based batch work is easier to reason about than webhook fan-out once you accept three facts: you own the polling loop, the queue is where the money gets decided, and cold start is a fixed cost you amortise rather than avoid. Everything else is bookkeeping.
Polling Is the Interface When Nothing Calls You Back
Without webhooks, the contract collapses to four moves: upload, submit, poll, download. Submit returns a handle — a batch identifier — and from that moment your client is a reader, not a listener. Every state change is something you discover on your own schedule, which means your schedule is the system’s latency floor. Poll every sixty seconds and no result can reach your downstream pipeline faster than a minute, however quickly the model ran.
The shape of the batch API matters here. On Y-PAD the sequence is upload, an optional dry run to validate items, submit, then poll until the batch settles and download the outputs. The Ai-api reference documents that flow and the JSON shapes, which are deliberately modelled on Replicate’s so existing client code ports with modest edits. Two details bite operators first: upload links are valid for one hour, and finished outputs are retained for seven days. A poller that submits a batch and checks back next week will find nothing to download.
Polling rhythm is the main tuning knob. Exponential backoff with jitter is the safe default — start at a few seconds, double to a ceiling of thirty to sixty seconds, stop at a terminal state. Jitter matters so a fleet of workers that submitted at the same moment does not synchronise into a thundering herd against the status endpoint. For small interactive jobs a tighter interval is fine; for overnight image pipelines, a two-minute interval costs nothing and saves a lot of noise.
Per-item state is the part people forget. A batch is not a single boolean. Items succeed, fail or get cancelled independently, and only successful items are charged — failed and cancelled items cost nothing. If your poller records only “batch done”, you lose the information needed to retry intelligently and you re-run work you already paid for. Persist the batch ID and per-item results in durable storage, not process memory, so a restart mid-poll does not orphan a half-finished batch.
Queues, Holds and the Economics of Batch Shape
Submitting a batch is not only a scheduling act; it is a pricing act. Y-PAD places a hold at submit equal to the sum of each item’s maximum units multiplied by the list price, then settles against what actually succeeded — launch price if you hold a launch seat, list price otherwise — and returns any unused hold. The consequence is that batch shape is a balance-sheet decision. Declare generous maximums across a large batch and you tie up prepaid balance in holds while the queue drains; declare too little and items may not fit.
Unit minimums push in the other direction. Whisper large-v3 bills by the second with a thirty-second minimum per file, Qwen3-TTS has a 500-character minimum, and DeepSeek V4.1 Flash has a 100-token minimum. Splitting one long recording into fifty five-second clips does not divide the cost by fifty — it multiplies the minimum fifty times. Conversely, since every request gets batch pricing with no minimum charge and one file costs the same per unit as a thousand, packing work together is essentially free. The practical rule: batch by unit economics, not by convenience. Group items whose cost is dominated by a minimum, and keep them whole.
Then there is the retry economy. Because failed and cancelled items are never charged, an aggressive retry policy is cheap in money and expensive only in wall-clock time and queue slots. That argues for smaller batches than the API maximum — up to 2,000 items per batch and 100 files per upload call — since a batch is the smallest unit you can retry without resubmitting everything. One thousand-image batch that stalls near the end forces an awkward decision; ten batches of a hundred isolate the damage.
Treat every fare as provisional. Published beta prices carry explicit check dates, so confirm current figures on the live pricing page before you size a budget. The mechanism — hold at submit, settle on success, return the remainder — is the durable part; the numbers are not.
Cold Starts: Designing for Seconds You Do Not Control
A cold start is the gap between “queued” and “a worker is actually processing this”. Expect ten to thirty seconds, occasionally minutes; nothing tighter is promised. It is less a bug than the arithmetic of on-demand capacity. The useful detail is that later items in the same batch ride on the same start — once a worker is warm, the remainder of the batch flows behind it. Cold start is therefore largely a per-batch cost, which sharpens the tension from the previous section: the batch size that isolates failures also multiplies wake-ups.
That tension is the whole design problem. Ten batches of a hundred images give clean retries and ten cold starts. One batch of a thousand gives a single cold start and a coarser failure unit. The answer depends on item duration: if each item is a sub-second upscale, cold start dominates and you should pack; if each item is a ten-second video segment, cold start is noise and you should split. Measure it — log time from submit to first item completing, and from first to last — and you will know which regime you are in within a day.
Execution location matters for reasons beyond latency. Video models such as Seedance 2.5, Pruna P-Video and WAN 2.6 run on Runpod’s hosted endpoints rather than Y-PAD’s own GPUs, and DeepSeek V4.1 Flash runs on Google Cloud Vertex AI, where the prompt file leaves Y-PAD storage and is sent to Google for each run. If your data cannot cross a given boundary, that routing detail belongs in architecture review, not in a postmortem. If it can, the practical effect is that wake-up behaviour and throughput differ by model family, so an interval tuned for image upscaling may be far too slow for text classification.
A Runbook That Survives Restarts
Persist the batch ID and every item ID before polling begins, and never keep the only copy in memory. Poll with backoff and jitter to a fixed ceiling. Record per-item terminal states so retries target only failures and cancellations. Download outputs well inside the seven-day retention window and refresh any upload links you still need inside the one-hour window. Alert on batch age — “this batch has been open for ninety minutes” — rather than on individual poll responses, because a slow batch and a stalled batch look identical from a single call. Finally, budget cold start into your timeouts: a client that gives up after fifteen seconds will abandon healthy batches, and because submission already incurred a hold, it will do so repeatedly.
