When nothing is waiting on the answer

There are two service levels for AI work. The one you wait on, which is fast and full price, and the bulk lane, which is half the cost and can take up to a day. A surprising amount of work that feels like it has to be immediate is fine on a queue.

Common questions

1. What is the actual trade?
   Half the cost, a processing window of up to 24 hours, and no guarantee about when inside that window. You give up speed and immediacy, and in return you pay half price.
2. How do I decide which lane a job belongs in?
   One question: is anything waiting on this? Nothing waiting, use the bulk lane. Something waiting, pay for the immediate answer.
3. What does "something waiting" look like?
   Work where something downstream is stalled until the answer comes back, like a check a pull request cannot merge without. An overnight report is the opposite, because it does not matter whether it took twenty minutes or six hours.
4. Does it tell me when it is finished?
   No, it is poll-based. You hand the work off and check its status until it reads ended, then collect the results. It does not call you back.
5. What can the bulk lane not do?
   Anything needing a round trip back to you. It does one turn and returns, so a loop where the model asks for a tool, you run it and feed the result back cannot happen inside a single request.
6. So what has to be true of each request?
   It has to be self-contained and answerable in one turn. Anything that completes without pausing is fine, and anything that has to hand control back to you and resume is not.
7. The results came back jumbled. How do I match them up?
   By the id you set on each request when you submit, which comes back on each result. Results arrive in any order, so you can never match by position: answer three does not necessarily go with request three.
8. Some requests failed. Do I resubmit the whole thing?
   Only the failed ones, and only the ones worth retrying. Re-running the successes wastes money and time when you already have their answers.
9. Can I just resend a failed request unchanged?
   Not always. A request that blew the context limit fails again if you resend it as is, so the fix is to split the oversized input into smaller pieces first, then submit those.
10. How do I promise a turnaround to my own users?
    Work backwards. A request waits for the next scheduled submission, then up to 24 hours to process. The submission interval is the lever you control, so pick one where the wait plus 24 stays inside your promise.
11. Anything to do before pressing go on a big run?
    Prove the prompt on a small but representative sample first, covering the variations the real data will contain. A systematic flaw does not show up until hours and full cost later, and the fix means redoing the whole thing.

Back to Ways to run it for real. The craft around it is Working with AI.