Some work doesn't need its answer right now: last night's inbox can be classified by the morning. For work like that there is the Batch API. You hand in many requests at once, they are processed in the background, and you pay half the price.
A batch is a list of requests. Each request has a custom_id, a name you choose, and params, the same arguments you would give messages.create. A batch that translates two product names could hold these two requests:
{"custom_id": "teapot", "params": {"model": ..., "messages": [...]}}
{"custom_id": "lamp", "params": {"model": ..., "messages": [...]}}
A batch has three moments. batches.create hands it in and answers at once, with an id and the status in_progress: the work has only been accepted. batches.retrieve tells you how it is going, so you ask from time to time until the status is ended. This is polling. Then batches.results gives one result for each request.
The results may come back in any order, and every request succeeds or fails on its own. The custom id is the only link between an answer and its message.
When your program is done, it prints:
Batch: msgbatch_01 in_progress Status: in_progress Status: ended m1 refund: I want my money back for the lamp. It arrived broken. m2 shipping: Where is my package? I ordered ten days ago. m3 product: Does the teapot work on an induction stove? m4 other: Hi! Do you have a job for a student this summer? 10,000 inboxes like this, one by one: 1.72 10,000 inboxes like this, as batches: 0.86
Your task: the inbox, the prompt and the example prices are already in the editor. Put the four messages into one batch, hand it in, wait until it has ended, keep the replies by custom id, print each label next to its message, and compute what 10,000 inboxes like this cost one by one and as batches.
Assignment
Follow the Coding Tutorial to classify the inbox with one batch and to compare the two prices.
Hint
Don't count on the order of the results: store each reply in replies under item.custom_id, then read them in the inbox's own order. For the cost, put the sum in parentheses before dividing by 1000000.
Free plan: 30 tutor messages. PRO is unlimited.
