Start learning free

LESSON 108 OF 129 · AI ENGINEERING: PRODUCTION

Cost per Request in Python

Practice

Every call to a model costs money, and you pay per token: one price for the tokens you send in, a higher price for the tokens the model writes out. Each reply reports both counts in reply.usage. The bill at the end of the month is one number, too late to ask which part of your app spent it. So you record the cost of every request when it happens.

The cost of one request: prices are given per million tokens. Say a reply used 200 input tokens at $3.00 and 50 output tokens at $15.00:

200 × 3.00 + 50 × 15.00 = 1350
1350 / 1,000,000 = $0.00135

A total per group: a dict can keep one running total per name. The first time a name shows up its total starts at 0, then you add to it. Here it counts colors:

counts = {}
for color in ["red", "blue", "red"]:
    if color not in counts:
        counts[color] = 0
    counts[color] = counts[color] + 1
# counts is {"red": 2, "blue": 1}

This shop uses the model for two features: support answers customers and descriptions writes product texts. The program makes five requests. When you are done it must print:

support: Do you ship to Canada?
descriptions: Write a product description for the teapot.
support: How long is the warranty on a lamp?
descriptions: Write a product description for the desk lamp.
  WARNING: the day's cost 0.000993 is over the budget of 0.0008
support: Can I pay with a gift card?
  WARNING: the day's cost 0.001135 is over the budget of 0.0008
support 6 in, 29 out, cost 0.000151
descriptions 11 in, 71 out, cost 0.000366
support 9 in, 23 out, cost 0.000124
descriptions 12 in, 68 out, cost 0.000352
support 7 in, 27 out, cost 0.000142
support total: 0.000417
descriptions total: 0.000718
Day total: 0.001135

Your task: the loop that calls the model is already written, and so are the prices, the budget and the empty log, totals and day_cost. Compute the cost of each request and record it in log, print the records, keep and print a total per feature, and keep the day's total, with a warning after every request that leaves it over DAILY_BUDGET.


Assignment
Follow the Coding Tutorial to record the cost of every request, total it per feature and for the day, and warn when the day's cost passes the budget.


Hint
The token counts are usage.input_tokens and usage.output_tokens. Put the sum in parentheses before you divide by 1000000, and print money with round(..., 6). Lines that record or add go inside the first loop; lines that print the log and the totals go after it.


STEP 1 OF 12
    Ctrl + Enter to check

    Ai Cost Per Request in Python

    Keep going with Python1,600+ step-by-step lessons, from your first line of code to coding interviews. Free to start.
    Start learning free