Start learning free

LESSON 105 OF 129 · AI ENGINEERING: PRODUCTION

Cache Repeated Questions in Python

Practice

Every call to a model costs tokens and makes a customer wait. In a shop, many customers ask the same thing. A cache is a place where you keep answers you already paid for, so the next customer with the same question gets the answer without a new model call.

A dict as a cache: the key is what was asked, the value is the answer. Before the slow work, you look in the dict. Here is the idea on another value, a weather lookup:

weather = {}
weather["paris"] = "Sunny, 21 degrees"

print("paris" in weather)   # True: use the saved value
print("Paris" in weather)   # False: a key must match exactly

The second print is the catch. People never type a question the same way twice: capital letters, a missing question mark, extra spaces. So you normalize the question before you use it as a key: make it lowercase, remove the punctuation, and keep one space between the words. Then every way of typing it gives one key.

Six customers write to the shop, and five of them ask about shipping to Canada. When your program is done, it prints this:

Do you ship to Canada? -> Yes, we ship to Canada. It costs a flat $15 and takes 7 to 10 business days.
How long is the warranty? -> Lamps have a 2-year warranty, teapots and mugs a 1-year warranty.
Do you ship to Canada? -> Yes, we ship to Canada. It costs a flat $15 and takes 7 to 10 business days.
do you ship to canada? -> Yes, we ship to Canada. It costs a flat $15 and takes 7 to 10 business days.
Do you ship to Canada -> Yes, we ship to Canada. It costs a flat $15 and takes 7 to 10 business days.
Do you ship   to Canada? -> Yes, we ship to Canada. It costs a flat $15 and takes 7 to 10 business days.
Questions: 6
Model calls: 2

The check then adds one line with the number of requests the course model really received: 2.

Your task: the program already sends every question to the model and counts the calls. Add a cache dict, save each answer under a key made by normalize(question), return the saved answer when the key is already in the cache, and make normalize ignore capital letters, the marks ? ! . , and extra spaces. Every customer still gets the answer, but the model is called only twice.


Assignment
Follow the Coding Tutorial to add the cache and to build normalize one rule at a time.


Hint
Look in the cache before the model call and store the answer after it. Run the program after each rule you add to normalize: the number of model calls goes 5, 4, 3, 2.


STEP 1 OF 10
    Ctrl + Enter to check

    Ai Cache Questions in Python

    Keep going with Python1,600+ step-by-step lessons, from your first line of code to coding interviews. Free to start.
    Start learning free