Every call sets max_tokens. It is a hard limit on the answer: the model writes token after token, and when it reaches the limit it stops, wherever it is. There is no error and no warning. You just get half a sentence.
Your program can't tell a finished answer from a cut one by reading it. It doesn't have to. Every reply says why the model stopped, in reply.stop_reason: "end_turn" means the model finished on its own, and "max_tokens" means it ran into the limit.
An example: a model asked for a notebook slogan could stop in two ways. Only the stop reason tells them apart for sure.
max_tokens=8 "Write it down before the idea" stop reason: max_tokens max_tokens=100 "Write it down before the idea runs away." stop reason: end_turn
A check on the stop reason is a plain if:
if reply.stop_reason == "end_turn":
print("The model finished on its own.")
This lesson's program asks for a description of the Corner Shop's teapot, first with too little room and then with enough. It must print:
Text: This classic white ceramic teapot holds four cups of tea and keeps them warm for Stop reason: max_tokens The answer was cut. Asking again with more room. Text: This classic white ceramic teapot holds four cups of tea and keeps them warm for an hour. It is easy to clean and makes a lovely gift for any tea lover. Stop reason: end_turn
Your task: the client, MODEL and messages are already there. Send messages with max_tokens=20 and print the reply's text and its stop reason. Then, if the stop reason is "max_tokens", print that the answer was cut, ask again with max_tokens=200, and print the new text and the new stop reason.
Assignment
Follow the Coding Tutorial to make a call that gets cut, read its stop reason, and ask again with more room when the answer was cut.
Hint
The second call and its two prints go inside the if, one level deeper. Keep the new result in the same variable, reply, so it replaces the cut one.
Free plan: 30 tutor messages. PRO is unlimited.
