Try again with a wait

Wait before a new try

Lesson 9 separated the errors that a new try can fix. For these errors, a program tries the call again. It does not try again at once. A service that has too much work gets more work from each call. A call that comes at once usually fails for the same reason.

So the program waits before each new try, and it waits longer after each failure. This method has the name exponential backoff. The wait doubles each time: 1 second, 2 seconds, 4 seconds, and so on. A maximum stops the wait from growing without limit.

Failed attempt, counted from 0	Wait in seconds, with a maximum of 30
0	1
1	2
2	4
3	8
4	16
5 and more	30

When the service tells you the wait

A rate limit error often tells you how many seconds to wait. Use that number, and not your own. In this course, the number is the attribute retry_after of the RateLimitError.

Stop after some attempts

A loop that tries until the call works can run for hours. Set a maximum number of attempts. When the last attempt fails, let the error pass to the code that called your function. That code decides what happens next.

Inside an except block, the statement raise with nothing after it raises the same error again.

for attempt in range(max_attempts):
    try:
        return client.chat(messages)
    except ServerError:
        if attempt == max_attempts - 1:
            raise
        wait(backoff(attempt))

One except line can name more than one class. Put the classes in parentheses: except (ServerError, ModelTimeout):.

Do not catch an error that a new try cannot fix. A BadRequestError must reach you at once, so that you correct the code.

The wait in a lesson

A real program waits with time.sleep(seconds). A lesson must finish in a few seconds. So the simulation has the function wait(seconds). It does not wait. It appends the seconds to the list waits, and you can read the list.

from sim_model import wait, waits

wait(4)
waits   # [4]

Two facts about real programs

• A random part in the wait. Many programs can fail at the same moment. If all of them wait for the same time, all of them try again at the same moment. So real programs add a small random number of seconds to each wait. This lesson does not, so that the waits are the same on each run.
• The package of the service. The Python packages of many services already try a temporary error again, some times. Read the documentation of your package before you add your own loop. Two loops multiply the number of calls.

Your turn

main.py imports the error classes and the functions plan and wait. Do the tasks in order.