Lesson · 12 min

The cost of a call

How a service sets its price

A model service does not charge for a call. It charges for tokens. The price list of a service gives two prices for each model.

The input price is for the tokens that you send: all messages of the call.
The output price is for the tokens that the model writes. It is higher than the input price, often by a factor of 3 to 5.

The prices are for 1,000,000 tokens. The simulated service has this price list. The prices are made up.

Model	Input, for 1,000,000 tokens	Output, for 1,000,000 tokens
`sim-small`	$0.25	$1.00
`sim-large`	$3.00	$12.00

The dictionary MODELS has the same numbers.

from sim_model import MODELS

MODELS["sim-large"]["input_price"]    # 3.0
MODELS["sim-large"]["output_price"]   # 12.0

The price of one call

The response tells you the tokens of the call under the key usage.

usage = {"input_tokens": 2000, "output_tokens": 500}

For the model sim-small, the price of this call is:

2000 * 0.25 + 500 * 1.00 = 1000
1000 / 1,000,000        = 0.001 dollars

One call costs a tenth of a cent. That looks like nothing.

Small prices add up

A program does not make one call. A support tool that classifies 20,000 comments each day makes 600,000 calls in a month. Then the model, the length of the instructions, and the length of the answer decide the bill.

Calculate the cost before you build. You need three numbers: the tokens of a usual call, the price of the model, and the number of calls.

How to lower the cost

Use the smallest model that does the job well. Lesson 13 compares two models on quality and on cost.
Keep the instructions short. The system message goes with each call, so you pay for it each time.
Ask for a short answer. Output tokens cost the most. One word costs less than a sentence.
Set max_tokens. The limit stops an answer that is much longer than you expect.
Do not ask twice. Store the answer for a text. When the same text comes again, use the stored answer.

Your turn

main.py has the instructions and a comment. Do the tasks in order.

Tasks

5 to do

Task 1, not checked yet. Write a function call_cost(usage, model). usage is a dictionary with input_tokens and output_tokens. The function returns the price of the call in dollars. Use the prices of the model in MODELS. They are in dollars for 1,000,000 tokens.
Task 2, not checked yet. Call the model with a system message that has the content SYSTEM, and a user message that has the content comment. Store the dictionary of the token counts in a variable usage.
Task 3, not checked yet. Make a variable cost_small with the price of this call for the model sim-small, and a variable cost_large with the price of the same tokens for the model sim-large. Use your function.
Task 4, not checked yet. Make a variable times_more. It says how many times more the large model costs for this call: cost_large divided by cost_small, rounded to 1 decimal place.
Task 5, not checked yet. Write a function month_cost(usage, model, calls_each_day). It returns the price of 30 days in dollars, when each day has calls_each_day calls with these tokens. Round the result to 2 decimal places. Then print the result for 20,000 calls each day, for each of the two models.

Task 1 of 5

Write a function call_cost(usage, model). usage is a dictionary with input_tokens and output_tokens. The function returns the price of the call in dollars. Use the prices of the model in MODELS. They are in dollars for 1,000,000 tokens.

All tasks