Quiz, Model APIs

Choose an answer for each question. Some questions have more than one right answer.

Question 1 of 14

A program classifies the comments of riders. The rules for the labels are the same for each call. Which message holds the rules?

A message with the role user.A message with the role system.A message with the role assistant.A message with the role rules.

Question 2 of 14

Which statements about tokens are true? Choose all that apply.

Choose all answers that apply.

A token is a short piece of text, such as a word, a part of a word, or a punctuation mark.Each service has its own rules for how it cuts text into tokens.You pay for the input tokens only.The context window of a model limits the number of tokens of one call.

Question 3 of 14

A response has this stop reason. What does it tell you?

response["stop_reason"]   # "max_tokens"

The model finished its answer.The messages had more tokens than the context window.The answer reached the limit max_tokens, and the service cut it.The key has no tokens left.

Question 4 of 14

A call uses 4,000 input tokens and 1,000 output tokens. The input price is 3 dollars for 1,000,000 tokens, and the output price is 12 dollars for 1,000,000 tokens. What does the call cost?

0.012 dollars0.015 dollars0.024 dollars0.075 dollars

Question 5 of 14

Why does the model not know the name in the second call?

client.chat([{"role": "user", "content": "My name is Rosa."}])
client.chat([{"role": "user", "content": "What is my name?"}])

The second call must use a higher temperature.The model sees only the messages of one call. The second call does not have the first message.The first call was too short to store.The model stores names only when a system message tells it to.

Question 6 of 14

A chat program sends the complete history with each call. What happens in a long conversation? Choose all that apply.

Choose all answers that apply.

Each call has more input tokens than the call before, so it costs more.After many turns, the history does not fit the context window.The model gets faster, because it knows the text already.The output price goes down.

Question 7 of 14

A program compares the answer of the model with the text "complaint". Which setting and which step make the comparison reliable?

temperature=1, and no other step.temperature=0, and the program normalizes the answer: no spaces at the ends, no period at the end, lower-case letters.max_tokens=1, and no other step.A longer system message, and no other step.

Question 8 of 14

Which instructions give an answer that a program can read with json.loads most reliably?

Classify the comment.Classify the comment. Answer in JSON.

Classify the comment as one of: praise, complaint, question, other. Answer with only JSON. Use the keys "label" and "reason".

Classify the comment, and be careful with the format.

Question 9 of 14

Your program allows the labels praise, complaint, question and other. The model gives this answer. json.loads reads it without an error. What does the program do?

{"label": "unclear", "reason": "The comment is short."}

It uses the label unclear, because the JSON is valid.It rejects the answer, because the label is not one of the allowed labels. Then it asks again, or it uses a default.It changes the label to praise.It stops and deletes all results.

Question 10 of 14

For which errors can a new try of the same call work? Choose all that apply.

Choose all answers that apply.

RateLimitError: too many calls in a short time.ServerError: the service has a problem.ModelTimeout: the answer did not come in time.AuthError: the key is wrong.BadRequestError: the messages have a wrong form.

Question 11 of 14

A function tries a call up to 4 times. After each failure with a ServerError, it waits for min(2 ** attempt, 30) seconds, and attempt starts at 0. The first three calls fail, and the fourth call works. For how many seconds did the function wait in total?

3 seconds6 seconds7 seconds8 seconds

Question 12 of 14

Where does the API key of a real service belong?

In the code, in a variable with a clear name.In an environment variable. The program reads it with os.environ.In the first message of each call.In the README of the project, so that the team finds it.

Question 13 of 14

A program processes 5,000 comments in a loop, with one call for each comment. Call number 3,200 raises a ServerError, and the loop has no try. What happens?

The loop skips the comment and continues.The program stops. The comments after number 3,200 have no result, and the results before it are lost if the program did not write them to a file.The service repeats the call.Python tries the call again 3 times.

Question 14 of 14

On a test set of 200 comments, a small model gives 172 right labels, and a large model gives 190. The large model costs 12 times more for each call. Which statement is correct?

The large model is always the correct choice, because 95 percent is more than 86 percent.The small model is always the correct choice, because it costs less.The choice depends on what a wrong label costs and on how many calls the job makes. The test set gives you the numbers for the decision.A test set cannot compare two models.

Answer every question to check.