Waiting and flaky tests

The page is not ready when the test is

A real page does not show everything at once. It loads data from a server, and the server takes time. A person waits and looks. A test is faster than a person, so it can look too early.

The starter test counts the todos right after page.goto('/'). At that moment the list has no todos yet, so the count is 0. The test fails with Expected: 3 and Received: 0. A moment later the todos arrive, but the test has stopped.

The cause is a one-time read. count() asks one time, and it gives the answer at that moment. expect(count).toBe(3) compares a number that is old already.

The fix: let the assertion wait

A web-first assertion does not read one time. It reads, compares, and reads again, until the fact is true or the time limit ends. In this practice browser the limit for an assertion is 2 seconds. In a real project it is 5 seconds, and you can change it.

// Wrong: one read, no waiting.
const count = await page.getByTestId('todo-item').count();
expect(count).toBe(3);

// Right: the assertion waits.
await expect(page.getByTestId('todo-item')).toHaveCount(3);

The same rule holds for every value that you take from the page. textContent(), innerText(), inputValue(), and isVisible() read one time. Do not use them to check a fact. Use toHaveText, toHaveValue, and toBeVisible.

Never wait for a fixed time

await page.waitForTimeout(1000) stops the test for one second. It looks like a fix. It is not, for two reasons.

• If the page needs 1.2 seconds on a slow day, the test fails.
• If the page needs 0.1 seconds, the test wastes 0.9 seconds every time. With hundreds of tests, this adds up to minutes.

A test that passes on most days and fails on some days is flaky. Flaky tests are the most expensive kind. People stop trusting them, and they ignore real failures. Fixed waits are the most common cause.

Wait for a fact, not for a time. Ask yourself: what will be true when the page is ready? Then write an assertion for that.

Other ways to wait for a fact

• await expect(locator).toBeHidden() waits until the element is gone. Use it for a loading message.
• await locator.waitFor() waits until the element is there, then goes on.
• await page.waitForURL('/account') waits for the address to change.
• await expect.poll(() => value).toBe(3) repeats a function until its result is right.
• await expect(async () => { ... }).toPass() repeats a block of code until it stops failing.

Actions wait too

An action such as click() waits until the element is visible, stable, enabled, and not covered. You do not need to wait for a button before you click it. You do need to wait for a result, such as a new row, before you check it. Actions return when the click is done, not when the page has reacted.

Your turn

The file has one test that reads the list too early. Fix it. Then write two more tests that wait for the facts.