Lesson · 22 min

Tests that catch bugs

A test that passes proves little

A test can pass for two reasons. The app can be right. Or the test can check too little to notice a problem. When a test passes, you do not know which one it is.

There is a way to find out: break the app on purpose, and run the test again. If the test still passes, it did not look at the thing that you broke. A good test fails when the app is wrong.

How the checker works

The practice browser has sample apps, and each app can have bugs. A bug is a mistake that a developer could make, for example "the delete button removes the first todo, not the one you clicked".

When you press Check my work, the checker does three runs for each task that names a bug.

It runs your tests on the normal app. They must all pass.
It runs them again on the normal app, with no delays. They must still pass. If a test fails here, it depends on a delay of the app, and it is flaky.
It runs them on a copy of the app that has the bug. At least one test must fail.

If your tests still pass on the app with the bug, the message tells you what the bug is, in plain words. Then you must change a test, or add one, so that it looks at the thing that is wrong.

How to write a test that catches a bug

Ask this question: what would be different if the bug were there? Then check that difference.

Take the bug "the app adds a todo that has no text". With the bug, the list gets a fourth row, and the app shows no alert. Your test can check either one. The best test checks the alert and the count, because then it tells the reader exactly what the right behavior is.

Three rules help.

Check the result, not the action. A test that clicks and checks nothing catches nothing.
Use exact values. toHaveText('1 item left') finds a count that does not change. toBeVisible() does not, because the count is visible in both cases.
Check what must stay as well as what must go. A test that checks only the number of todos after a delete cannot tell which todo was deleted.

Your turn

You write three tests. For each one the task names the bug. The checker runs your tests on that bug. If the test does not fail, you see the bug in the message, and you can try again.

Tasks

3 to do

Task 1, not checked yet. Add a test named does not add a blank todo. It must fail on an app that adds a todo with no text. Click Add with an empty box. The app must show the alert Write something first., and the list must still have 3 todos.
Task 2, not checked yet. Add a test named counts the todos that are left. It must fail on an app whose count does not change when you complete a todo. At the start the count shows 2 items left. After you tick Buy milk, it shows 1 item left.
Task 3, not checked yet. Add a test named deletes the todo that was clicked. It must fail on an app that deletes the first todo, not the one you clicked. Delete Write a test. Check that the list has 2 todos, and that Write a test is gone while Buy milk is still there.

Task 1 of 3

Add a test named does not add a blank todo. It must fail on an app that adds a todo with no text. Click Add with an empty box. The app must show the alert Write something first., and the list must still have 3 todos.