Skip to main content

Running Tests

There are two ways to start a run, and they answer different questions.

  • Run Now, on a feature, runs that one test straight away. It answers did I just break this?
  • A test plan runs a whole set of tests, in chosen browsers and locations, possibly on a schedule. It answers is this build good?

Either way the work happens in the cloud, not in your browser. Studio opens your application on its own machines, replays the recorded steps, and records what it sees. You can close the tab; the run carries on.

Watching a run​

Test Runs, in the header, opens the Currently Running panel. It lists every execution in flight, whatever started it — a Run Now, a plan, a schedule that fired overnight. When nothing is running it says so: "There is no active test executions."

There is a companion panel for generation — Test Gen., right next to it — which does the same for negative-test generations. See Generating negative tests.

The three outcomes​

A test ends one of three ways, and the difference between the last two is the one worth internalising.

PassedThe application did what the test expected.
FailedThe application did something else. An assertion did not hold — this is usually a finding about your application.
ErrorThe test could not finish. A page never loaded, an element never appeared, the environment was down — this is usually a finding about the test, or where it ran.

A report full of errors rarely means the application is broken in a hundred different ways. It can indicate that one thing broke early and everything downstream of it never got the chance to run. Read the first error, not the hundredth.

Where the run goes​

Studio keeps the result of every run. When one finishes, read it in Reports: the latest run in detail, all runs as a trend, and — if the plan was set to raise them — the defects it opened.