Running Serenity tests in parallel
One of the most common ways to run UI test suites faster is to run them in parallel. There are ways you can make Serenity tests run in parallel, and in this section we will walk through a few of them.
If you are using Maven, the easiest way to run tests in parallel is to use the
maven-failsafe-plugin to run your tests. This feature is well documented but can be complicated to set up. However, when running Serenity tests, a few simple variations generally work well.
One of the simplest ways to run your tests in parallel is to run them in parallel threads. For example, the following (partial) configuration will run your test classes across two threads per CPU core:
<plugin> <artifactId>maven-failsafe-plugin</artifactId> <version>3.0.0-M3</version> <configuration> <!-- Parallel execution in the same JVM --> <parallel>classes</parallel> <threadCount>2</threadCount> </configuration>
parallel configuration element takes several options, including
all. However in practice, the options that work best with Serenity are
Selenium’s ability to scale on a single machine is not unlimited. In practice, the more threads you use, the higher the chances of sporadic, hard-to-reproduce test failures due to timeouts and other related issues. The added performance gain of each thread also tends to drop off for higher numbers of threads. The Maven documentation has this to say: "the concurrency happens within the same JVM process. That is efficient in terms of memory and execution time, but you may be more vulnerable towards race conditions or other unexpected and hard to reproduce behavior".
Another way to run your tests in parallel in Maven is to use Forked Test Execution. Using Forked Test Execution, new JVM processes are spun up to execute your tests, up to a configurable maximum number of processes. This creates better separation between your tests, which can improve their reliability. However the forked JVMs take longer to set up, so there is a minor performance hit.
You tell Maven to spawn new processes by adding the
forkCount configuration element, as shown here below. This can either be a number (the maximum number of forks) or a multiplier (the number of forks per CPU). The example below uses the value of "2C", which means 2 forked processes per CPU:
<plugin> <artifactId>maven-failsafe-plugin</artifactId> <version>3.0.0-M3</version> <configuration> <!-- Parallel execution in the same JVM --> <parallel>classes</parallel> <threadCount>2</threadCount> <forkCount>2C</forkCount> </configuration>
Using multiple forks can be a good alternative to running all tests in a single JVM, and can reduce the risk of certain types of errors. The exact configuration that will work best for your environment will vary, so it is best to experiment with different options to see what gives the best results.