Run Tests in Parallel
How to execute tests simultaneously in more parallel executing threads?
By default, CAT runs tests one by one untill all of them are executed. It does not start execution of a test until the previous one is completely evaluated.
Why Should I Care?
If you have many tests and their execution takes long time, you should consider running tests in parallel. That means CAT will not execute one test, but two or more tests at a given point in time.
Before you blindly tweak the setting, consider this.
Running to many queries against your server(s) in parallel may cause significant performance problems for other users / applications. It is a responsibility of the tester / developer to evaluate the right level of parallelism.
How Do I Set It?
As all other stuff in CAT, you manage the degree of parallelism in your CAT project file:
# multithreading settings: Threads: 3
Put that to the root of your project file - that means there must be no whitespace before the
Setting the value to 1 is the same like deleting the line - it has the same effect. CAT will use only a single thread.
Setting the value to 0 or less will generate an error.
What Does It Do?
If you set
3, like in the example above, CAT will do this:
It creates a queue of tests to be executed (this one is common for all threads).
It starts 3 threads, all of them run independently on each other at the same time.
Each thread “steals” one test from the queue (so the others will not see that one anymore) and evaluates it.
A thread finishes once if finds out the queue is empty.
All results from all threads are assembled together and processed (to MS Excel file, JSON, TRX etc.). This is already done in a single (in the “main”) thread.
If you don’t have any reason to use parallelism, don’t use it.
If your tests run long time, consider trying out the parallelism in “steps”, like first in two threads, then in three…
If you are about to tweak the parallelism, communicate this with your database administrator(s). Let them know you might create a heavier load.
Monitor with your DBA the traffic your tests create on the servers.
If you use
Excel@2provider, beware that every thread first loads all the specified worksheets to memory. Especially if you test worksheets with hundreds of thousands of rows, avoid using parallelism or separate those tests into a dedicated project file that uses one thread.
In log files you can find details about what thread executed how many tests, when tests started etc. This may help you understand what CAT does and when. This is only available in
Debuglogging level. Use it only for troubleshooting or evaluating how CAT executes tests.