Maximum Errors Logged
By default you get first row that failed the test. Use this setting to get more.
What This Setting Means
CAT will compare the sets for you - in fact, you don’t need to care about this setting. When CAT finds the first non-matching row, it ends the evaluation and marks the test as failed.
Problem might be that this may lead to an undesired “circle”
- run the test and check the error
- provide a fix
- run the test again just to see it still fails on another row…
In these cases, it makes sense to instruct CAT to continue the comparison of the sets despite first error was already found and CAT already knows the test will fail. Simply said, detailed error message is more valuable than performance.
Where to Set it
As part of test definition, you can specify Maximum Errors Logged. This setting influences how many error rows you will see in the error message. It is also crucial for performance. Because of that, the setting is only on “test level”, you cannot set it globally.
The setting is supported regardless you store your test defintions in your project file, in a relational database, MS Excel sheets or anywhere else. Just provide a column named Maximum Errors Logged.
Allowed Values
| Maximum Errors Logged | Behavior |
|---|---|
| 0 | No data will be logged anywhere. Comparison of the two sets ends immediately when it spots first difference. |
| 1 | First error is reported to the log. Comparison ends immediately when it finds first difference. |
| between 2 and 50 | Comparsion continues after first non-matching value is found, despite the fact CAT knows the test result will be failed. Comparison ends when at least the specified number of errors is found. |
| 51 or higher | CAT reports a warning to logs and continues as if the setting was set to 50. |
Default value is 1. Minimum is 0. Maximum is 50, higher values are treated as 50.
Because of the default comparison algorithm specifics, you may get slightly more errors in the error message compared to what you specified in the Maximum Errors Logged setting. This has no additional peformance penalty.
Performance Impact
Remember the gain (you get more rows that cause the test to fail) is not for free. This setting has impoact on performance. Use it wisely.
Imagine two sets with a few milions of rows, with difference on first and on the last rows only. By default, CAT completes the test in no time. When you raise Maximum Errors Logged, in this case the execution time would be much longer, because additional millions of rows need to be read and examined.
Settings 1 or 0 lead to best possible performance. Use other values only when you need to work with the found errorenous rows and you don’t have other easy way to get them. When troubleshooting performance, review these settings and lowering them will lead to faster tests execution.
I Don’t Want ANY Data in Logs!
Then just set the Maximum Errors Logged to 0 for in all test definitons.