Dashboards - Deprecated Tests showing up in Automation Ratios
complete
Tamara McNelly
Given a test case has been deprecated
When using certain widgets (Such as Automation Ratio)
Then the deprecated test cases should NOT be factored into the numbers
Scenario: We have the automation widget on dashboards - when the project has deprecated test cases that were manual it's skewing our automation ratio number - making it look lower than it should be.
Ilya Volkov
complete
Ilya Volkov
Hi Tamara McNelly and Derek Knox!
We are aware of the problem, and have already taken the steps to resolve this.
For some of our widgets, we have already added the additional filters.
For example, if you add a "Single Value"-type widget "Test cases automation ratio", it can be filtered by "Status", to show only the ratio for cases that are "Actual" (see screenshots).
Besides that, filters will also be added to the "Time series"-type widgets down the line as well, so it will also be possible to filter by status on these.
And one more idea you can look into implementing to combine with the "Single value" widget for automation ratio - if you wanted to dig deeper into the actual cases - is the QQL-based widget which has been added recently too. You can prepare and save a query that would look up the cases that are "Automated" and in actual status, like on my screenshot #5. Once it's been saved, you can add a dashboard widget of "Table"-type, "QQL", which would show you the feed of the cases collected by a query, like on my screenshot #6.
Here are some additional resources about QQL and saved queries:
Hope this helps!
Derek Knox
Ilya Volkov Thanks! We totally missed that this option was added.
Derek Knox
Ilya Volkov, this is a major bug for our team. Our leadership loves to use the automation ratio in his reports to top-level executives. We really need it to be accurate and to not count Deprecated tests.