Inconsistent Test Run Wizard behavior for complete test runs
under review
T
Taber Hust
Issue
If a test run is completed, but the setting "Allow to add results for cases in closed runs", the wizard behaves inconsistently. If I click on a test that is still in "Untested" status, the wizard opens and allows me to add results to that test case. However, after submitting results to that test case, the wizard moves to the next Untested test case in a read-only view, forcing the tester to close the wizard and then reopen the wizard for each and every test case.
Why wouldn't the wizard consistently allow updating test results for all Untested tests?
Work around
You can mark a single (or multiple) tests to "Rerun" which will re-open the whole test run; however, this seems unintuitive for cases where you don't need to retest, you just need to reopen the test run to complete what was left Untested when it was closed
Request
Can we either A) add a button to "Reopen" a test run rather than forcing a "Retest" action or B) update the wizard to be consistent when navigating to the next Untested test case automatically?
Background
Our team operates in a hybrid automation/manual test run flow (using Playwright). We originally set
complete: false
in the playwright reporter, since our test plans contain manual test cases that must be executed by the team prior to completing a test run.However, for teams that very frequently integrate code (good CI), they do not always complete the manual test case portions of every single test run that gets generated by the automated tests. Therefore, we'd like for the playwright reporter to auto-complete the test runs that are generated, and then only on the "latest" iteration for a release, the team can complete the manual portions of the test run. The current behavior of the wizard makes this experience clunky though.
T
Taber Hust
Thanks Enyindah Okwakpam for looking into this!
I think this part from your comment is my point... "The only instance where cases automatically progress to the next untested one is when the run is still in progress. For a closed run, I have to navigate to each case individually."
That aspect of the wizard is what's confusing. If I have enabled adding/updating test results for completed runs for my project, why must I individually go into each and every test case. It's a very weird experience to close a test case, and for the wizard to move automatically to another test case that its readonly mode. Then I have to close and reopen that test case to make an update. Can the wizard not just automatically move to the next test and allow results to be added (only if we've enabled that setting).
Again, another option maybe is just allow us to "Reopen" the test run via a simple button. But currently the experience is just strange.
Manjunatha Reddy
under review
Enyindah Okwakpam
I am sorry to read that you are having this issue.
I tested it on my end but couldn’t reproduce the problem. I tried it with a manual run successfully and also in the video attached, I tried it by creating an automated test run from a test plan. The automated run was completed and then I was able to manually assign a status to the untested cases with no issues.
The only instance where cases automatically progress to the next untested one is when the run is still in progress. For a closed run, I have to navigate to each case individually.
