Turn a bug's failing run into a regression check
Make the failure behind a bug a check of its own; when it passes again the card closes itself, and the check stays to guard against the bug coming back.
In short
A bug filed from a failing run can test itself. "Make this a regression check" copies the failing API request (masked) into the project's regression suite - or, for UI Tester, uses the track itself. When the check passes again on the shared environment, the card moves to done with the evidence, and the check stays as a guard.
Before you start
- Who: you need "Change cards" on the card and Edit on the API Tester project. A locked card only by the board's managers.
- Plan: Boards on every plan, and API Tester or UI Tester for the check to run.
- First: the bug card links a failing API Tester run of a saved request, or a UI Tester run (see /help/boards/links-and-checks).
Steps
- Open the bug card and, in "Related", find its linked failing run under "Links and tests".
- Press "Make this a regression check".
- Check the line "Regression check: ..." on the card.
- In the board's rules, turn on "Move cards when the work happens" if the card says so.
Result: the guard row is marked "Regression check". When it passes again, the card closes itself with the evidence.
Good to know
- The copy leaves out sensitive headers and masks query values and the body. Its authentication stays the project's.
- API checks go into one regression suite per project, in the group "Regression checks". Making it twice changes nothing.
- A passing guard never counts as evidence by itself: it starts a run of the card's linked tests, and the card moves only when they all pass.
- Nothing closes while the recipe is off, the card is locked or it is already done.
- Making a check needs Contribute on the card and Edit on the API project; a locked card only by its board's managers.
- What happens next: the check is kept in the project's regression suite and keeps running with it. The card closes itself only when its linked tests all pass.
- To undo: delete the request from the "Regression checks" group in API Tester. Making the check again changes nothing.
If something goes wrong
- "Name the failed run this regression check guards against." - the card has no failing run linked.
- "That kind of record cannot become a regression check." - only API Tester runs of saved requests and UI Tester runs can.
- The card does not close: turn on "Move cards when the work happens". See /help/boards/cards-follow-the-work.
Related
- /help/boards/bug-triage
- /help/boards/cards-follow-the-work
- /help/api-testing/flaky-tests