Find out why a run failed

Press Why did this fail? on a failed run to get what failed, the likely cause and what to try next.

In short

When a run fails, "Why did this fail?" asks the assistant to read what that run kept and compare it with the last run that passed. It answers in three short parts - what failed, the likely cause and what to try next - with its sources. "Create bug" is one click away.

Before you start

• Who: anyone who can open the failed run. A run in another workspace, or someone else's private run, is not found.
• Plan: AI is part of every plan, Free included. On Free the platform's key answers questions about how {{brand}} works. For questions about your own work, add your own key (it is free to add) or choose a paid plan (see /help/ai/answers-left). One explanation uses one of your AI answers.
• First: open a run that failed. A run that passed has no such button.

Steps

1. Open the failed run: an API run, a failed step of a suite, a monitor that is down, a SQL run, a cache check or a UI run.
2. Press "Why did this fail?". The assistant opens and reads what that run kept.
3. Read the three parts: What failed, Likely cause, What to try next. The numbers in the text point at "Sources" under the answer - the failed run, the last run that passed, and the Help guide.
4. Press "Create bug" to file it on a board with the steps and the evidence filled in, or ask a follow-up in the same conversation.

Result: you know what broke, what probably caused it, and what to do next.

Example: Checkout answers 500 on Staging. The answer says the request got status 500 after 1,200 ms. It also says the request last passed yesterday with status 200 in 300 ms, and that it was changed since, so the change is the likely cause. Last, it says to compare the two runs and send the request again.

Good to know

• The button is on the page of a failed API run and on each failed step of a suite run. It is also on the health sheet of a monitor that is down, under a failed SQL result, beside each failed cache check and on a failed UI run. A run that passed has no such button.
• The answer is built from three things. The first is the failed run's own evidence: what was expected, what happened, the status or error, the environment, the time and the steps that repeat it. The second is the last run of the same check that passed. The third is the module's Help guide.
• It reads only that one run, and only if you can open it yourself. A run in another workspace, or someone else's private run, is not found - for everyone, a site admin included.
• Values the tester already masks stay masked. Before anything is sent to the model, anything that looks like a password, a token, a key or a cookie is hidden - in the answer too. Response bodies, cache values and result rows are not sent at all.
• One explanation uses one of your AI answers, like a question. On the Free plan it needs your own key, which is free to add, or a paid plan.

How to read the answer

• What failed repeats facts from the run, word for word.
• Likely cause is a reasoned guess and says so: probably, most likely. Check it before you act on it.
• What to try next is a short list of things you can do yourself.
• "The evidence doesn't show why" is an honest answer. The run did not keep enough to tell, and the answer says what would show it - for example running again with the response kept.

The assistant never makes up a value, a status or a date that is not in the evidence, and never runs or changes anything by itself. Text inside a response or a page cannot give it instructions.

Create a bug, or do something about it

• "Create bug" opens the same dialog as on the run itself: pick the board, and the card gets the steps, the expected and the actual result and the evidence. It works when Boards is on for you.
• "Do something about it" puts a request in the box, already written and to be planned as steps, such as Create a card to fix "Checkout answers 500". Change it if you want, then send it. Nothing happens until you confirm: see /help/ai/ask-or-do.
• Ask a follow-up, for example "What changed between the two runs?". The conversation keeps the explanation.

If something goes wrong

• "No model explained this, but you can still file it as a bug.": no AI model is set up for you. "Create bug" still works; add a key in Settings › AI for explanations.
• "Your organization has switched AI off.": an organization admin can switch it on in Organization settings → AI. A tester the organization keeps away from AI is not read either.
• The button is missing: the run passed, or the assistant is switched off for you.
• The answer says the evidence doesn't show why: run the check again with the response kept, then ask again.

Related

• /help/ai/asking-the-assistant
• /help/ai/ask-or-do
• /help/api-testing/what-happened-before-a-failure