Run only the tests a change touches
Add --affected to kamocore run, and a pull request runs only the suites its changed files touch - every suite, with the reason, when it cannot tell.
In short
A small change should not wait for every suite. With --affected, the CLI reads the changed files from git, asks {{brand}} which suites of the project they touch, and runs only those. No AI is involved: it follows the Designer's code readings and the links between services, endpoints and suites.
Before you start
- Connect your code in the Designer, so {{brand}} knows which folders hold which service. See /help/designer/connect-your-code.
- Link the services to their API project, requests or suites on the service page.
- Give the access token the scope Designer:read as well as ApiTester:read and ApiTester:write.
- Who: an access token with the scopes ApiTester:read, ApiTester:write and Designer:read.
- Plan: the Designer and API Tester must both be on your plan.
Steps
- Open the project's "Run in CI" and find "Run only what a change touches".
- Press "Copy command" and put it in your pipeline in place of the usual run.
- In a pull request, add --base with the commit to compare with, such as the merge base.
Result: the CLI prints which suites it runs and why, runs them in turn, and writes one JUnit file per suite (results.xml, results-2.xml, ...). The exit code is the worst of them.
Good to know
- A service lives in the folders of the files that name it. A change inside its deepest folder touches it, and so does a change to a file that names it, such as a compose file.
- Every operation of a touched service counts, so every suite that calls one of them runs.
- When {{brand}} cannot tell - no git, an unknown base, too many changes, the Designer off, no code reading or nothing linked - every suite runs and the reason is printed.
- Without --base the CLI compares with the last commit; a shallow checkout then runs everything.
- --base takes a commit id only.
If something goes wrong
- Every suite runs with "designer-off": the token lacks Designer:read, or the Designer is off for you.
- Every suite runs with "no-code-reading": connect your code in the Designer first.
- Every suite runs with "nothing-mapped" or "no-tests-linked": link the service to its suites or requests on the service page.
Related
- /help/api-testing/run-in-a-pipeline
- /help/api-testing/tests-after-a-deploy
- /help/designer/connect-your-code