Start the affected tests after a deploy

Tell {{brand}} from your pipeline that an API was deployed, and it runs the suites the change touches.

In short

Right after your pipeline deploys an API, one command tells {{brand}}, and {{brand}} runs the suites the change touches in that environment. With --wait the command waits for them, so the pipeline step fails when a suite fails.

Before you start

• Who: an access token with the scope ApiTester:write. The suites run as a person who can run them (see Good to know).
• Plan: the suites run on {{brand}} and count against that person's allowance. Comparing a new OpenAPI document needs a plan with API changes.
• First: install the CLI and give each deploy and stage its own id.

Steps

1. Install the CLI and put an access token (made in "Settings", under "Access tokens") in KAMOCORE_TOKEN, as for a run: /help/api-testing/run-in-a-pipeline#run. The token needs the scope ApiTester:write.
2. After the deploy step, run: kamocore deploy --project <projectId> --environment <name> --deploy-id <id> --wait
3. Give each deploy its own id for each stage, such as the build number with the stage's name. In Azure Pipelines that is $(Build.BuildId)-$(System.StageName).
4. To check the API's contract as well, add --openapi with the file or the address of the new OpenAPI document. Add --fail-on-breaking to fail the step when the new version breaks the contract.

Result: the suites run on {{brand}}, and with --wait the command ends with their result.

Which suites run

• With an OpenAPI document that changes the contract: the suites that call what changed.
• With no document, or one that changes nothing: every suite of the project.
• At most ten suites start for one deploy. The rest are listed as not run.

Good to know

• The same id told twice for the same environment starts nothing new.
• --wait waits up to 30 minutes, and --timeout changes that. When the wait runs out, the suites keep running in {{brand}}.
• The document is compared like a version you add by hand, so it needs a plan with API changes and permission to change the project. See /help/api-testing/api-changes.
• The suites run on {{brand}}'s servers, not on your pipeline's machine. To test an API on a private network, run the suite from the pipeline instead.
• Each suite runs as a person, with that person's access: whoever's token reported the deploy; for a service account, the suite's owner, then the project's owner. These runs count against that person's request allowance.
• With nobody who may run a suite, it is listed as not run, with the reason. Nothing runs as nobody.

If something goes wrong

• "Name the environment that was deployed to.": add --environment.

Related

• /help/api-testing/run-in-a-pipeline
• /help/api-testing/api-changes
• /help/api-testing/can-we-ship