Automation

Automatically report test execution results from your automation frameworks and CI/CD pipelines to TestKase.

Overview

TestKase's automation integration lets you automatically report test execution results from your test frameworks and CI/CD pipelines directly into TestKase test cycles. The @testkase/reporter CLI parses your test output files, matches test results to test cases via Automation IDs, and pushes execution statuses to your TestKase project.

5 CI/CD platforms

GitHub Actions, GitLab CI, Jenkins, Azure DevOps, CircleCI

6 test formats

JUnit, Playwright, Cypress, TestNG, NUnit, Cucumber

15 frameworks

Jest, Playwright, Cypress, Selenium, pytest, and more

Auto matching

Maps results to test cases via Automation IDs

Flexible config

Dry run, verbose logging, custom build IDs

How It Works

Tests run — Your test framework or CI pipeline executes tests and generates output files.
Parse results — The @testkase/reporter CLI reads JUnit XML, Playwright JSON, or other supported formats.
Match IDs — The reporter extracts 5-digit Automation IDs from test names using the [XXXXX] pattern and maps them to test cases in your TestKase project.
Report — Matched results are pushed to a TestKase test cycle with pass/fail/blocked statuses.

Prerequisites

A TestKase account with an active project.
A Personal Access Token (PAT) — generate one from API Keys settings.
Test cases with Automation IDs set (used to match results to test cases).
A test cycle created in TestKase to receive the results.

Install the Reporter

npm install --save-dev @testkase/reporter

Or use it directly with npx (no installation required):

npx @testkase/reporter report --help

Automation IDs

Automation IDs are the key mechanism that connects your test results to TestKase test cases. Each Automation ID is a system-generated 5-digit number (e.g., 48271) that you embed in your test names using a [XXXXX] bracket pattern. The @testkase/reporter CLI extracts these IDs from your test results via regex and maps them to test cases in your TestKase project.

How It Works

Open a test case in TestKase and go to the Automation IDs section.
Click Generate to create a new 5-digit ID (e.g., 48271), or click Link Existing to attach an existing ID.
Multiple Automation IDs can be linked to a single test case.
In your test framework, embed the ID in the test name using brackets: [48271] should login with valid credentials.
The reporter CLI parses your test results, extracts IDs matching the pattern \[(\d{5})\], resolves them to test cases via the API, and reports execution status.

How Matching Works

The reporter parses your test results file and extracts 5-digit IDs from test names using the regex \[(\d{5})\].
For each extracted ID, it looks up the corresponding test case in your TestKase project.
If a match is found, the execution status is updated in the specified test cycle.
If no match is found, the test result is reported as "unmatched" in the output log.

Embedding Automation IDs

Embed the 5-digit Automation ID in your test name using square brackets [XXXXX]:

Test Framework	Where to Embed the ID	Example Test Name
Jest	In the `test()` name	`test('[48271] should login with valid credentials', ...)`
Playwright	In the `test()` title	`test('[48271] should accept valid credentials', ...)`
Cypress	In the `it()` name	`it('[48271] should login with valid credentials', ...)`
JUnit (Java)	In the `@DisplayName` annotation	`@DisplayName("[48271] testValidLogin")`
pytest	In the test function/method name or a marker	`def test_valid_login_48271(self):` with `[48271]` in the parametrize ID or via `pytest.mark`
TestNG	In the `@Test(description)` or test name	`@Test(description = "[48271] testValidLogin")`
NUnit	In the `[Test]` description or method name	`[Test, Description("[48271] ValidLogin")]`
Cucumber	In the Scenario name	`Scenario: [48271] User logs in with valid credentials`
Mocha	In the `it()` name	`it('[48271] should authenticate user', ...)`
Robot Framework	In the test case name	`[48271] Valid Login Test`

Use --dry-run to see exactly how the reporter extracts Automation IDs from your results file. This helps you verify that the [XXXXX] patterns are being detected correctly before pushing real results.

Automation ID Best Practices

Generate IDs in TestKase first. Always generate or link an Automation ID on the test case before embedding it in your test code.
Embed IDs early. When creating test cases for automated tests, generate the Automation ID and embed it in your test name immediately so results can flow in from the first CI run.
Keep IDs unique. Each 5-digit Automation ID maps to a specific test case. Do not reuse the same ID across unrelated test cases.
Multiple IDs per test case. You can link multiple Automation IDs to a single test case if the same scenario is covered by different tests across frameworks.

Next Steps

CLI Reference — Full CLI reference, supported formats, and debugging.
Test Frameworks — Setup guides for 15 test frameworks.
CI/CD Platforms — Platform-specific pipeline setup guides.
Best Practices & FAQ — Best practices, troubleshooting, and common questions.

Automation

On this page