INTEGRATION • AZURE DEVOPS

Azure DevOps test management, in Jira

Build in Azure Pipelines, plan and track testing in Jira. Every run’s results are already where your team works.

4.6/5 on Atlassian MarketplaceFree for up to 10 users

Classic Test / Automated Tests

Work item keyFrameworkStatusMore

Test Execution ⇅	Source ⇅	Framework ⇅	Status ⇅	Creator ⇅	Created at ⇅	Completed at ⇅
SHOP-131	Import	JUnit	Success	ECEmma Clark	2 minutes ago	1 minute ago
SHOP-129	Import	TestNG	Success	ASAna Silva	1 hour ago	1 hour ago
SHOP-127	Import	Robot Framework	Success	ECEmma Clark	3 hours ago	3 hours ago
	Import	Cucumber	Fail	LMLucas Meyer	5 hours ago	5 hours ago
SHOP-126	Import	Cucumber	Success	LMLucas Meyer	5 hours ago	5 hours ago
SHOP-124	Import	JUnit	Success	ECEmma Clark	Yesterday	Yesterday
SHOP-122	Import	NUnit	Success	ASAna Silva	Yesterday	Yesterday
SHOP-119	Import	Behave	Success	LMLucas Meyer	2 days ago	2 days ago
SHOP-117	Import	xUnit	Success	ASAna Silva	2 days ago	2 days ago

‹12›Rows per page 10

Azure pipeline #20261104.3Results imported to SHOP-131

Works withJUnitNUNUnitTeTestNGxUxUnitRobot FrameworkCucumberBeBehave

AUTOMATIC IMPORT

Azure Pipelines results, no copy-paste

Stop pasting run summaries into tickets. Results sit in Jira beside the requirements and bugs your team already tracks.

Create or updateA new Test Execution per run, or add to an existing one.

No duplicatesMatching tests are reused, never copied.

Statuses and stepsPass, Fail, Skipped and more, read from the report.

Screenshots attachedEvidence from the report lands on the test run.

#20261104.3 · Update checkout testsshop-web · main · Manually run by Ana Silva

Job4m 08s

Initialize job2s

Checkout shop-web@main4s

Maven pom.xml3m 52s

Run a one-line script6s

Update history log2s

Post-job: Checkout1s

Finalize Job1s

Test ExecutionsSHOP-131×

Overall Execution Status (Total tests: 128)94% Pass

Key	Summary	Run status
SHOP-218	CheckoutTest.returnsOrderId	Pass
SHOP-226	PaymentTest.card3dsChallenge	Fail
SHOP-231	AddressTest.savedAddressAutofill	Skipped
SHOP-233	OrderTest.emailSentOnce	Pass

AgileTest

Test Environments and PlansRemote Trigger

Trigger pipeline for Test Execution SHOP-131 successfully×

Run a manually trigger on:

Azure DevOps trigger

Run

Recent runs:

Azure DevOps pipeline run is finished.SUCCESS5 minutes ago

Trigger Azure DevOps pipeline6 minutes ago

Azure DevOps pipeline run is finished.FAILED1 hour ago

Trigger Azure DevOps pipeline1 hour ago

TRIGGER FROM JIRA

Queue an Azure pipeline from Jira

Testers start the regression run themselves, right from the Test Execution they’re working in.

1
Set up onceAdd an Azure DevOps configuration with your organization, project, pipeline ID and a personal access token. Optionally set the test plan, environment, fix version and milestone.
2
Click RunFrom the AgileTest panel on any Test Execution. Nothing else to fill in.
3
Follow the runThe run result posts back to Recent runs.

TRACEABILITY

From pipeline run to release decision

Know whether a requirement is ready to ship, from its automated and manual results together.

Linked to requirementsLink each automated test case to the requirement it checks.

Latest statusOK or NOK on each requirement after every run.

Defects from failuresLog a bug from a failed run, linked back to it.

Reports in JiraTest Coverage and Traceability reports.

Projects/Shop/SHOP-87In progress

SHOP-87

Guest checkout

AgileTestLatest test status NOK

To Do 1Pass 2Fail 1

Test Cases 4Test ScriptsTest Sessions

Key	Summary	Type	Latest status
SHOP-214	Apply discount code as guest	Manual	Fail	Re-run
SHOP-215	Pay by card	Manual	Pass	Re-run
SHOP-218	Checkout API returns order ID	Generic	Pass	Automated
SHOP-217	Order email is sent	Manual	To Do	Run

Azure pipeline #20261104.3JUnit results imported

120 passed3 failed5 skipped

azure-pipelines.yml

API · script stepCLI · bash stepCLI · Data Center

trigger:
  - main

pool:
  vmImage: ubuntu-latest

steps:
  - task: Maven@4
    inputs:
      mavenPomFile: "pom.xml"
      publishJUnitResults: true
      testResultsFiles: "**/surefire-reports/TEST-*.xml"

  - script: |
      # 1. Get a temporary token
      token=$(curl "$(BASE_AUTH_URL)/api/apikeys/authenticate" -X POST \
        -H 'Content-Type:application/json' \
        --data '{"clientId":"$(CLIENT_ID)","clientSecret":"$(CLIENT_SECRET)"}' | tr -d '"')
      # 2. Send the JUnit report
      curl -X POST "$(BASE_URL)/ds/test-executions/junit?projectKey=$(PROJECT_KEY)" \
        -H "Content-Type:application/xml" -H "Authorization:JWT $token" \
        --data-binary "@target/surefire-reports/TEST-calculateTest.xml"
    displayName: "Run a one-line script"

steps:
  - task: Maven@4
    inputs:
      mavenPomFile: "pom.xml"
      publishJUnitResults: true
      testResultsFiles: "**/surefire-reports/TEST-*.xml"

  - bash: |
      docker run --rm -i \
        -e AGILETEST_BASE_URL=$(BASE_URL) \
        -e AGILETEST_AUTH_BASE_URL=$(BASE_AUTH_URL) \
        -e AGILETEST_CLIENT_ID=$(CLIENT_ID) \
        -e AGILETEST_CLIENT_SECRET=$(CLIENT_SECRET) \
        ghcr.io/agiletestapp/agiletest-cli \
        test-execution import \
        --framework-type junit --project-key $(PROJECT_KEY) \
        <target/surefire-reports/TEST-calculateTest.xml
    displayName: "Upload report"

# Data Center: DC_TOKEN = Jira personal access token
  - bash: |
      docker run --rm -i \
        -e AGILETEST_BASE_URL=$(BASE_URL) \
        -e AGILETEST_DC_TOKEN=$(DC_TOKEN) \
        ghcr.io/agiletestapp/agiletest-cli \
        --data-center \
        test-execution import \
        --framework-type junit --project-key $(PROJECT_KEY) \
        <target/surefire-reports/TEST-calculateTest.xml
    displayName: "Upload report"

SETUP

One extra step in azure-pipelines.yml

Keep your existing test task. One more step sends the report, whether you run Jira Cloud or Data Center.

Script or CLIA curl script step, or the CLI Docker image in a bash step.

Cloud and Data CenterClient ID and Secret, or a Jira personal access token.

Pipeline variablesCredentials stay in secret pipeline variables.

Run status in JiraAn extra step posts the job status to the Test Execution.

HOW IT WORKS

Set up with the official guides

FAQ

Questions, answered

Does AgileTest run my automated tests?

No. Azure Pipelines keeps building and running your tests. AgileTest imports the results into Jira, and can queue a configured pipeline from a Test Execution.

Should I use the AgileTest CLI or the REST API?

Either works. The REST API is a short script step with two curl calls. The CLI runs the ghcr.io/agiletestapp/agiletest-cli Docker image in a bash step.

Which report formats are supported?

JUnit, NUnit, TestNG, xUnit and Robot Framework (XML), plus Cucumber and Behave (JSON). Tools that export JUnit XML, such as Playwright, Cypress and pytest, work too.

Can I start an Azure pipeline from Jira?

Yes. Add an Azure DevOps configuration in AgileTest (organization, project, pipeline ID, API version and a personal access token), then click Run on a Test Execution.

How does the run status get back to Jira?

An extra pipeline step posts the job status to the Test Execution. It needs the Test Execution key, either as a TEST_EXECUTION_KEY variable or set at queue time. The setup guide covers both.

Does it work on Jira Cloud and Data Center?

Yes. Cloud uses an AgileTest Client ID and Client Secret. Data Center uses a Jira personal access token.

Bring your Azure DevOps test results into Jira

Try it free Read the docs

Questions? Contact us · Visit the 24/7 Support Desk

ALSO WORKS WITH