Xray Tutorial: Test, Test Set, Execution, and Test Plan in Jira

This Xray tutorial is a practical walkthrough for QA engineers who need to move from “we track bugs in Jira” to “we manage tests in Jira with structure.” Xray is a Jira app that adds test specification, organization, execution, and coverage on top of the backlog you already use.

You will learn what Xray is, how its core issue types fit together, how to create a first Test and link it to a story, how to execute once and read results, and where Xray differs from plain Jira. For the full process picture, keep the Jira test management guide open as the pillar.

What Xray is

Xray is a test management app for Jira Cloud and Data Center. It introduces issue types and relationships designed for testing:

• Test — the reusable specification (manual steps or automated reference)
• Test Set — a group of Tests you reuse as a pack
• Test Execution — a concrete run of Tests against a version/environment
• Test Plan — an optional planning layer for a release or milestone

Xray also provides coverage views that connect requirements (stories, epics, or requirement issue types) to Tests and their latest results. That is the difference between “we have a checklist” and “we can show what protects this story in version 1.9.”

Xray does not replace good case design. Weak steps stay weak inside fancy issue types. Pair this tutorial with how to write QA test cases and the QA test case template.

Install and enable overview (conceptual)

Exact admin clicks change as Atlassian and Xray evolve, so treat this as the capability checklist—not a pixel-perfect installer script.

What admins typically configure

What QA leads should verify before the team piles in

1. You can create a Test in the target project.
2. You can link a Test to a Story.
3. You can create a Test Execution and add Tests.
4. You can set Pass/Fail and see it on the requirement coverage view.
5. A shared label/component convention exists (publish it).

If any of those fail, fix configuration before training twenty people into a broken project. Comparison context for whether you need Xray at all is in Jira vs Xray.

The four issue types that matter

Test

A Test is the specification. For manual tests, it holds preconditions and steps with expected results. For automated tests, it often stores an identifier or definition that CI maps into results.

Properties of a healthy Test:

• Stable title focused on one behavior
• Runnable without tribal knowledge
• Linked to the requirement it verifies
• Tagged for packs (smoke, regression, API)

Creation details: how to create test cases in Jira.

Test Set

A Test Set is a reusable basket of Tests—smoke pack, checkout regression, mobile sanity. You rarely want to cherry-pick 80 Tests by hand every release; you want to add a Set to an Execution.

Design guidance lives in Xray Test Sets. Rule of thumb: Sets mirror how you actually schedule work, not your org chart fantasies.

Test Execution

A Test Execution is evidence. It answers: who ran which Tests, on which build/environment, with which outcomes? Executions have dates and statuses; Tests themselves should not be rewritten every run.

Deep dive: Xray test execution and the broader Jira test execution guide.

Test Plan

A Test Plan aggregates the testing ambition for a version or milestone. Teams use Plans to track overall progress across multiple Executions (smoke, regression, UAT). Small teams can start without Plans and add them when release reporting needs a single rollup. For a comparison-first view of Plans versus runs, see Xray Test Plan vs Test Execution.

Mental model: specification vs run

If you find yourself editing Test steps to record what happened yesterday, you are mixing specification and evidence. Put yesterday in the Execution.

First walkthrough: create a Test and link a story

Work a single vertical slice end to end. Do not configure twenty Sets first.

Scenario

Story summary: “Logged-in user can update their display name from profile settings.”

Acceptance sketch:

• Display name field is editable
• Valid names save and appear in the header
• Empty names are rejected with an error

Step 1 — Open the story and confirm criteria

If criteria are fuzzy, clarify before writing Tests. Patterns: acceptance criteria examples.

Step 2 — Create the first Test

Create issue type Test.

Title: Valid display name saves and appears in the header

Preconditions:

• User is logged in
• Profile settings page is reachable
• No open defect blocking profile edits

Steps:

Add a second Test for the empty-name negative path. Linking both to the same story starts real coverage.

Step 3 — Link Test to Story

From the Test (or Story), add the Xray relationship so the Story shows “tested by” and the Test shows “tests.” Verify both sides. Traceability habits scale via requirements traceability in Jira and Xray.

Step 4 — Label and component

These fields matter when you build Sets and filters later—not only for this one run.

First walkthrough: execute once

Step 5 — Create a Test Execution

Create a Test Execution issue.

Summary pattern: profile-display-name — staging — smoke

In the description or fields, record:

• Build or commit identifier
• Environment URL
• Browser or device if relevant
• Tester name if not obvious from assignee

Step 6 — Add Tests

Add the Tests you just created (or add a Test Set if you already grouped them). Confirm the Execution’s tested version matches the story’s fix version when your process uses versions.

Step 7 — Run and set status

Open the execution player / test run view (wording varies). For each step, compare actual behavior to expected.

Status guidance:

On FAIL:

1. Capture evidence (screenshot, console error, request ID).
2. Create a Bug linked to the failed Test / Execution.
3. Do not silently retest “until green” without recording the fail if the build truly failed.

More on evidence and gates: Jira test execution guide.

Step 8 — Read results where stakeholders look

Check:

• The Execution’s overall progress (passed / failed / blocked counts)
• The Story’s coverage panel (latest status for linked Tests)
• Any board or dashboard filter your team uses for release readiness

If the Story still shows empty coverage, the link is missing—fix the relationship before blaming Xray.

Where Xray differs from plain Jira

Plain Jira can start the cultural habit of linking verification work to stories. Xray pays off when history, coverage, and reuse become weekly needs. Decision framing: Jira vs Xray. Process around roles and cadence: Jira QA workflow.

Building out after the first success

Once one story works end to end, expand deliberately:

1. Convert related Tests into a Test Set (feature smoke).
2. Add negatives and edges using basic / standard / deep coverage.
3. Create a regression Set from high-value feature Sets—see regression test cases.
4. Introduce a Test Plan for the next named release.
5. Adopt operating rules from Xray best practices.

Avoid boiling the ocean: do not migrate 2,000 spreadsheet rows before the team can execute twenty Tests cleanly.

Common beginner mistakes

Creating Executions but never linking Tests to Stories

You get activity metrics without requirement coverage. Always link specifications to requirements.

One Test Execution per Test forever

That works for demos and fails for releases. Group by scope: smoke execution, feature execution, full regression execution.

Using Test Plans as dumping grounds

A Plan with no clear version or exit criteria becomes a junk drawer. Plans should map to a releasable milestone.

Teaching Xray UI before teaching case quality

People learn buttons quickly and still write “test the profile page.” Teach titles, preconditions, and expected results first.

Ignoring BLOCKED

Leaving items TODO while environment issues linger hides risk. BLOCKED with an owner is healthier than silent stalls.

Best practices for new Xray teams

1. Run the vertical slice (story → tests → execution → coverage) before bulk import.
2. Name Executions with version, scope, and environment.
3. Keep smoke Sets small and trusted.
4. Define evidence expectations for FAIL.
5. Review orphan Stories and orphan Tests weekly.
6. Document label meanings in one short wiki page.
7. Separate manual and automated Tests visually via labels or type.
8. Retire obsolete Tests instead of skipping them forever.
9. Align on who creates Tests during refinement—see Jira QA workflow.
10. Re-read Xray best practices after your first real release using the tool.

Related reading

• Pillar: Jira test management guide
• How to create test cases in Jira
• Jira test execution guide
• Xray Test Sets
• Xray test execution
• Xray Test Plan vs Test Execution
• Jira vs Xray
• Requirements traceability in Jira and Xray

Conclusion

Xray gives Jira a real test management model: Tests as specifications, Sets as packs, Executions as evidence, and Plans as release rollups. Learn it by completing one story end to end—create, link, execute, read coverage—before scaling imports and dashboards.

When the first vertical slice is clean, organization and reporting become multiplicative instead of decorative. Resist the urge to import every legacy checklist on day one; a small library with honest steps and linked stories beats a thousand hollow Test issues that nobody trusts enough to run. Once that discipline is in place, expand coverage deliberately using basic, standard, and deep coverage as a guide for how much depth each risk area deserves.

FAQ

Is this Xray tutorial enough to replace vendor docs?

It teaches the mental model and first workflow. Use vendor docs for admin settings, permissions, and version-specific UI. Concepts here stay stable even when menus move.

Do we need Test Plans on day one?

No. Many teams start with Tests, Sets, and Executions. Add Plans when a release needs a single progress narrative across multiple executions.

Can developers execute Xray tests?

Yes, if permissions allow. Shared execution during feature hardening often catches issues earlier. Agree on evidence standards so Pass means the same thing for everyone.

How does automation fit?

Automated tests map to Test issues; CI publishes results into Test Executions. Keep identifiers stable. Manual exploratory work can still sit beside automation in the same Plan.

What if our project already has hundreds of Task-based “tests”?

Pilot Xray on one component. Migrate high-value cases first. Do not mass-convert junk. Writing quality still follows how to write QA test cases.

Where should I go next after this tutorial?

Create more cases with how to create test cases in Jira, structure packs via Xray Test Sets, then deepen execution skills in Xray test execution.

Improve your QA workflow faster

QA Workflow Assistant helps QA engineers generate structured test cases from Jira stories, improve coverage, and organize testing workflows.

→ Try QA Workflow Assistant

See Pricing and Docs for how teams connect generation into their Jira and Xray workflow.