Zephyr Test Management for Jira
eazyBI for Jira
In eazyBI, Zephyr Scale is renamed to Zephyr to match the add-on name.
The new naming applies to eazyBI for Jira Cloud and since eazyBI version 9.1 (not released yet).
Zephyr - Test Management and Automation for Jira (former Zephyr Scale) is a full-featured test management solution, seamlessly integrated into Jira's user interface. From a central location, coordinates all testing activities, including test planning, authoring, execution, tracking, and reporting. Test Management for Jira is an app developed by SmartBear.
On this page:
Data Import
If you have installed Zephyr - Test Management and Automation for Jira, you will see Zephyr in the Jira import options.
Data import in Cloud version
You should authorize data import for Zephyr on Cloud. Please specify the Zephyr Access Token to import Zephyr data.
Go to the Zephyr API Access Tokens page and create a new token; enter it in the eazyBI import options.
Limit imported test executions
To reduce import time and the testing data in the cube, you can import only the most recent test executions.
Set a relative time limit for test executions, like 3 months to import test executions from the last three months, or a particular date, like Jun 14 2020 to set a date range from that date until now. Test executions are imported if they are created (test is added to the cycle) within the time limit (for scheduled test executions) or actual start date is within the time limit (for executed or in progress test executions). If no limit is set, all test executions are imported.
You can set a limit to all eazyBI accounts through advanced settings.
# parameter name for eazyBI version 6.2 and 6.3 [jira.tm4j] test_executions_limit = "3 months"
Since eazyBI version 6.4, the parameter name is changed to jira.zephyr_scale.
# parameter name starting eazyBI version 6.4 [jira.zephyr_scale] test_executions_limit = "3 month"
Another option is to set a limit for an individual eazyBI account. In the import options, add the period in the field "Imported Test Execution limit".
If import limits are applied to the whole eazyBI instance via advanced settings, the import options won't exceed those limits. In this way, the overall limits can be set on the eazyBI account, and, in an individual account, this time can be reduced further.
Dimensions
After the import, the following Zephyr-specific dimensions are added:
Zephyr Defect | Defect issues added during the testing. The dimension is organized in Project and Defect levels. In addition, you can add Issue hierarchy to group defect issues. |
|---|---|
Zephyr Environment | Environments of scheduled, in progress, and completed test executions. |
Zephyr Iteration | Iterations of test cycles. Dimension "Zephyr Iteration" is not available for the Jira Cloud version. |
Zephyr Requirement | Requirement issues that have linked test cases. The dimension is organized in Project and Requirement levels. In addition, you can add Issue hierarchy to group requirement issues. |
Zephyr Test Assigned To | Users who are assigned to tests within test cycles. |
Zephyr Test Case | Created test cases. The dimension is organized by default hierarchy (in Project and Test Case levels) or by folder hierarchy (in Project, Folder, Subfolder, and Test Case levels). You can add custom hierarchy based on Test Case properties. This feature is available on Jira Cloud and since eazyBI version 8.1. |
Zephyr Test Case Label | Current labels of test cases. If several labels are entered for a test case, then it is counted for each label. |
Zephyr Test Case Owner | Owners of test cases. When used with measure Zephyr Test Cases created, then returns the current test case owner. When used with test case execution measures, it returns the test case owner during test execution. |
Zephyr Test Case Priority | Priorities of test cases. When used with measure Zephyr Test Cases created, then returns the current test case priority. When used with test case execution measures, it returns the test case priority during test execution. |
Zephyr Test Case Status | Statuses of test cases. When used with measure Zephyr Test Cases created then returns the current test case status. When used with test case execution measures, it returns the test case status during test execution. |
Zephyr Test Cycle | Created test cycles. The dimension is organized by default hierarchy (in Project and Test Cycle levels) or by folder hierarchy (in Project, Folder, Subfolder, and Test Cycle). You can add custom hierarchy based on Test Cycle properties. This feature is available on Jira Cloud and since eazyBI version 8.1. |
Zephyr Test Cycle Owner | Current owners of test cycles. |
Zephyr Test Cycle Related Issue | Jira issues related to test cycles. The dimension is organized in Project and Related Issue levels. |
Zephyr Test Cycle Status | Current statuses of test cycles. |
Zephyr Test Executed by | Users who have executed tests. |
Zephyr Test Plan | Created test plans. The dimension is organized by default hierarchy (in Project and Test Plan levels) or by folder hierarchy (in Project, Folder, Subfolder, and Test Plan levels). You can add a custom hierarchy based on Test Plan properties, for example, Test Plan Status. This feature is available on Jira Cloud and since eazyBI version 8.1. |
Zephyr Test Plan Status | Statuses of test plans (current and historical). The "Zephyr Test Plan Status" dimension is not available in Jira Cloud and since eazyBI version 8.1. Use the property "Zephyr Test Plan status" instead to see the current status of the Test Plan. Or add a custom hierarchy to the "Zephyr Test Plan" dimension to group and filter test plans by current status. |
Zephyr Test Result Status | Statuses of test case executions (Passed, Failed, In Progress, Not Executed, etc.) |
Zephyr Test Script Type | Types of test scripts (Plain Text, Steps, etc.) |
Zephyr Test Execution (hidden) | The hidden dimension of test executions. Member names are execution IDs. |
Measures
The following Zephyr -specific measures are imported. These measures can be analyzed by the mentioned Zephyr-specific dimensions and by a few standard dimensions: Time, Project, Component, and Fix Version. Note that the measures used in the report define the context of standard dimensions and represent the project, component, or version of test cases or test cycles. For example, when using execution-related measures, such as Zephyr Execution time, Zephyr Test Cases executed, Zephyr Test Cases with Test Cycles, and others, then the standard dimensions Project and Fix Version represent data on Test Cycle.
You can interact with report and see a list of test cases, test cycles, and executions behind the value. In the report, click on the cell value or chart item to open the Detailed list of test cycles, which contains executions with additional details on the execution date, the user who executed it, the execution status, and the comment. Or you can Drill through → Test Case or Test Cycle to get a list of relevant Test or Test Execution issues with an active hyperlink to Jira issues.
Zephyr Test Cases created | How many test cases are created. |
|---|---|
Zephyr Test Cycles created | How many test cycles are created. |
Zephyr Execution time | How much time, in minutes, took to execute test cases. |
Zephyr Test Cases executed | How many test cases are executed within selected dimensions; test executions are not in "Not Executed" or "In Progress" statuses. |
Zephyr Test Cases executing | How many test cases are currently executed (in progress) within selected dimensions; test executions are in "In Progress" status. |
Zephyr Test Cases scheduled | How many test cases are scheduled within selected dimensions but not yet executed; test executions are in "Not Executed" status. |
Zephyr Test Case defects | How many defects are registered for test executions. A unique count of different defects is returned for each row in a report. |
Zephyr Executions | How many times test cases have been executed, are executing, or are scheduled for execution. |
Zephyr Automated executions | How many times test cases have been executed, are executing, or are scheduled for the execution with an automation tool. |
Zephyr Test Cases with Test Cycles | How many test cases have at least one test cycle scheduled, in progress, or completed. |
Zephyr Execution first date | Date when a test case is executed for the first time within selected dimensions. |
Zephyr Execution last date | Date when a test case is executed for the last time within selected dimensions. |
Zephyr Estimated execution time | Estimated execution time in minutes. |
Zephyr Average execution time | Average execution time, in minutes, per completed executions. Average of measure Zephyr Execution time within selected dimensions Zephyr Test Cases with Test Cycles. |
Zephyr Execution summary | List of test result statuses separated by a comma, and how many Zephyr Executions are for each test result status. |
Zephyr Actual Test Cases | How many created test cases are still relevant within the selected dimensions? |
Issues in Fix version | How many Jira issues with or without any testing coverage or relation to test cycles are assigned to the selected Fix version. This measure is based on a predefined measure Issue created, and should be used together with the dimension Fix Version. |
Properties
The following Zephyr-specific properties will be created.
Zephyr Test Case last execution date | Last execution date of the test case. This is a test case property and should be used together with Zephyr Test Case dimension. |
|---|---|
Zephyr Test Case last executed by | The user who did the last execution of the test case. This is a test case property and should be used together with Zephyr Test Case dimension. |
Zephyr Test Case estimated time | Current estimated execution time in minutes of the test case. This is a test case property and should be used together with Zephyr Test Case dimension. |
Zephyr Test Case last execution status | Last execution status of the test case. This is a test case property and should be used together with Zephyr Test Case dimension. |
Zephyr Test Case last execution version | The version of the last execution of the test case. This is a test case property and should be used together with Zephyr Test Case dimension. |
Zephyr Test Case created date | Registered creation date of the test case on the Data Center version. The first execution date on the Cloud version. This is a test case property and should be used together with Zephyr Test Case dimension. |
Zephyr Test Case Archived? | An indication of whether the test case is actual or archived. This is a test case property and should be used together with Zephyr Test Case dimension. |
Zephyr Test Cycle planned start date | Planned start date of the test cycle. This is a test cycle property and should be used together with Zephyr Test Cycles dimension. |
Zephyr Test Cycle planned end date | The planned end date of the test cycle. This is a test cycle property and should be used together with Zephyr Test Cycles dimension. |
Zephyr Test Plan labels | Current labels of the test plan, separated by a comma. This is a test plan property and should be used together with Zephyr Test Plan dimension. |
Zephyr Test Plan created date | Registered creation date of the test plan. This is a test plan property and should be used together with Zephyr Test Plan dimension. Property "Zephyr Test Plan created date" is not available for the Jira Cloud version. |
Zephyr Test Plan status | Current status of the test plan. This is a test plan property and should be used together with Zephyr Test Plan dimension. The property Zephyr Test Plan status is available on Jira Cloud and since eazyBI version 8.1. |
Zephyr Test Plan Archived? | An indication of whether the test case is actual or archived (deleted). This is a test plan property and should be used together with Zephyr Test Plan dimension. |
Zephyr Requirement status | Current status of the requirement issue. This is a test requirement property and should be used together with Zephyr Requirement dimension. |
Zephyr Test Plan related issues | Keys of the Issues linked to the test plan. This is a test plan property and should be used together with Zephyr Test Plan dimension. |
Issue properties
Dimensions Zephyr Defect, Zephyr Requirement, and Zephyr Test Cycle Related Issue are Jira issues that are also represented in the Issue dimension. You can define a calculated measure to get any issue property that is imported into eazyBI, also for those three Zephyr-specific dimensions. To access the issue property, use the function get(). For example, to get priority for Zephyr Defect, copy a formula of issue property Issue Priority and replace dimension name [Issue] with [Zephyr Defect] like this:
[Priority].[Priority].getMemberNameByKey(
[Zephyr Defect].CurrentHierarchyMember.get('Priority ID')
)
Another example, to get the resolution date for Zephyr Requirement, copy a formula of issue property Issue resolution date and replace dimension name [Issue] with [Zephyr Requirement] like this:
[Zephyr Requirement].CurrentHierarchyMember.get('Resolved at')
Add Issue hierarchy
You can group issues representing linked requirements and defects by Epic hierarchy or any other hierarchy you have built for the Issue dimension.
Additional hierarchies for Zephyr dimensions are defined in eazyBI advanced settings (or site settings for Cloud) and are marked with the setting [[jira.zephyr_scale.custom_hierarchies]]. Provide the dimension name for which to enable the hierarchy, the hierarchy name, and the top-level member name. List all hierarchy levels where the bottom level is a requirement/defect issue (sub-task level is not supported).
For example, enable Epic hierarchy for the "Zephyr Defect" dimension with three hierarchy levels: project of defect issue, Epic related to defect issue, and defect issue.
[[jira.zephyr_scale.custom_hierarchies]]
dimension = "Zephyr Defect"
hierarchy = "Epic"
all_member_name = "All Defects by Epics"
levels = [
{name="Project",key_column = "project_key"},
{name="Epic",key_column="epic_key"},
{name="Defect"}
]
If you have already constructed an additional Issue hierarchy, you can also enable it for Zephyr dimensions. Suppose you have a Features by Stories hierarchy as described in this example Separate hierarchy level for Story between parent and epic. Specify the dimension name for which to make the hierarchy, the hierarchy name, and the top-level member name. The levels should match the additional issue hierarchy description, except for the Sub-task level, and there should be no parameters for the lowest (requirement or defect) level.
[[jira.zephyr_scale.custom_hierarchies]]
dimension = "Zephyr Requirement"
hierarchy = "Features by Stories"
all_member_name = "All Issues by Features"
levels = [
{name="Feature",key_column="customfield_featuree",issue_type="Feature"},
{name="Epic and risk",key_column="customfield_epice",issue_type="Epic"},
{name="Story",key_column="customfield_storye",issue_type="Story"},
{name="Requirement"},
]
You can also enable the Plans hierarchy. In that case, list all hierarchy levels and parent keys matching the hierarchy configuration in Jira. The example below describes the Plans hierarchy for Initiative → Epic → Story and enables it to the "Zephyr Requirement" dimension.
[[jira.zephyr_scale.custom_hierarchies]]
dimension = "Zephyr Requirement"
hierarchy = "Plans"
all_member_name = "All requirements by Initiatives"
levels = [
{name="Initiative",key_column = "jpoh_parent_3"},
{name="Epic",key_column="jpoh_parent_2"},
{name="Story"}
]
Sample reports
If you selected the Import sample reports option, then additional sample reports are created in the Samples Zephyr folder. Please try these reports to see which measures would be useful for your specific needs. If you would like to customize some reports, then save them and change the name (as the default sample reports are overwritten during each Jira import). 