Page tree
Skip to end of metadata
Go to start of metadata

 

Xray for Jira is a complete, full-featured Atlassian verified test management add-on for Jira. Xray supports both manual and automated tests and a complete testing life cycle: test planning, test designing, and test execution to help developers improve the quality of their systems.

Xray for Jira is an add-on developed by Xpand Add-ons, an Atlassian Solution Partner.

On this page:

Xray Data Import

This documentation page describes the latest eazyBI app version 4.5 for Jira Server.

Xray import option is available starting from eazyBI app version for Jira server 4.2.0. and is not supported for Jira Cloud.

If you have installed Xray add-on for Jira Server then in Jira import options you will see Import Xray option.

Xray Dimensions

After the import the following additional dimension will be added:

Xray Requirement

Requirement issues which are linked to test cases. Will be imported all issues with types selected in "Requirement Issue Types" and with link type "Tests". The dimension organized in Project and Requirement levels.

Xray Test Plan

Test plan issues. The dimension is organised by default hierarchy (in Project and Plan levels) or by Test Plan boar hierarchy (in Project, Plan, Folder and Subfolder).

Xray TestTest issues. The dimension is organised by default hierarchy (in Project and Test levels) or by Test repository hierarchy (in Project, Folder, Subfolder and Test).

Xray Test Set

Test set issues. This dimension is organized in Project and Set levels.

Xray Test Pre-condition

Test pre-condition issues. This dimension is organized in Project and Pre-condition levels.

Xray Test Execution

Test execution issues. This dimension and organized in Project and Execution levels.

Xray Defect

Defect issues added during testing. This dimension is organized in Project and Defect levels.

Xray Test Run AssigneeUsers who are assigned to a test runs.
Xray Test Executed byUsers who executed tests.
Xray Test Execution StatusTest execution statuses – by default there are TODO, PASS, EXECUTING, FAIL and ABORTED statuses, but also there can be other user-defined statuses.
Xray Pre-condition TypeTypes of pre-conditions - Manual, Cucumber.
Xray Test TypeTypes of tests (Manual, Cucumber, Generic, etc). The dimension is organised in Type and Scenario Type levels.

Custom Dimensions

If you do not find Xray custom field Test Environments in import options, you may import Xray 2.0 custom field Test Environments using Advanced settings for custom fields. You may use following custom field definition example (replacing NNNNN with a corresponding field ID in Jira):

[jira.customfield_NNNNN]
data_type = "string"
dimension = true
multiple_values = true
name = "Xray Test Environments"
dimension_annotations = {group = "Xray"}

Xray Measures

The following measures will be imported. These measures can be analyzed by the mentioned Xray specific dimensions as well as by Issue dimension (where individual tests executions are registered as issues with Test Execution type) and by other standard dimension members which correspond to the issue with Test Execution type.

Xray Tests created

How many tests are created.

Xray Tests scheduledHow many tests are scheduled within selected dimensions but not yet executed.
Xray Tests executingHow many tests are executing within selected dimensions.
Xray Tests executedHow many tests are executed within selected dimensions.
Xray Tests defect countHow many defects are registered for a test execution (including individual test steps). Unique count of different defects is returned.
Xray Tests with executionsHow many tests have at least one test execution (scheduled, in progress or completed).
Xray Test RunsHow many times tests have been executed, are executing or scheduled for execution.

Xray Calculated Fields

Also following calculated members will be created.

Xray Test last executed date

Last execution date for a test issue

Xray Test last executed byAn last executor for a test issue
Xray Test last execution statusLast execution status for a test issue
Xray Tests executed first dateDate and time for first executed test within selected dimensions.
Xray Tests executed last dateDate and time for last executed test within selected dimensions.
Xray Tests execution summaryTest statuses within selected dimensions.
Xray Defect status

Current status of the defect issue.

This is a defect property and should be used together with Xray Defect dimension.

Xray Defect created

Creation date and time of the defect issue.

This is a defect property and should be used together with Xray Defect dimension.

Xray Defect assignee

Current status of the defect issue.

This is a defect property and should be used together with Xray Defect dimension.


Xray Report Examples

Overall Xray Test Run Results

You can create a simple test execution report to quickly see the overall snapshot of your test execution results. You can specify additional filters, for example Xray Test Plan, Test Environment, Test Set, Time period, or other Xray dimensions.

Copy and import report definition into your eazyBI account:

Report definition
{
  "cube_name": "Issues",
  "cube_reports": [ {
     "name": "Overall Test Runs results",
     "folder_name": "Xray Reports",
     "result_view": "table",
     "definition": {"columns":{"dimensions":[{"name":"Xray Test Execution Status","selected_set":["[Xray Test Execution Status].[Status].Members"],"members":[],"bookmarked_members":[{"depth":0,"name":"All Xray Test Execution Statuses","full_name":"[Xray Test Execution Status].[All Xray Test Execution Statuses]","drillable":true,"type":"all"}]}]},"rows":{"dimensions":[{"name":"Measures","selected_set":["[Measures].[Xray Tests Run count]","[Measures].[Xray Tests Run count %]"],"members":[]}],"nonempty_crossjoin":true},"pages":{"dimensions":[{"name":"Xray Test Plan","selected_set":["[Xray Test Plan].[All Xray Test Plans]"],"members":[{"depth":0,"drillable":true,"full_name":"[Xray Test Plan].[All Xray Test Plans]","name":"All Xray Test Plans","type":"all"}],"bookmarked_members":[],"current_page_members":["[Xray Test Plan].[All Xray Test Plans]"]},{"name":"Xray Test Environments","selected_set":["[Xray Test Environments].[All Xray Test Environments]"],"members":[{"depth":0,"name":"All Xray Test Environments","full_name":"[Xray Test Environments].[All Xray Test Environments]","drillable":true,"type":"all"}],"bookmarked_members":[],"current_page_members":["[Xray Test Environments].[All Xray Test Environments]"]},{"name":"Xray Test Set","selected_set":["[Xray Test Set].[All Xray Test Sets]"],"members":[{"depth":0,"name":"All Xray Test Sets","full_name":"[Xray Test Set].[All Xray Test Sets]","drillable":true,"type":"all"}],"bookmarked_members":[],"current_page_members":["[Xray Test Set].[All Xray Test Sets]"]},{"name":"Time","selected_set":["[Time].[All Times]"],"members":[{"depth":0,"name":"All Times","full_name":"[Time].[All Times]","drillable":true,"type":"all"}],"bookmarked_members":[],"current_page_members":["[Time].[All Times]"]}]},"options":{"nonempty":true,"total":"columns"},"view":{"current":"table","maximized":true,"table":{}}}
  } ],
  "calculated_members": [{"name":"Xray Tests Run count","dimension":"Measures","annotations":{"group":"Xray","drill_through_dimension_levels":"[Xray Test].[Test], [Xray Test Execution].[Execution]"},"formula":"Sum(\n  Filter(Descendants([Xray Test Execution].CurrentMember,\n    [Xray Test Execution].[Execution]),\n    [Measures].[Xray Tests with executions] > 0),\n  [Measures].[Xray Tests with executions]\n)","format_string":""},{"dimension":"Measures","format_string":"#0%","formula":"-- annotations.group=Xray\n[Measures].[Xray Tests Run count] / \n([Measures].[Xray Tests Run count], [Xray Test Execution status].DefaultMember)","name":"Xray Tests Run count %"}]
}

Created and Executed by Test and Test Set

Analyze created and executed Xray tests by project and test set. This will give you a full overview of test execution status across all your project portfolio.

Report definition
{
  "cube_name": "Issues",
  "cube_reports": [ {
     "name": "Created and Executed by Test and Test Set",
     "folder_name": "Xray Reports",
     "result_view": "bar_chart",
     "definition": {"columns":{"dimensions":[{"name":"Measures","selected_set":["[Measures].[Xray Tests created]","[Measures].[Xray Tests executed]"],"members":[]}]},"rows":{"dimensions":[{"name":"Xray Test","selected_set":["[Xray Test].[Project].Members"],"members":[],"bookmarked_members":[]},{"name":"Xray Test Set","selected_set":["[Xray Test Set].[Set].Members"],"members":[],"bookmarked_members":[]}],"nonempty_crossjoin":true},"pages":{"dimensions":[]},"options":{},"view":{"current":"bar_chart","maximized":true,"bar_chart":{"stacked":false,"vertical":false,"swap_axes":false,"data_labels":false,"series_options":{"Xray Tests created":{"color":"#6795C4"},"Xray Tests executed":{"color":"#AAC458"}},"y_axis":{"0":{"max":"6","step":"1"}}}}}
  } ],
  "calculated_members": []
}

Cumulative Xray Test Burn-Down Chart

Analyze the progress of a Test Plan to see how it evolves over time. How many tests are created, how many of them have been completed, and whether the tests are To-do, Executing, Pass, Fail, or Aborted. Spot issues more easily, make better project forecasting, and see whether or not the testing goes according to schedule.

Report definition
{
  "cube_name": "Issues",
  "cube_reports": [ {
     "name": "Cumulative Test Evolution Report",
     "folder_name": "Xray Reports",
     "result_view": "line_chart",
     "definition": {"columns":{"dimensions":[{"name":"Measures","selected_set":["[Measures].[Cumulative tests created]","[Measures].[Cumulative tests remaining]","[Measures].[Cumulative test execution history]"],"members":[]},{"name":"Xray Test Execution Status","selected_set":["[Xray Test Execution Status].[All Xray Test Execution Statuses]"],"members":[{"depth":0,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Xray Test Execution Status].[All Xray Test Execution Statuses]","name":"All Xray Test Execution Statuses","type":"all"}],"bookmarked_members":[]}]},"rows":{"dimensions":[{"name":"Time","selected_set":["[Time].[Day].Members"],"members":[],"bookmarked_members":[]}],"nonempty_crossjoin":false},"pages":{"dimensions":[{"name":"Xray Test Plan","selected_set":["[Xray Test Plan].[All Xray Test Plans]"],"members":[{"depth":0,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Xray Test Plan].[All Xray Test Plans]","name":"All Xray Test Plans","type":"all"},{"depth":1,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Xray Test Plan].[AmitProject]","name":"AmitProject","parent_full_name":"[Xray Test Plan].[All Xray Test Plans]"},{"depth":1,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Xray Test Plan].[Calculator]","name":"Calculator","parent_full_name":"[Xray Test Plan].[All Xray Test Plans]"}],"bookmarked_members":[],"current_page_members":["[Xray Test Plan].[All Xray Test Plans]"]}]},"options":{"nonempty":true},"view":{"current":"line_chart","maximized":true,"line_chart":{"area":true,"stacked":true,"swap_axes":false,"data_labels":false,"series_options":{"All Xray Test Execution Statuses;Cumulative tests created":{"type":"area"},"All Xray Test Execution Statuses;Cumulative tests remaining":{"type":"line"},"Cumulative tests created;All Xray Test Execution Statuses":{"type":"line-Dash"},"Cumulative tests remaining;All Xray Test Execution Statuses":{"type":"line-Dash"}},"y_axis":{"0":{"step":"1"}}}}}
  } ],
  "calculated_members": [{"dimension":"Measures","format_string":"","formula":"-- annotations.group=Xray\nSum({PreviousPeriods([Time].CurrentHierarchyMember),\n      [Time].CurrentHierarchyMember},    \n    [Measures].[Xray Tests created]\n  )\n)\n","name":"Cumulative Xray tests created"},{"dimension":"Measures","format_string":"","formula":"-- annotations.group=Xray\nCache(\n  Sum({PreviousPeriods([Time].CurrentHierarchyMember),\n      [Time].CurrentHierarchyMember},\n    [Measures].[Xray Tests with executions]\n  )\n)\n","name":"Cumulative Xray tests executed"},{"dimension":"Measures","format_string":"","formula":"-- annotations.group=Xray\nCase when \n[Xray Test Execution Status].CurrentMember.Level.Name = \"Status\" \nthen\n[Measures].[Cumulative Xray tests executed by status]\nend","name":"Cumulative test execution history"},{"dimension":"Measures","format_string":"","formula":"-- annotations.group=Xray\ncase when\n[Xray Test Execution Status].CurrentMember is [Xray Test Execution Status].DefaultMember \nthen\n[Measures].[Cumulative Xray tests created]\nend\n","name":"Cumulative tests created"},{"dimension":"Measures","format_string":"","formula":"-- annotations.group=Xray\ncase when\n[Xray Test Execution Status].CurrentMember is [Xray Test Execution Status].DefaultMember \nthen\n[Measures].[Cumulative Xray tests created]- \n[Measures].[Cumulative Xray tests executed by status]\nend\n","name":"Cumulative tests remaining"},{"dimension":"Measures","format_string":"mmm dd yyyy","formula":"-- annotations.group=Xray\n(DateParse(Generate(Tail(Filter(\n  {[Time].CurrentHierarchy.Level.Members.Item(0):\n     [Time].CurrentHierarchyMember},\n  [Measures].[Xray Tests with executions] > 0\n)).Item(0),\nFormat([Time].CurrentHierarchyMember.StartDate, 'yyyy-mm-dd'), \n \",\"\n)))","name":"Xray test last status date"},{"dimension":"Measures","format_string":"#,##0","formula":"-- annotations.group=Xray\ncache(SUM(\n  Descendants([Xray test].CurrentMember, [Xray test].[test]),\n  case when\n    [Measures].[Cumulative Xray tests executed] > 0\n  then\n    Case when \n      [Xray Test Execution Status].CurrentMember.Level.Name = \"Status\" \n    then\n      case\n      when \n        Tail(Order([Xray Test Execution Status].Members,\n            [Measures].[Xray test last status date])).Item(0) is [Xray Test Execution Status].CurrentMember\n      then\n        1\n      end\n    else\n      1\n    end\n  end)\n)","name":"Cumulative Xray tests executed by status"}]
}

Test Status History Report

Detailed information about each test: which tests have been scheduled, executed, or completed most often; how many times and how long each test has been run; what statuses have been returned; which tests have been failing most often.

Report definition
{
  "cube_name": "Issues",
  "cube_reports": [ {
     "name": "Test Status History Report",
     "folder_name": "Xray Reports",
     "result_view": "table",
     "definition": {"columns":{"dimensions":[{"name":"Measures","selected_set":["[Measures].[Xray Tests scheduled]","[Measures].[Xray Tests executing]","[Measures].[Xray Tests executed]","[Measures].[Xray Tests Run count]","[Measures].[Xray average Test Run duration]","[Measures].[Xray Tests execution summary]"],"members":[]}]},"rows":{"dimensions":[{"name":"Xray Test","selected_set":["[Xray Test].[Test].Members"],"members":[],"bookmarked_members":[]}],"nonempty_crossjoin":true},"pages":{"dimensions":[{"name":"Project","selected_set":["[Project].[All Projects]"],"members":[{"depth":0,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Project].[All Projects]","name":"All Projects","type":"all"}],"bookmarked_members":[],"current_page_members":["[Project].[All Projects]"]},{"name":"Fix Version","selected_set":["[Fix Version].[All Fix Versions]"],"members":[{"depth":0,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Fix Version].[All Fix Versions]","name":"All Fix Versions","type":"all"},{"depth":1,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Fix Version].[Calculator]","key":"CALC","name":"Calculator","parent_full_name":"[Fix Version].[All Fix Versions]"},{"depth":2,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Fix Version].[Calculator].[Unreleased]","name":"Unreleased","parent_full_name":"[Fix Version].[Calculator]"}],"bookmarked_members":[],"current_page_members":["[Fix Version].[All Fix Versions]"]},{"name":"Xray Test Execution Status","selected_set":["[Xray Test Execution Status].[All Xray Test Execution Statuses]"],"members":[{"depth":0,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Xray Test Execution Status].[All Xray Test Execution Statuses]","name":"All Xray Test Execution Statuses","type":"all"}],"bookmarked_members":[],"current_page_members":["[Xray Test Execution Status].[All Xray Test Execution Statuses]"]}]},"options":{},"view":{"current":"table","maximized":true,"table":{}}}
  } ],
  "calculated_members": [{"name":"Xray average Test Run duration","dimension":"Measures","format_string":"MinutesFormatter","annotations":{"group":"Xray","drill_through_dimension_levels":"[Xray Test].[Test], [Xray Test Execution].[Execution]"},"formula":"[Measures].[Xray Test Run duration] / 60"},{"name":"Xray Tests execution summary","dimension":"Measures","annotations":{"group":"Xray","drill_through_dimension_levels":"[Xray Test].[Test], [Xray Test Execution].[Execution]"},"formula":"Generate(\n  Filter(\n    Descendants([Xray Test Execution Status].CurrentMember,\n      [Xray Test Execution Status].[Status]\n    ),\n    [Measures].[Xray Tests executed]\n    + [Measures].[Xray Tests scheduled]\n    + [Measures].[Xray Tests executing] > 0\n  ),\n  [Xray Test Execution Status].CurrentMember.Name || ' ' ||\n  Format([Measures].[Xray Tests executed]\n         + [Measures].[Xray Tests scheduled]\n         + [Measures].[Xray Tests executing], '#'),\n  ', '\n)","format_string":""},{"name":"Xray Tests Run count","dimension":"Measures","annotations":{"group":"Xray","drill_through_dimension_levels":"[Xray Test].[Test], [Xray Test Execution].[Execution]"},"formula":"Sum(\n  Filter(Descendants([Xray Test Execution].CurrentMember,\n    [Xray Test Execution].[Execution]),\n    [Measures].[Xray Tests with executions] > 0),\n  [Measures].[Xray Tests with executions]\n)","format_string":""}]
}

Top Defects Report

List all distinct tests who executed with defects. Find out what defects are captured in each version and how many tests does that affect?

Report definition
{
  "cube_name": "Issues",
  "cube_reports": [ {
     "name": "Top Defects",
     "folder_name": "Xray Reports",
     "result_view": "table",
     "definition": {"columns":{"dimensions":[{"name":"Measures","selected_set":["[Measures].[Xray defect status]","[Measures].[Xray defect created]","[Measures].[Xray defect assignee]","[Measures].[Xray Tests executed]"],"members":[]}]},"rows":{"dimensions":[{"name":"Xray Defect","selected_set":["[Xray Defect].[Defect].Members"],"members":[{"depth":2,"full_name":"[Xray Defect].[Agile Projet].[(none)]","key":"(none)","name":"(none)","removed":true},{"depth":2,"full_name":"[Xray Defect].[AmitProject].[(none)]","key":"(none)","name":"(none)","removed":true},{"depth":2,"full_name":"[Xray Defect].[Calculator].[(none)]","key":"(none)","name":"(none)","removed":true},{"depth":2,"full_name":"[Xray Defect].[Sandbox Project].[(none)]","key":"(none)","name":"(none)","removed":true}],"bookmarked_members":[]}],"order_by":{"expression":["[Measures].[Xray Tests executed]"],"order":"DESC"},"filter_by":{"conditions":[{"expression":["[Measures].[Xray Tests executed]"],"operator":">","value":"0"}]}},"pages":{"dimensions":[{"name":"Fix Version","selected_set":["[Fix Version].[All Fix Versions]"],"members":[{"depth":0,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Fix Version].[All Fix Versions]","name":"All Fix Versions","type":"all"},{"depth":1,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Fix Version].[Calculator]","key":"CALC","name":"Calculator","parent_full_name":"[Fix Version].[All Fix Versions]"},{"depth":2,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Fix Version].[Calculator].[Unreleased]","name":"Unreleased","parent_full_name":"[Fix Version].[Calculator]"}],"bookmarked_members":[],"current_page_members":["[Fix Version].[All Fix Versions]"]},{"name":"Xray Test Environments","selected_set":["[Xray Test Environments].[All Xray Test Environments]"],"members":[{"depth":0,"drillable":true,"drilled_into":false,"expanded":true,"full_name":"[Xray Test Environments].[All Xray Test Environments]","name":"All Xray Test Environments","type":"all"}],"bookmarked_members":[],"current_page_members":["[Xray Test Environments].[All Xray Test Environments]"]}]},"options":{},"view":{"current":"table","maximized":true,"table":{}}}
  } ],
  "calculated_members": [{"dimension":"Measures","format_string":"","formula":"-- annotations.group=Xray\n[Status].[Status].getMemberNameByKey(\n  [Xray Defect].CurrentHierarchyMember.get('Status ID')\n)","name":"Xray Defect status"},{"dimension":"Measures","format_string":"yyyy-mm-dd hh:nn:ss","formula":"-- annotations.group=Xray\n[Xray Defect].CurrentHierarchyMember.get('Created at')","name":"Xray Defect created"},{"dimension":"Measures","format_string":"","formula":"-- annotations.group=Xray\n[Assignee].[User].getMemberNameByKey(\n  [Xray Defect].CurrentHierarchyMember.get('Assignee name')\n)","name":"Xray Defect assignee"}]
}
  • No labels