Exercise 15.2 - Creating TestRail Artifact and Relation Types for Mapping
Objectives
The new learning objectives of this exercise are to create a TestRail artifact type in Syndeia, use it to create a custom SysML-TestRail mapping, and to use the mapping to create a model transform connection. Note that the Syndeia TestRail interface has no default model transform mappings.
Preparation
This exercise assumes the student has
Cameo System Modeler 18.0 - 19.0 (or MagicDraw with SysML plug-in) installed correctly on his or her machine with a valid license for use, and
Syndeia 3.3 installed as a plugin for Cameo/MagicDraw with a valid license for use
Syndeia TestRail interface, correctly installed with a valid license for user.
The user can. It is generally advisable to carry out these exercises in a non-production repository, a "sandbox", set up for training and practice purposes.
We recommend that the student view the videos 1.6, 1.7, and 1.8 in the Introduction Section of the Syndeia training materials before attempting this exercise. These videos provide important background on creating custom mappings. Hands-on tutorials omit much of the logic behind the actions in the interest of clarity and brevity.
We will use the MagicDraw project Syndeia_TestRail_Tutorial.mdzip (get it here - https://intercax.atlassian.net/wiki/spaces/SYN34/pages/2407364009) provided with the tutorials for this exercise, which loads the profile Syndeia_Mapping_Tutorials_Profile.mdzip. We also use the syndeia34Tutorial.mappings file provided, which has been installed in the .syndeia folder and renamed as syndeia.mappings.
For this exercise, you may use any TestRail repository you have available with the ability to create and edit TestRail elements of types Test Run and Test.
Background – TestRail
TestRail is a test management solution from Gurock Software GMBH. The generic schema as supported by Syndeia is shown in Figure 1.
All elements are contained within a Project.
A Suite is composed of zero-to-many Sections.
A Section is composed of zero-to-many Cases.
A Milestone is composed of zero-to-many Test Runs.
A Test Run is composed of zero-to-many Tests and references a specific Test Suite.
A Test holds a specific test result and references a specific Case.
Figure 1 TestRail Schema
The six relationships shown are the ones supported by Syndeia for custom mapping.
Special Note on TestRail Artifact IDs
Before a custom mapping between SysML and TestRail can be created, Syndeia must be taught to recognize the TestRail artifact and relation types to be used in the mapping. In Step 3 below, you will need to enter a specific TestRail artifact ID as a Template to retrieve a list of artifact attributes. There are two options:
If you have not customized TestRail by adding custom properties, type 1 as the ID. There is always a suite, section, case, or run with id = 1, so you will get the generic out-of-the-box properties.
If you have customized TestRail by adding custom properties, you will need to enter the ID of an artifact with those custom properties. In the figure to the right, the ID of Test Suite AB2 is 136, not S136 (ignore the lead character). The IDs of the Test Cases are 266 and 267, respectively. The same rule holds for Test Runs and Tests. However, the ID of Sections is not displayed in the same way. For Sections, open the Suite that the Section belongs to in your browser, select the Section, and look for the group_id value in the URL at the top, e.g. group_id=48..48 is the ID of that Section.
Exercise
If not already open from the previous exercise, open the SysML project Syndeia_TestRail_Tutorial.mdzip (get it here - https://intercax.atlassian.net/wiki/spaces/SYN34/pages/2407364009), right-click the top-level package, and launch the Syndeia dashboard. Select a Syndeia Cloud project, if necessary and go to the Mappings tab, which should look similar to Figure 2.
Figure 2 Syndeia dashboard, Mappings tab
Our objective is to create a new artifact type, TestRail Run, that could be used in creating a custom mapping. In the left column, right-click on Syndeia Mappings/M 1 Model Types/TestRail, and open the Add TestRail Artifact Type window.
Select a Repository from the pull-down list under TestRail Repository at the top.
Click Get TestRail Types button. Select TestRail Run, the TestRail artifact type to be added to the Model Types list.
Edit the name and description fields, if desired.
Enter the ID of an existing artifact of this type in the Template field before clicking the Retrieve TestRail Properties button. This sample artifact should have all the attributes that you wish to be able to map. See "Special Note on TestRail Artifact IDs" at the beginning of this exercise.
Click Select All to make all attributes available for mapping, or chose a subset by checking or unchecking the boxes in the left column of the attributes table. The final window should look similar to Figure 3
Click OK to save the Artifact type.
Figure 3 The Add TestRail Artifact Type window completed
The names of the new artifact type should appear under TestRail Model Types list in the left column (see Figure 4).
Figure 4 Final Mappings tab in Syndeia dashboard, new artifact type highlighted in red
Our second objective is to create a new SysML-TestRail mapping, Block (Test Run) – TestRail Run. In the left column, right-click on Syndeia Mappings/2 Mapping Groups/[↔] SysML - TestRail and select Add Mapping window (Figure 5).
Figure 5 Mappings window
Choose Source Artifact Type as Block (Test Run) from the pull-down menu.
Choose Target Artifact Type as TestRail Run from the pull-down menu.
Enter Name and Description as shown in Figure 6.
Choose Structure Level as CHILDREN_IMMEDIATE from the pull-down menu.
Choose Include Attributes as Include Attributes from the pull-down menu.
Click Retrieve Available Attribute Definitions below the Attribute Definition Mappings table. Using the Plus button, add the four attribute mappings line-by-line as shown in Figure 6.
Choose Include Relations as Exclude Relations from the pull-down menu.
Click Save Mapping at the bottom before moving on.
Figure 6 Final Mappings window
Execute the model transform just created by dragging TestRail Tutorial Test::Test Run 01 onto a Test Suite in in your TestRail repository with the capability (see Figure 7). Choose the mapping just created, Block (Test Run) – TestRail Run. Note that in TestRail, a Test Run must be associated with a Test Suite. Syndeia makes that association by having the user drop the SysML element on a Test Suite.
Figure 7 Connection Manager tab in Syndeia dashboard, drag Test Run 01 to Master Test Suite
When the model transform is complete, expand the Test Run package in the right column. Test Run 01 is displayed. Note that TestRail has automatically assigned to it a series of Tests corresponding to each of the Test Cases within the Test Suite Master.
Figure 8 Connection Manager tab in Syndeia dashboard after model transform
In the TestRail column, right-click Test Run 01 and select Open to open TestRail. Test Run 01 in the TestRail web browser should look similar to Figure 9.
Figure 9 My Test Plan in JIRA web browser, showing related Issues and Test Cycles after model transform
From the Syndeia dashboard, Connection Browser or Connection Search tab, execute Compare Source and Target on the connection from Test Run 01. The results should appear the same as Figure 10.
Figure 10 Comparison Results across Test Run 01 Connection
The differences in Figure 10 occur because these are set automatically when the TestRail Run is created, but not automatically written back to the SysML block. To sync the values, return to the Connection Browser or Search tabs and execute Sync Target -> Source on the Test Run 01 connection, then repeat Compare Source and Target. The results should now appear similar to Figure 11.
Figure 11 Comparison Results across Test Run 01 Connection after syncing from TestRail