Data Science and the Digital Thread | Part 5

In Part 4 of this blog series, we demonstrated the use of graph analysis with Gremlin for analyzing the digital thread. This probably raised several questions in the reader’s mind.

• How do we automate these queries so we don’t need to manually enter each one?
• How can we perform additional analysis on the results with a more general-purpose computer language, preferably one where powerful data science libraries are widely accessible?
• How do we present and record the results of our queries to pass them to others in easily readable form?

For this, we’ll bring Jupyter notebooks, described briefly in Part 2, into play.

What we have been doing with the web dashboard is also available to user-written applications. Syndeia was designed as an API-first set of “ready to integrate” microservices. The user’s API experience is as important as the user interfaces provided by Intercax. The API is fully documented to the Open API standard. Several language clients, including Python, Java and JavaScript are available.

We have built a Jupyter notebook to query a digital thread involving Jama requirements connected to SysML requirements in Teamwork Cloud (TWC) connected to JIRA issues, as shown in Figure 1.

Figure 1  Schema for Jupyter Notebook Example

A full walk-through of the notebook is provided in the accompanying video, but we will highlight a few key features. The code is in Python and several libraries for analysis and visualization are loaded at the beginning. The initial sections also allow the notebook to log into Syndeia Cloud using my credentials. It has been set up to require me to enter my Syndeia Cloud password at the beginning of each session. This could be incorporated into the notebook permanently, but then anyone in possession of the notebook would be able to log into my Syndeia Cloud account.

Figure 2  Cell 12 of Jupyter Notebook with Results (truncated)

Figure 2 shows the code and results for Cell 12, one of the cells which passes a Gremlin query (at top in red) to Syndeia Cloud. The objective of the query is all inter-model relations in Syndeia project UGVZ01 to or from Teamwork Cloud SysML elements. The results are parsed for key (Syndeia identifier) and source and target terminal artifacts (source and target are determined by the direction of the inter-model relation). Using the pandas Python library, the results are formatted as shown at the bottom of the figure. Entries in blue represent live links to the element, which will open the JIRA or Jama browser interface when clicked.  This table can be saved as part of the notebook after the session or copied into another document for distribution.

Figure 3 shows results from a later section of the notebook. Unlike Figure 2, where the results were for a query of the Syndeia Cloud database, Figure 3 shows results obtained from the Jama repository via the Syndeia Cloud REST API. For that access, the Jupyter notebook required me to provide my Jama access credentials so that Syndeia could access information stored only in Jama. This included attributes of Jama requirements such as status and priority, which have been incorporated into the table along with color-coding for easier readability.

Figure 3  Cell 18 of Jupyter Notebook with Results

Figure 4 is simply an example of alternative visualizations available through open-source libraries, such as this graph connectivity chart, using pyvis and network Python libraries.

Figure 4 Cell 20 of Jupyter Notebook with Results

In this blog series, we have tried to support several general observations:

• Data Science is invaluable to the systems engineering of real, large-scale systems.
• Data Science objectives cannot be met with a single algorithm or user interface.
• The Digital Thread is an effective approach to collecting, managing and preparing the data.
• The Digital Thread can be architected so that a wide range of open source and proprietary Data Science tools can be applied.

For those that are interested in learning more about how Syndeia can support the combination of Data Science and the Digital Thread, the Intercax website provides full access to User Guides and Tutorials. Or contact us at info@intercax.com with your questions and start the discussions.

Please note this video does not contain audio.

For more blogs in the series:

• Data Science and the Digital Thread | Part 1
• Data Science and the Digital Thread | Part 2
• Data Science and the Digital Thread | Part 3
• Data Science and the Digital Thread | Part 4
• Data Science and the Digital Thread | Part 5 (This post)

Tweet