Research‎ > ‎

Traceability

Full traceability among all artifacts involved in software development continues to be a major challenge within software development. We imagine a strategy and platform to solve the problem once and for all.

User Scenarios

When Software development is done in a classic product driven model the process often demands that high-level product descriptions drive feature requirements, which drive detailed design specifications, which are broken down into tasks or change requests, which are implemented in commits, which are verified by instantiated builds and tested with test results and later released to artifacts and deployment packets which must be accompanied by release notes.

Pheew!

The thing is, that all these items, activities and artifacts must trace to each other so that given an arbitrary deployment package one could trace back to find for instance the detailed design specification it was related to.

In many businesses such as medical, safety critical and ISO this traceability is not optional. It's required. And around the Wold there are many software development teams spending many resources on maintaining this traceability analysis for every single update of their product.

Take The US Food & Drug Administration (FDA) as an example: In their guidance for content in pre-market submissions of software in medical devices, they require a "Tracability Analysis":

Traceability Analysis
A Traceability Analysis links together your product design requirements, design specifications, and testing requirements. It also provides a means of tying together identified hazards with the implementation and testing of the mitigations.  (...)The Traceability Analysis commonly consists of a matrix with line items for requirements, specifications and tests, and pointers to hazard mitigations.

And in the general principles for software validation from FDA states that:

A traceability analysis should be conducted to verify that the software design implements all of the software requirements. As a technique for identifying where requirements are not sufficient, the traceability analysis should also verify that all aspects of the design are traceable to software requirements.

It's evident, that within development of software classified as Medical Devices maintaining the traceability report is a huge - and expensive - effort.

There are many requirements management tools out there, but they are all limited to implement traceability among requirements and hazards (text in documents or statements stored in databases) but none of the existing tools are capable of tracing efficiently beyond the requirement scope.

In the other end of the food-chain there are a lot of really good integrations between task management systems and Version Control Systems - but they don't trace well back to requirements or forward to test results.

So even if you were an ambitious software development team, who wanted to maintain your  your traceability analysis automatically and continuously, you would have to tie several open-ended reports together.

We want a generic, extendible strategy and implementation for managing, maintaining, querying, browsing and visualizing traces - among all software related artifacts.

Another issue which might seem far from the FDA regulated demands but nevertheless is related to traceability, is the way dependencies are defined in software. Typically the static dependencies are managed by the programming language itself by clauses such as use, include etc while dynamic dependencies required at compile time, are managed by the build technology such as make, Ant, Maven, MS-Solutions etc. And quite often run-time dependencies are only implicitly managed by assertions; if a dependency is broken, the application fails.

In an effective Continuous Delivery setup, these decencies mus be consistently managed, and we want to be able to do that homogeneously despite the technologies that are used. We simply want to be able to define these software dependencies - regardless what level they appear on - as traces among software related artifacts.

Design Proposal

First of all we need a thorough and theoretically founded discussion of the entity objects in our model: eg: What is a trace? what is a software related artifact? how do they interact? 

We need this, because we want to create a platform, a system and a protocol that can become the core and foundation of a system that is extendible with plugins, that will eventually enable the platform to integrate with any system and thereby interact with any artifact and make it all traceable.

Our idea of the design is roughly this:

We define a set of generic entity objects such as; task, commit, test case, test result, dependency, change set - etc. This is potentially a never ending story, in the future we could trace cars to gas stations ...but for now we'll stick to artifacts related to software development.

Each object is defined by and XSD that describes what a validated object looks like, all object must support some kind of nested composite structure, that will enable it to carry - a yet undefined amount of - data.

The objects will be persisted in XML, validated by the XSD's.

Any object should be able to trace to any other object, and traces should themselves be objects that can carry - a unknown amount of - data.

The core system itself is like a bus that carries the objects - from and to any system that wants to interact with it.

So for any system that produces software related artifacts that should be traceable the task i  to classify the data into an object, put it on the bus, and make it available.

Systems will be hooked up to the platform by the means of plugins. A plugin for Jira, Git, Redmine, Subversion, IBM Rational Quality Manager, MS-Visual Studio, Artifactory, Doors, etc, it should really be unlimited.

In many ways this systems is comparable to the Jenkins CI system which basically takes the simple responsibility of orchestrating and automating other tools - and capture and display their outputs. One of the reasons behind Jenkins' success is that it's really quite unpretentious. It doesn't try to rule the entire world, it just offers to be the the baton in the hands of the conductor - a simple and humble tool.

We would like to introduce a sibling platform to Jenkins CI. A platform that offers to keep track and trace of all things traceable. But should resemble it in terms of extensibility, ease of use ...and usefulness.

The study of successfulness is really something that we would like to add to the this assignment: The study of what makes an Open Source system successful.  For instance: Following the conclusions published in Schweik and English: "internet success - A study of Open-Source Software Commons" is it then possible to design a successful Open Source Software platform?

An along with that, we would also like to investigate: How can service companies be profitable on Open Source? To some extend we know part of the answer already: We are a small service firm, we work with, and create Open Source software - and we are profitable. But the question is really; How can we grow from the current eight employees to say 150 - and still develop open source software, and still be profitable? Does our business model scale? 

We believe that in order to do that, we must create - at least one - desirable and popular solution to an important and painful problem.

We believe that this - could be it.

Estimate

Huh! This is a biggie!

We think that this could be scoped to become the basis of a profession PHD. Bit's and parts of the implementations that would derive from such a study/development are already requested by some of our current customers, and it's likely that they could be persuaded to contribute with, genuine user demands, volume testing, feed-back on implementation and required process changes in their organizations - and possibly even some dough.

As laid out previously, this demand for traceability is present within in medical device development. So we believe that stakeholders could be found within this segment.

Automotive and Safety Critical would be other segments that we believe could be turned on by the idea, of  the existence of such a traceability platform.

Status

This assignment is not started - it's up for grabs.

References

http://mitpress.mit.edu/books/internet-success
http://jenkins-ci.org
http://www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/GuidanceDocuments/ucm073778.htm
http://www.fda.gov/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm085281.htm
Call Lars on +45 30 87 25 30 or mail to him on lak@praqma.net if you are interested in this Blue Print.