A systems approach to the development and application of technical metrics to systems engineering projects across an enterprise

Large-scale systems engineering projects usually exist within complex problem contexts, which on occasion lead to the usual plethora of project ills such as late delivery, cost overruns, operational issues, and when particularly bad, unwelcome media headlines. The Systems Engineering function within Thales UK seeks to minimise the impact of, or eliminate, these problematic projects. Here, problematic projects are defined as ones where the Systems Engineering group on a project was expected to deliver certain characteristics against cost/time constraints arising from the problem context, however the project did not or will not meet expectations by a significant margin. Thales UK employs areound 3000 engineers, over half of which are systems engineers, delivering solutions from transport systems and secure transactons to integrated communications, naval sensors and air defence systems. A traditional view is that the impact to the Company of problematic projects can be minimized by early detection, and subsequent intervention by systems engineering leadership. As such, Thales UK seeks to implement and approach that will alert SE staff and leadership to the presence or development of problematic projects, such that appropriate interventions can be made.

Literature regarding systems engineering technical metrics explores the development, and less frequently, the use of metrics to provide information to project teams to support judgements about current versus desirable positions. No literature has been identified that describes how systems engineering technical metrics could be used en-masse to provide insight in to the performance of a diverse portfolio of systems engineering projects. Thales UK has mandated the collection of a set of Systems Engineering technical metrics on qualifying projects, and understands that the reported data alone means nothing specific without interpretation and relation to problem context. The problem context of each project is different and dynamic. This presents a challenge when attempting to use this data to draw conclusions regarding the health of all projects across an enterprise, and the health of the operation of an entire engineering function.

In the research activities described in this paper we perform a quasi-experiment with SE leaders and SE technical metrics from a range of domains within Thales UK to test whether a project can be determined to be problematic, or not problematic from metric data alone. A separate research activity develops a grounded theory of a metrics approach that could possibly be used across a diverse range of SE development projects to detect whether SE projects exhibit characteristics of being problematic, or being unable to detect whether they are problematic.