Value Analysis: Measuring Performance

SMS provides services and advice on how to evaluate software development productivity, time-to-market, quality and functionality for performance analysis. We can offer appraisal or benchmarking for management and improvement, either periodically or as required.

Effective, relevant and visible metrics are essential to the management of successful change and improvement programmes. It is a well known saying that if you can't measure it, you can't manage it, and this is no less true of software performance.

It is the active use of measurement by creative, client-facing staff that makes it effective. Senior management review of so-called ‘key performance indicators’ (KPI) can help stimulate use, but all too often these KPI are somehow disconnected from day-to-day practice. This can lead to dysfunctional behaviour, especially where ‘measurement targets’ are mistaken for the true ‘business goals’.

Functional Size Measures

FSM measures the functionality delivered by software to the user - ie the output. Most other measurement methods, including concepts such as Story Points, are essentially measuring input. While the output of software projects is not of itself a measure of value delivered, it can be used by management teams as a factor in determining value.

Using effective modern functional sizing methods allows an organisation to:

• establish a baseline
• identify opportunities for improvement
• demonstrate added value delivered by the improvement programme
• quantify supplier performance

Benchmarking provides comparative project performance data which can be used to identify and justify improvement opportunities. It also provides information on ease-of-use and cost-to-implement, which can be used:

• to size requirements, and therefore base estimates on user needs and expectations;
• for better quality control and change management of requirements;
• to plan projects and track progress with respect to the functionality required, delivered and/or changed;
• to normalise performance and product quality measures (e.g. Productivity is expressed as <Size Delivered/Effort Expended>; Velocity is expressed as <Size Delivered/Time To Deliver>; Defect Density is expressed as <#Defects/Size> – all these metrics need a measure of functional size as a key input).

Fast Data Collection

A benchmark enables management and staff understand recent performance with respect to other organisations and competitors.

Benchmarks can be derived for Application Development & Enhancement projects, or for Maintenance & Support activities. They establish baseline measures of performance from which the results of improvement initiatives can be measured.

Once a client has agreed to participate in a benchmark exercise, data must be collected retrospectively for a number of recent projects performed over an agreed period. Our professional consultants perform such work frequently so we can improve data collection times by an order of magnitude, using our Fast Data Collection process.

SMS Functional Size Pilot Study

Any performance measurement or benchmarking exercise will start with some form of scoping exercise. SMS has put together a rapid, low-cost consulting package to help you get started using functional size measurement. Each FSM Pilot Study is designed to be completed in one week and includes:

• An initial discussion to agree the scope, objectives and deliverables of an FSM Pilot Study in the context of your organisation
• A Kick-Off Workshop to explain the process with the project team(s) selected for inclusion in the pilot study
• Application of the chosen FSM technique by experienced consulting analysts to the selected project(s)
• Collection of related data and calculation of derived measurements
• Presentation of the results of the FSM Pilot Study to the key stakeholders in your organisation, including details of the benefits

Support and training

SMS's experience convinces us that most organisations are unlikely to achieve lasting performance improvements without a sustained, successful, measurement regime. We therefore want to ensure that our clients succeed with their adoption, implementation and continued use of software measurement.

Over time, experience has taught SMS that successful adoption of methods such as COSMIC, the IFPUG & MkII FPA methods, GQ[I]M, COCOMO.II.2000, and other estimating techniques, etc. is rarely achieved by providing training only in the form of courses such as the typical 2 day formal class. Recognising this, SMS has developed a broader approach to more effectively transfer knowledge into an organization.

Overview of SMS Measurement Support Services

How are these SMS services related?

The SMS services for Benchmarking and Performance Measurement utilise similar Tools and Techniques but can be regarded as otherwise independent of each other: A scoping study often forms a standard component of a Benchmark exercise.