SMS Services - Benefits of measurement

In all aspects of software development and maintenance, measurement and analysis should be applied to focus money and resources on delivering optimum value to the business user and end customer.

Without an informed and objective view of current performance levels, there is a significant danger that short-term cost-cutting can lead to increased overall costs. It can also compromise the capability - and in worst case, even the viability - of the organisational supply chain.

The cost of getting early expert advice is negligable in relation to the savings to be made. The cost of failing to take the right action at the right time can be catastrophic.

Focus on value pulls quality and cost

All the evidence from high performing organisations shows that focusing on the desired outcome pulls through reduced costs, higher quality and better predictability. Objective analysis of current performance levels reveals the opportunities for improvement. Getting this analysis right first time by using industry experts saves time, money and wasted effort.

Measurement programmes should be focussed on the critical concerns of the business. For measurement to add value to the process of software development and/or maintenance, the goals of the measurement programme must be:

The first consideration in the measurement process is to define the actual measures in numerical terms to be used. This is logically followed by measurement, data collection, validation, collation and usage, finishing with analysis and reporting.

Baselining

Before embarking on any improvement programme or change of sourcing, it is vital to baseline current performance levels. Otherwise, there is no starting point from which to measure improvements. Periodic reassessment allows improvement to be quantified and thus made visible to management and executive sponsors.

SMS has long been distinct in offering both quantitative benchmarks of productivity and contract compliance, and also qualitative assessments of capability, process performance and quality.

Please contact us to arrange a specific industry benchmark or assessment of your choice.

Estimating

Using a reliable method to estimate the size of a planned software development or enhancement underpins accurate estimates of cost, effort and duration.

As soon as the first high level requirements and constraints are known a rough estimate can be produced. As requirements become better understood more techniques can be employed to refine the estimates.

It is almost never too early to estimate. Using Rules Relative Size Scale, very early indications of the likely cost and chances of successful realisation for any given project can be arrived at.

As Systems and Software Application Development & Enhancement (AD&E) and Maintenance & Support (M&S) are labour intensive, most of a project's cost is determined by the Effort expended and Duration of project

Cost is usually calculated from an estimate of effort and duration. Cost may be estimated directly providing a suitable database of past projects is available. Clients have for example used SMS to calculate the replacement cost of software; in this case past project data may not be available (as the assumption is that software will be developed externally). In order to calculate cost in these circumstances we make use of our own data and generally available industry data. Sophisticated parametric estimation models are also used to explore various cost possibilities:

The use of a variety of approaches allows us to produce a triangulation model of the cost thus we can provide an estimate with upper and lower bounds.

The first step in producing an estimate of effort is to obtain a measure (or estimate) of the size of the project in terms of either a functional sizing method such as FPA, or in lines of code. The estimator(s) then makes comparisons with past projects that are similar to the proposed project to arrive at an estimate of the effort. They need to utilise their experience in order to allow for project specific factors, which might influence the amount of effort required.

Once Effort has been established the estimated project Duration or schedule can be completed to include Start Date, Milestone Dates and End Date to provide reliable timescales so a total estimated cost for the project can be produced.

Fast Data Collection

A Fast Data Collection Study delivers a complete, consistent and verified benchmark submission in the form either of a populated data collection tool or a set of paper forms, whichever is the agreed instrument. The use of experienced specialists minimises the time needed to collect the data (and hence to deliver the report). Feedback is more timely, relevant and useful. The impact on existing staff and schedules is also minimized.

The Full Study Report will briefly summarise the scope and execution procedure of the study and provide early feedback to management regarding results and ‘quick-win’ improvement opportunities. The submission will contain estimates of the precision of the collected data.

Consistent data is collected and reviewed by experienced software measurement practitioners. An Issues Log, with details of status and resolution, will be maintained, as will a ‘Who’s Who’ catalogue detailing all contributors and information sources.

Function Point Audit

SMS conducts measurement practice audits for all variants of function point measurement. Our standard FP Audit Process evaluates:

the process followed
the results produced
and the conformance to agreed standards (taking account of any previously agreed Local Counting Practices and Case Law as necessary).

For any specific FP Analysis, the FP Audit Team will prefer to consult with the developers & FP Analysts as early as possible to reach agreement of the Purpose, Type of Count, Scope, Context, Functional Users and Boundary of the requirements that are the subject of the analysis. We find this helps avoid rework.

Subsequently, the SMS FP Audit Team will expect to receive the results of each completed FP Analysis that is to be audited, along with all the supporting documentation & references. If a sampling approach has been agreed with the client, the SMS FP Audit Team will expect to work with the FP Analysts and developers to plan the sampling regime to ensure that the sample provides adequate coverage of the work analysed.

The FP Audit will include SMS team members ‘sitting in’ as silent observers in at least a sample of the interviews conducted by FP Analysts with the developers. The purpose of this is to evaluate the process of analysis, to determine the likelihood of errors of omission as well as commission.

FP Audit results will be collated and compiled into reports for the client periodically as agreed. It will be the client’s decision whether to share audit results with others.

Functional Size Pilot Study

SMS provides services to set up and run measurement programmes. We can also enable organisations to gather software metrics themselves and advise on effective application.

SMS has put together the following special consulting package to help you get started:

Output Pricing

Basing supplier remuneration on the ‘size of the functionality delivered’ ensures that the supplier is rewarded for improving efficiency. Although the productivity will vary on projects conducted using different processes and technologies, projects conducted using similar processes and technologies ought to exhibit similar productivity. It is perfectly possible, using benchmark data and cost models such as COCOMO.II.2000, to calibrate 'stereotypes' (a.k.a. 'project profiles') for the various types of work commissioned. Appropriate productivity rates for each 'stereotype' can be derived relatively simply, and then a fair and reasonable 'price per size unit' can be negotiated for projects with characteristics similar to the stereotype.

The work commissioned is then paid for based on a measurement of the functionality it delivers to the business users (establishing a link between IT cost and business benefits) multiplied by the agreed 'price per unit size'. Suitable safeguards regarding the audit of size measures, and the control of changes to scope late in the project life-cycle, are usually demanded and built into the contract, but the principles remain very straightforward… and effective. The approach incentivises the developers to focus on the business needs, as the customer only pays for functionality that is traceable to the business requirements.

The stereotypes identify relevant cost drivers. These can be categorised into those that are the sole responsibility of the supplier, those that are the responsibility of the customer, and those for which there is a mutual responsibility. Regular monitoring of process performance and the effects of cost drivers, combined with periodic review and adjustment of the unit prices, assures all the stakeholders that they continue to get a 'fair deal', while motivating the partners to remove unnecessary waste from the work processes.

Many output-based contracts incorporate agreement that the unit prices will progressively reduce over time, to reflect the desired business outcomes for continuous cost reduction and improved processes.

Achieving the goal of continuously declining costs, while improving product quality, increasing velocity and maximising deliveries of value to the business requires discipline and maturity. It requires attention to every step of the end-to-end network of processes. That is, to the entire value stream that takes raw ideas, requirements and effort, and transforms them into working systems and services. One-off, 'point initiatives', and local optimisations can't sustain the required trends. Even outsourcing to a low-cost offshore supplier will be a one-shot wonder unless both the outsourcer and the outsourcee integrate the upstream and downstream processes and incentivise the behaviours necessary for continuous improvement.

Really, it is all about managing risk effectively. Low cost is no good if the supplier fails to fulfill the business expectations

Rapid Application Portfolio Sizing

RAPS is an application of Function Point Analysis (FPA) used to determine a portfolio size in a meaningful and economical way.

Most portfolios are larger than 100,000 FP, which could take a man-year to size without the aid of RAPS. RAPS provides a meaningful and economical approach to an otherwise near-impossible task. It can increase an experienced FP analyst's full FPA average counting rate of 500 FP per day to over 5000 FP/day. In a recent exercise, SMS FP analysts achieved a counting rate of 6000 FP/day, mostly using the most accurate and therefore time-consuming of the approximation techniques.

A RAPS exercise also forces agreement on what exactly the portfolio comprises - vital if you are in the process of outsourcing. It will deliver some high-level documentation of the whole portfolio, which otherwise often does not exist.

RAPS uses established and effective sampling techniques. In the final step of the process, a few of the applications which were sized approximately are re-sized using the fully detailed FP method. This gives a separate check on the accuracy of the sizes obtained by the approximation method.

"Measure twice - cut once"