How many “successful” projects have you been part of, only to discover later that the product or service was never used or implemented?
Introduction
Securing benefits and benefits realization is much talked about but less often put into practice. This case study is about a Technology Programme Business Development Review that I managed for a public sector agency. The main aim of the review was “to improve the overall programme management of technology investment of circa £575k per annum, to ensure projects are delivered to time, cost and quality”. The following pie chart shows the main areas of budget spend.
It was generally felt that there was a need to secure better, more strategic outcomes from the annual investment of several million pounds, by this public body, in its technology programme. The aim was broken down into three key objectives.
Objectives
- ensure governance arrangements are fit for purpose and client focused
- improve the identification, measurement and realization of organizational benefits for all technology projects
- deliver efficiency savings.
The technology programme was managed by a technology board on an ongoing basis. The board was due to set out a new four year strategy, so the review was seen as an opportunity to “feed into and comment on the strategy setting process to ensure that it is operating effectively”.
There are different schools of thought around how and when benefits realization should be looked at. This review highlights why benefits realization ought to be embedded in project assessment processes and business case development. Clearly, mechanisms are also needed, long after a project has completed, to measure whether planned benefits are being delivered.
Scope
The scope of the review included the technology porgramme:
- governance arrangements
- programme management arrangements
- project selection and management arrangements
- process for aligning projects with business needs and current level of fit
- process for measuring organisational benefits and proposals for improvements
- resource management including: balance of in-house skills and knowledge with use of external specialist suppliers; procurement and contract management
- client/user engagement in determining project requirements and outcomes etc.
- identification of areas for efficiency savings
- how the cross cutting themes of equality, sustainability and health are addressed within the technology strategy, programme and projects
Outcomes
The outcomes set for the review were identified as:
- projects are aligned to business needs with delivery to time, cost and quality
- project outcomes are measured in terms of organisational benefits
- projects that deliver efficiency savings.
Approach
The approach I followed was a striped down PRINCE2 project management methodology, with an organizational structure including a Project Sponsor, Project Board and Project Team, plus a start up, evaluation and completion stage, with a management report at each stage.
There were two analytical work streams:
- Benchmarking against good practice
- Efficiency analysis
Benchmarking activities focused on a self assessment carried out by the technology team, against good practice models taken from a London Borough Council, KPMG and Transport for London. The self-assessment was based on the IdEA Project Portfolio Management self-assessment benchmarking model.
The IdEA model is designed to assess levels of maturity against a range of criteria as set out below. It would be verified by an external agency.
| Self Assessment Maturity Model | Vision and Leadership | People and Culture | Processes and Systems |
| Corporately owned and driven at all levels | |||
| Aligned with corporate and service policy, strategy, planning and resource allocation | |||
| Communicated and understood internally and externally | |||
| Managed in accordance with the principles of effective programme and project management | |||
| Outcomes tracked and analysed |
Efficiency analysis focused on identifying the potential for making efficiency savings based on productivity – the relationship between inputs and outputs. A key task for this work stream was “to identify what the programme outputs are and how much they cost to deliver (i.e. the cost of the inputs)”. This would be achieved through a combination of research into how others have identified and quantified technology programme savings and seminars involving relevant staff.
Evaluation
It quickly became apparent to me that, if money spent on technology projects was to support delivery of strategic outcomes, the business case for each project must be assessed against the organisation’s strategic aims and objectives. This was not happening.
The organisation, while it was reasonably proficient at managing projects, was not terribly effective at identifying strategic or corporate investment priorities and programmes. This was reflected in how the technology budget was managed and spent. Instead of technology projects being harnessed to delivering key strategic or corporate programmes, technology was a programme in its own right.
This approach is common and often is a key reason why organisations are seeing a poor return on their investment in technology. Technology budgets need to be harnessed to delivering corporate and strategic priorities and outcomes.
The issues included that:
- demand was being driven by middle managers with no mechanisms to assess which projects were more important – whoever screamed loudest usually got their way
- a plethora of uncoordinated, small scale projects were being agreed and undertaken
- there was a high level of failure in taking projects through to completion due to end user rejection
- there was no joined up thinking, with different departments doing their own thing, e.g. the database system used by finance could not talk to the one used by HR, so the data never really tied up
- change management processes were not working resulting in wasted time and money.
As a result of the self assessment and other evaluation activity including the external verification of our self assessment, a set of eleven areas for development were identfied:
- Reinforce the organisation’s project management methodology, as the standard way of managing internal projects.
- Work towards greater ownership of business case development by internal customers in the early stages of project initiation.
- Introduce stage reviews for long term or complex projects.
- Ensure, in accordance with the existing methodology, that project initiation documents include a clear explanation of how a project contributes to programme and/or corporate objectives and benefits.
- In accordance with the agreed project management methodology, ensure that key project and programme management roles are understood.
- The Technology Projects Newsletter should be better presented so that it shows how projects are contributing to programme and/or corporate objectives and benefits.
- Improve the approach to business change management by ensuring the business requirements rather than technology requirements are the drivers for change.
- Ensure that the next review of the project management methodology includes further guidance on programme management and benefits realisation.
- In accordance with the existing project management methodology, ensure that post project reviews/lessons learned exercises are rigorously carried out with contributions from all participating parties.
- In accordance with the existing project management methodology promote the findings from lessons learned to other teams so they may share in the findings.
- Widely communicate the transition to operational status and ensure the business understands that once operational a service is deemed complete and any further revisions should be submitted by way of planned work/a new project.
Completion
The evaluation process was relatively straight forward, what was more difficult was coming up with a set of recommendations that satisfied all stakeholders and that would be acted on. The main recommendations are summarized below.
Objective 1: ensure governance arrangements are fit for purpose and client focused
The key recommendations centered on the need to improve approaches to identifying business programmes and projects and setting up appropriate structures to manage them, rather than setting up separate arrangements to manage technology projects.
Methods were also needed to firmly move responsibility for outcome delivery into the lap of the business directorates and teams that owned the business outcomes being delivered.
Linked to this it was recommended that a small set of corporate objectives and business priorities – or ‘areas of focus’ – be identified at the start of the planning year, to guide business directorates in identifying associated technology/IT resource requirements.
I also compiled recommendations around the need to put someone in charge of identifying and communicating information on significant ICT innovations, trends, opportunities and risks, of relevance to the work of the organisation. This is vital if an organisation is to stay abreast of and be in a position to effectively harness new development.
Objective 2: improve the identification, measurement and realization of organizational benefits for all technology projects
We recommended that the Technology Strategy Board should be responsible for agreeing an annual corporate benefits framework and ongoing project assessment model.
The project assessment tool would need to include methods to forecast efficiency gains, as a result of the project (being a public body the organisation is required to compile an annual efficiency statement).
Programme and/or Project Managers and Business Change Managers would then have a framework for identifying, at the business case development stage, the potential contribution, of each programme/project, to the delivery of corporate benefits.
We suggested that a few large, ongoing or recently completed technology projects be used as case studies to assess whether benefits were or will be delivered against a draft corporate benefits framework.
Objective 3: deliver efficiency savings
The review did not succeed in verifying current productivity as there was insufficient data to assess the current state. Around £50k of savings were identified, to be gained from making improvements to project change management process. Identifying further efficiency savings would only be possible once the proposed corporate benefits framework and corporate project assessment model was operating effectively.
Outcomes
I worked with the authority to strengthen their annual strategic and corporate planning processes. In tandem with this, at a rather slower pace, corporate, programme management structures were developed. Initially there was little appetite for them, as they would inevitably cut across established political and operational power bases.
In public sector organisations, recommending change, let alone implementing it, is always a challenge, due to the complex political drivers and fragmented leadership matrixes.
Once an organisation has a clear view of its corporate and strategic priorities and a method for managing programmes to implement them, it is a relatively easy task to set up a benefits realization framework, that can support benefits planning and capture. But a good framework is important as it enables more effective prioritizations of investment in technology projects, alongside other investment priorities.
The diagram to the right illustrates a strategic framework for managing benefits realization.
Technology projects and investment in technology is best managed within this context, rather than sitting outside of it.
Notes
Benefits Realization is a term that has been coined by the UK Office of Government Commerce (OGC) as part of their Managing Successful Programmes guidance notes. The OGC uses the following definitions:
Benefit Profile – a complete description of each benefit expected from the programme
Benefits management – benefits are the quantification of the outcomes and are used to direct the programme and inform decision-making along the way. Benefits management is the activity of identifying, optimising and tracking the expected benefits through to their realisation. It is a core activity and a continuous management process running throughout the programme.
Follow this link for more information from OGC on Managing Successful Programmes
The NHS Institute for Innovation and Improvement’s website also includes information on Benefits Realisation and provides an example of a benefits capture and monitoring table.
