Measuring project success is a key aspect of project management. If a project is delivered on schedule, within budget and satisfying all stakeholders, it should be getting a bright green light. If a project is delivered late and over budget, with all the stakeholders dissatisfied with the outcome, then clearly it should be getting a flashing red light. However, it is not always as simple as that because there is a grey area between these green or red examples.

We need to consider from whose perspective we are viewing the project success. Stakeholders such as the buyer/customer or seller/contractor could have differing opinions.

For example, a cost-plus percentage fee type of project that went 200% over budget could be a financial disaster from the buyer/customer perspective, but could be very lucrative from the seller/contractor perspective. In project management, we usually take the buyer/customer perspective in judging overall project success.

One of the challenges with projects is that they cover a wide range of endeavours from, for example, a student studying to pass an examination (on the simple side of the spectrum), to a huge team of professionals planning and implementing an eight-year programme to stage the Olympic Games (on the complex side of the spectrum).

Can we use similar definitions and success criteria (such as the traditional: on time, on budget, and meeting requirements) for both of these?

This article proposes the following:

• We consider the “endeavour” as comprising of a “project” producing an “outcome”;
• An endeavour’s success should measure the performance of the project management separately from the outcome;
• An endeavour’s success should be measured against a set of customised and weighted success criteria; and
• Project management success and outcome success should be expressed as a percentage, rather than on a “yes” or “no” basis.

Success or failure?

A contentious issue is: do we measure the success of a project against the original approved baselines or the final approved baselines?

If a project encounters force majeure situations, or the customer changes the scope, it would be unfair on the project management team to use the original baseline. But if the project management team was incompetent in defining the scope in the first instance, then subsequent ‘corrective’ scope changes should detract from the success of the project.

Scope creep is another insidious problem. Scope creep is when features and functionality are added (increasing project scope) without addressing the effects on time, costs and resources, or without customer approval.

Let us consider the success of the following four projects, firstly in terms of only the baselines, then in the light of additional criteria.

Figure 1 on the following page shows the results of four projects A, B, C and D with a primary statement (in black text), followed by a qualifying statement (in white text) that can radically change the perception of project or endeavour success.

• Project A significantly exceeds all its final approved baselines;
• Project B exceeds its final approved cost and schedule baselines;
• Project C performs better than all its final approved baselines; and
• Project D meets all its final approved baselines.

It could be said that projects A and B are failures, but C and D are successful.

However, to emphasise how elusive the definition of project success can be, let us now consider the four projects’ success or failure when additional information is added:

• Project A significantly exceeds all its final approved baselines, but all the stakeholders are satisfied with the outcome;
• Project B exceeds its final approved cost and schedule baselines, by $1 and by one day respectively;
• Project C performs better than all its final approved baselines, but all the stakeholders are very dissatisfied with the outcome; and
• Project D meets all its final approved baselines, but the return on investment turns out to be far below expectations.

Now, it could be argued that projects A and B are successes, but C and D
are failures.

A low-cost housing scheme probably would not be considered a failure if it were handed over one day late, provided the cost and quality baselines were achieved.

However, if the opening ceremony of the Olympic Games were a day late, it would have enormous repercussions, and would certainly be considered a project management failure.

If the final actual cost of a project exceeds the final approved budget by $1, it is theoretically a cost management failure, but practically we would consider it on budget and successful.

So where does one draw the over-budget line: at $100, $1 000, $100 000?

Endeavour success

It may be more meaningful if we viewed the success of the project life cycle separately from the operations life cycle.

Typically, the product life cycle comprises a project life cycle followed by an operations life cycle.

The project life cycle comprises phases, for example: concept, development, implementation and close-out. Each phase produces deliverables, culminating in a project product, which is handed over to the end-users to realise benefits over the duration of the operations life cycle (which could be anything from performing a theatrical production for a few days, to operating a nuclear power station over many decades).

But not all projects produce a tangible product – some culminate in a service or a result. It could be misleading to talk of a product life cycle. A more appropriate term would be the “endeavour life cycle”.

In defining a project, the Project Management Institute and Association for Project Management use the word “endeavour”, so this should be an acceptable term.

Sometimes we deal with a programme, which is a group of related projects managed in a co-ordinated way to obtain outcomes or benefits and control not available from managing them individually.

The word “endeavour” is applicable to both project and programme.

The endeavour life cycle spans the project life cycle and the operations life cycle, as shown in Figure 2 below.

Endeavour life cycle = project life cycle + operations life cycle

The project life cycle should be preceded by a business case to justify the effort of undertaking the project.

An endeavour can only be considered a complete success if the project management of the project life cycle’s outputs meets all the project’s scope, time, cost and quality baselines and the client/end-users were satisfied with the project outcomes during the operations life cycle.

We should attempt to measure the endeavour’s success in terms of:

• Output success – Did the project management team achieve the time, cost and quality requirements (project efficiency)?
• Outcome success – How successful was the outcome in achieving the expected benefits (endeavour effectiveness)?

Traditional success metrics

It is not always correct to say that the project has failed if the final cost, schedule or scope exceeds the original baseline. There could have been value-adding scope changes that make the project’s product more valuable.

Force majeure situations, which are always beyond the control of the project manager, may require baseline adjustments as well.

Even if these aspects are taken into account, a simple “yes” or “no” answer to the question: “If a project exceeds its final approved baselines, is it a failure?” is not always sufficient.

It will be useful to separate the judgment of the success of the project management effort from the success of the product in the operations life cycle.

The success equation would be:

Endeavour success = Project management success (efficiency)

Were the project outputs on time, within budget, and to requirements?

+ Operations success (effectiveness)

Did the outputs or outcomes satisfy the end-user and customer needs? Was the return on investment acceptable?

It is suggested that we move away from the “yes” or “no” judgment, and rather use a success percentage based on success criteria that have been rated and weighted in an endeavour success matrix as shown in the next section.

Endeavour success matrix

In the endeavour success matrix approach, the project management performance is separated from the product performance (outcome) in the operational life cycle.

The “yes” or “no” judgment of success is avoided.

Success criteria are identified for the specific project, and each criterion is rated according to how successful its performance is assessed by the
key stakeholders.

Table 1 (top right) suggests an approach to rate the budget, schedule performance or stakeholder satisfaction (this could be shown using three separate tables).

Table 2 above suggests an approach to rating the scope and quality aspects.

The project management office can adopt standard tables for all projects, or adapt them depending on the nature and size of the specific project type.

Each criterion is weighted according to how important its contribution is in achieving endeavour success (Table 3).

The ratings and weightings are multiplied and summated for project management and outcomes respectively.

A percentage success can by reported for each aspect. Worked examples using this approach for projects A and C used previously are shown in Table 4 opposite. (Unlike a football scoreboard, Table 4 does have a comments column.)

Conclusion

The traditional metrics for classifying projects as successful or not, are unsatisfactory. Using a yes/no approach can be misleading. The endeavour success matrix permits drilling down to find out what the success criteria for an endeavour were, to what extent they were achieved, and read a brief comment on each.

The author believes the endeavour success matrix approach gives better insight into the performance of the management of the endeavour during the project life cycle, for which the project manager is accountable; and the successful outcome of the endeavour in the operations life cycle, for which the customer, sponsor or others may be accountable. 

Bibliography

• Project Management Institute – Guide to the Project Management Body of Knowledge (PMBOK®) 4th Edition. www.pmi.org.
• Association for Project Management – Body of Knowledge (BoK) 5th Edition. www.apm.co.uk.

Terry Deacon, PMP

Deacon is the founder and chief executive officer of ProjectPro Management Services based in South Africa.

He has more than 35 years of project
management experience.

Deacon is a consultant and has facilitated project management training in Africa, the Middle East as well as Southeast Asia.

Contact him on This e-mail address is being protected from spambots. You need JavaScript enabled to view it or visit www.projectpro.co.za.

Add comment