Posted 3rd December 2014
SROI and Cost Benefit Analysis: Spot the Difference, or Chalk and Cheese?
This blog will aim to look at similarities and differences by comparing the two from the perspective of each SROI principle.
Though SROI does draw from Cost Benefit Analysis (CBA), it developed drawing on two other traditions; sustainability accounting and financial accounting.
1. Stakeholder involvement
This principle is fundamental to the SROI approach, and is followed in all aspects of SROI. It is especially important to involve stakeholders when trying to determine outcomes, or the changes that result from an activity.
CBA can focus on a particular policy issue that is being considered, and doesn’t implicitly require involvement in deciding what outcomes are, though they may be consulted.
2. Understand change
Both CBA and SROI focus specifically on change, predominantly change in situation, capacity or behaviour with related changes in wellbeing. The only slight difference is that SROI has an assurance process to ensure completeness of change.
3. Only include what is material
This principle identifies the most notable difference between the two approaches.
CBA is an aspect of welfare economics, so begins from the perspective that all welfare effects will be included. In practice, however, it often focuses on a particular policy outcome with some recognition of unintended consequences. Whilst it is not possible to consider all welfare effects, focusing on a policy objective without stakeholder involvement risks omission of important effects.
SROI on the other hand recognises these limitations and aims to include material outcomes, drawing on financial and sustainability reporting, which hold materiality as a central tenet.
Materiality is not the same as proportionality. Materiality requires a decision to ensure that the outcomes included, both positive and negative, are those that, if omitted, would affect the decisions of the stakeholders. This process requires a judgement on the extent to which stakeholder groups are split into smaller groups which experience different material outcomes – a balance is required between considering every stakeholder as an individual case with a personal outcome profile, and aggregating stakeholders together and so risking a loss of understanding how different smaller groups experience different outcomes.
Proportionality means that the extent or depth of the analysis should be tailored to the relative size, impacts, and risks of the activity under analysis.
4. Do not overclaim
This need to consider benchmarks, counterfactual, attribution and displacement is shared by both approaches.
5. Value what matters
Whilst both approaches use money to value benefits, the valuation technique and/or perspective from which the valuation is taken can differ.
Many CBAs focus on value from the perspective of real, potential or imagined savings to the public sector. This value can either be expressed as an indirect financial value to the taxpayer, or sometimes as a proxy for achieving a stated social policy goal.
Although some SROI analyses do use these types of values, they also aim to value outcomes from the perspective of the stakeholder. This could be by using revealed preference, stated preference, or (more recently) wellbeing valuation techniques, applied as appropriate to the outcome or stakeholder.
6. Be transparent
This principle should be another area of overlap between SROI and CBA.
7. Verify the result
SROI follows accounting and some sustainability reporting by requiring appropriate verification of the result. This may well take the form of an external assurance process, but could include a range of other methods of verification (such as going back to the stakeholders and asking their opinion on the results of the report).
The reasoning behind this principle is that an SROI analysis will inevitably include judgements about what is included or excluded, and these judgements need to be reviewed as reasonable or unjust.
CBA does not have an audit process. However, since judgments on what is included and excluded are made and the audience need to know whether these judgements are reasonable, an external assurance of those decisions, acting on behalf of those affected, is required.
Other similarities and differences: Audience and purpose/rigour
CBA tends to be used by the public sector or quasi-public sector, so is often applied with a reasonably high level of rigour.
In contrast, SROI principles can be used at any level of rigour, as long as it is ‘good enough’ for the type of decision it is being used to inform. In this respect it is similar to financial accounting, which aims to provide information that is good/accurate enough predominantly to inform investors, as opposed to social science levels of rigour.
At one end of the spectrum SROI can use similar levels of rigour as CBA, with additional requirements for consideration of materiality and assurance/verification. At the other end of the spectrum a much lower level of rigour could still be good enough for board-level strategic decisions within organisations.
An illustration of this would be the Maryland Scientific Methods Scale. This is a scale that ranges from 1-5, and indicates the level of scientific rigour of a particular study. At the top end of the scale lies Randomised Control Trials (RCTs), which can be used in an SROI context but depends on use and purpose for the SROI analysis.
At the bottom end of the scale would be an assessment of the counterfactual simply by asking beneficiaries what would have happened otherwise when they enter a program, and does not have any other use of control groups etc. This type of rigour would be unlikely to influence policy, but is still useful for designing services.
This different application of rigour in SROI applies to valuation as well; an estimate for a value may be good enough for a particular audience. Similarly, in accounting we see examples of judgement being used in figures such as the bad debt provision.
In summary, whilst CBA and SROI do have areas of overlap, their differences originate from the fact that SROI is additionally informed by financial accounting and sustainability reporting, particularly with respect to materiality, verification of a result, and use of different levels of rigour according to use and audience.