Outcomes Monitoring as Therapy?

Outcomes and impact mearsurement have become hot topics and for good reason: if you want funding you have to show me why you deserve it. What do you do? What do you achieve? What makes your organisation a good (social) investment? Funders and donors offer support, guidelines, frameworks and systems to help organisations to properly evaluate and monitor their work and best show off their achievements and thus achieve more funding (See the recent NPC blogpost Caring and Sharing).

But there is a danger that with this approach, especially among the smaller charities, outcomes and impact monitoring becomes something external, something imposed from the outside. There is a fear of looking too closely as inevitably the bad will be revealed with the good - I heard this view last week at a meeting to oppose cuts to local services. After listing all the amazing things that the organisation had achieved the chair of the meeting expressed the view that monitoring outcomes was a trick imposed by the government to legitimise cuts. Evaluation and monitoring wasn't something they needed to do, it was something someone else wanted them to do.

So why Outcomes Monitoring as therapy? Well, therapy helps you identify the good as well as the bad. To see what's working and to see what needs to change. And you have therapy for yourself, not because your employer or your spouse or your father tells you to. But the effects trickle over into everything: more confidence; better working practices; stronger relationships.

So while organisations will still need to convince their funders that they are value-for-money they can do that because they know who they are, what they do and how and why they do it well. The motivation for outcomes monitoring must come from within the organisation, only then can funders and donors see an organisation at its best.

Charities Evaluation Service - new review of IT systems

Charities Evaluation Service have just published an ICT overview document helping voluntary organisations that are thinking about using a system to help with their monitoring and evaluation.

There's some useful advice about how to approach this, questions to ask, resources you'll need.

We're also pleased to come out really well!  Fourteen different systems have been reviewed independently and Lamplight was (joint) highest ranked - and all but one of the others cost much more!  So Lamplight gets you as much as the more expensive systems out there.

This is the first time that we're aware of that there's been an attempt to assess and compare different systems.  Up til now we fairly frequently get asked "who are your competitors and why are you better" - which is always a curious question to be asked!  So we're sure that this independent review will help organisations choosing a system.

(A bit more detail on the comparison: systems were ranked high, medium or low on flexibility, outcomes management, data security, and support.  Six of the 14 systems scored 3-high 1-medium, including Lamplight.  Twelve of those fourteen systems are in the 'medium' or 'high' cost bands.  Lamplight also has the second-shortest contract length at 3 months.  The research was carried out during Autumn 2010.)

The comparison document is here.

Bexley Snap join the Lamplight family

We don't often make much of a public fuss when an organisation starts using Lamplight - so it's nice of the organisation (Bexley SNAP) and NCVO to do so for us!  We're (obviously) really pleased that they chose Lamplight and are really looking forward to working with them...

Impact impact everywhere

Maybe it's just what I tend to read, but it feels like there's been more and more about voluntary sector impact around the place recently.  Last Thursday the Guardian ran an overview piece looking briefly at how any why impact measurement is important.

Of course New Philanthropy Capital have been working on this for ages - and they blogged earlier this month about work they're starting on standardising impact (/outcome) measures.  This is a good thing, in our view, but doesn't need inventing from scratch, and crucially needs to be 'free' - measures should be free for any and all to use.  The value of this sort of work would be much increased if data formats could also be agreed to easily allow #opendata learning and benchmarking.

Often the angle that these discussions come from is funding - like this one from NCVO discussing new forms of funding (e.g. social impact bonds) and the link with outcomes, and a similar brief on new forms of funding from Third Sector Foresight.  This is clearly a valid and important perspective (and again from the funders mouth itself - this time the Chief Exec of the Big Lottery Fund) - but it seems rarely to be said that understanding your impact is a good thing for staff, trustees, and probably beneficiaries of a charity too.  We'd like to hear a bit more about these internal benefits.

So my sense is that impact/outcome measures are going to become more and more important.  And in my view, NPC are correct that standards are really important in this discussion.

Firstly, it can be really hard to measure some of this stuff.  At least, it's hard to do well.  Using measures and approaches that have been developed and tested rigorously makes sense.

Secondly, all these measures are of little use at all if they are not communicated, internally and externally.  If you're developing your own impact measures, you've got to explain what and how and why you're measuring, as well as the results.  Which means you'll lose people.  If a set of standards emerges that become more generally understood by trustees, staff, funders and others, it becomes much easier to talk about your impact. And if those standards are trusted (which they'll really need to be) questions over methodology etc. should all-but disappear.

Thirdly, they allow comparison.  This is clearly more controversial: charity league tables anyone?  Martin Brookes at New Philanthropy Capital has spoken and written roughly along these lines - though ranking causes, not individual charities.  But one of the things NPC does is evaluate (ie compare) charities for funders.

There are two kinds of comparison: competitive (we're better than you) and collaborative (what can we learn from you?).  The second shouldn't be too scary.  The first happens already - whenever a funder makes a choice to fund one organisation and not another - it's just that the comparison and process is fairly opaque.  So perhaps it shouldn't be something to be scared of.

This kind of benchmarking would help organisations identify, communicate and celebrate their strengths, and know who to learn from to address weaknesses (or what to stop doing).  It will be rather controversial and perhaps painful.  But will it be a good thing?


[Tuesday 22nd update] The Guardian had a Q&A on this today: http://www.guardian.co.uk/voluntary-sector-network/2011/mar/18/q-a-measuring-impact?commentpage=all#end-of-comments and http://twitter.com/#!/search?q=%23volsecqa (though not much action on twitter).

NCVO conference- Stuart Etherington's state of the sector

At the NCVO conference Stuart Etherington, Chief Exec of NCVO has just given his keynote talk on the 'State of the Sector'.

I'm not sure how far his opening call that we should not be 'blighted by fear' resonated in the room.  While acknowledging and addressing the short-term challenges, we were encouraged to stop worrying about what the Big Society is and get on a do it.

Anyway, here's some immediate highlights:

Around £13 billion of the sectors' £35bn income comes from the state: so one argument put is that the sector has become overly dependent on the state, and that cutting funding will be A Good Thing by giving the voluntary sector it's independence.  Stuart's counter is that three-quarters of public funding is in the form of contracts - so in fact the state is dependent on the sector.

Stuart summarised the Big Society as

1. the transformation of public services
2. devolution of power
3. voluntary and community action

and outlined his calls to Government to support these, including:

• The sector has great strengths (longer-term outcomes focus; innovative; close to the ground) but also weaknesses in a commissioning world - lack of financial strength (of reserves, access to capital etc).  NCVO are asking government to look at tax policy (which apparently disadvantages the sector), and 'social clauses' in contracts that recognise and reward 'social value'.
• The aims of the Localisation Bill are right; the detail is all important.  Government needs to ensure that the 'right to bid' doesn't disadvantage the voluntary sector over the private.
• Local Authorities need to produce guidelines for Local Authorities to ensure power devolves beyond the Authority and down to communities.
• The importance of campaigning by charities - some MPs dispute this.  But it's essential if power is to pass down to local communities.
• Need to look at new ways to encourage individual giving - it's high, but plateauing.
• We'll all need to look again at consortia, partnerships and mergers.

For what they're worth, some of my initial thoughts:

• Fear and anxiety can be rather crippling.  'Calls to arms' are not pointless but it's going to take more.
• Do charities depend on the state or the state on charities?  Of course it's both; there is mutual dependence, but the balance of power (as almost always) is with the state as customer.
• Social clauses rewarding social value: if it's in a contract, it's surely got to be measurable and measured.  Which is possible, but will need some standardisation.
• I think there's still too much talk of 'benefiting organisations', rather than 'benefiting people'.  If that message gets muddled then calls to government do look like special pleading; if the focus remains on people and changing lives it doesn't.