Isabel Vogel emailed the group for advice and suggestions for a review for the UK's Department for International Development (DFID) on how theory of change (ToC) is being used by a range of organisations in international development, how people perceive its strengths and weaknesses, and how they are supporting its effective use. This turned into a useful discussion abut the difference between logframes and ToC’s between KBF members.

Key points from discussion
Key points from this discussion are summarised below with detailed contributions from members (named in brackets) below each heading:

A good logframe embeds a theory of change

•    A good logframe process develops a very robust theory of change which is captured in the interplay between the left and right hand columns (the goal/purpose/output/activity statements and the risks/assumptions column)  (Louise Shaxson)

•    A good logframe has a ToC embedded in its very structure:

•    “If we do these Activities and these assumptions hold, then these Outputs will be delivered. If these Outputs are delivered and these assumptions hold then this Purpose will be achieved, and if this Purpose is achieved and these assumptions hold then this Goal will be achieved. The whole thing is designed around an if...and then series of propositions.” (Rick Davies)

•    This vertical logic of a logframe is often lost sight of as people tend to focus almost exclusively on the "horizontal logic" - i.e. how the indicators fit with the associated narrative statement (Rick Davies)

•    For me, every development intervention has a theory of change embedded in it. Only that some theories of change are not explicit enough but if you listen to inherent assumptions, you can tease out certain implied theories (Charles Dhewa)

A theory of change helps explain the leap from outputs to impact in logframes

•    For most projects, they really struggle with the leaps of logic in the middle of the framework. Impact is easy e.g. health, wealth & happiness; Activities are easy as we know what we want to do! The difficulty is getting from Activities to Impact in only four steps. ToCs really help / force projects to explain to Output to Impact logic, as this is rather hard if you only have one logic step to span this yawning gap! (Julian Barr)

•    We always did struggle with the leap between outputs and outcomes – we recognised that while logframes were good for building dams or sending people to the moon they were much less good for research or development projects.  We often ended up with a series of intermediate outputs and outcomes that were sequenced in much the same way as a ToC.  In those days it was quite a flexible tool, particularly in the middle levels (Louise Shaxson)

There are challenges to representing a Theory of Change that conveys complexity and is easy to understand

•    One the basic challenges with ToC is finding good ways of representing them, which are both simple enough but not too simple (to borrow from Einstein) (Rick Davies)

•    Logframes were never reduced to these oversimplified WordArt arrows which I see cropping up all over the place (Louise Shaxson)

•    A LogFrame is one of a number of ways of representing a project's ToC. It is not the only way, and has problems as well as merits, some of which are elaborated on at the end of this webpage http://mande.co.uk/2008/lists/the-logical-framework-a-list-of-useful-documents/ (Rick Davies)

•    Whatever the tool, it is incumbent on the whole community to ensure that they continue to be used as they were intended, so that they give us valuable rather than just pretty information (Louise Shaxson)

Many of the intricacies behind logframes have got lost and they are not being used as originally intended

•    There’s been a huge collective memory loss about logframes, which are now just accounting templates rather than stakeholder engagement tools (which is how we used to use them). (Louise Shaxson)

•    I think one of the downsides of logframes is that they are a neat way of presenting information – so the intricacies and effort in developing one gets lost and it gets seen as a hoop to jump through (Louise Shaxson)

There is often a tradeoff between the explanatory and persuasive aspects of the underlying logic behind a Theory of Change and logframe

•    Theories of change (and tools that employ them) serve two purposes:
i)    to model a situation to better understand it and programme around it
ii)    to simplify a complex situation to help explain it to others and persuade them of the logic of your proposed intervention (e.g. for funding).

•    And in practice there is often a tradeoff between the explanatory and persuasive aspects of the underlying logic (Ian Thorpe)

Why has ToC taken off so much, and the ToC dimension of the log-frame been sidelined?

•    ToCs have been championed to get people back on track with thinking about what they are doing (Simon Batchelor, forthcoming ILT paper)

•    My sense is that one of the main reasons that logframes have become increasingly sidelined is because they've not been found to be useful in a lot of situations in development. As a result people have increasingly switched to using theories of change instead, which is why they're currently flavour of the month (Al Scott).

Are tools a distraction from the dynamics of development and embedding learning and reflection processes?

•    I am not sure whether grappling with theories of change doesn't detract us from practical dynamics of development (Charles Dhewa)                                 

•    “Are ToC, Logframes and OM (amongst other things) a distraction from the business of embedding learning processes and reflective practice?” (Simon Batchelor, forthcoming ILT paper)

Is theory of change destined for the same fate as logframes?

•    My fear is that theories of change may be destined for exactly the same fate as logframes. I'm detecting a distinct feeling of deja vu in discussions around them (Al Scott)

•    At the moment the Theory of Change is so unfamiliar to people that they have to wrestle with the idea and it forces them to think about their programme.  This has a limited window of opportunity, and may, over time, evolve into just another exercise to get the funds released.  As people become familiar with the ideas behind the ToC, and experts start offering consultancy to develop the ToC for programmes, the key idea behind them, getting people to think, may become as lost as it has with the logical Framework  (Simon Batchelor, forthcoming ILT paper)

Thoughts for future: conducting a comparative analysis of tools and generating ways to ensure robust thinking, reflection, learning and analysis is undertaken whatever tool used

•    It would be interesting to compare accounts of the use of logframes with accounts of the use of theories of change to see what significant differences, if any, have actually been experienced on the ground (Al Scott)

•    How can we facilitate the robust thinking and analysis, and move past the 'tick-box template' effect, whether for log-frames or theory of change? (Isabel Vogel)

•    We think that the Theory of Change is a good tool but may not in the long term tackle the heart of the problem – that there is a need to embed learning (Simon Batchelor, forthcoming ILT paper)

Shared resources

DCED Results Measurement Standard
The Donor Committee for Enterprise Development (DCED) has published a Standard for Measuring Achievements in Private Sector Development (PSD) which uses a theory of change

Peacebuilding with Impact: Defining Theories of Change
CARE International have recently produced a guidance note on ToC as a tool to improve peace building initiatives (in particular see p 3-9)

IDRC evaluations have included theories of change, for example: 
•    IDRC Strategic Evaluation of Capacity Development (pdf): "Doing things better? How capacity development results help bring about change", Peter Taylor and Alfredo Ortiz (2008)
•    Report of an evaluation about the policy influence of an ICT4D network, LIRNEAsia (pdf): Theory of change section (p10-24)

Superwomen example (pdf)
A guided walk through of “Project Superwomen”, a real program that was worked on by evaluators using theory of change.

Blogs on Theory of Change
•    Ian Thorpe: http://kmonadollaraday.wordpress.com/2011/06/23/ive-got-a-theory-of-change/
•    Duncan Green's blog on theories of change offers some useful insights: http://www.oxfamblogs.org/fp2p/?p=5864

Learning about theories of change and embedding reflection (forthcoming)
Impact and Learning Team (ILT) working practice paper

The Logical Framework: A list of useful documents
Useful resource on the MANDE website. Contents include: 1) Explanations of the Logical Framework, 2) Wider discussions of Logic Models, 3) Critiques of the Logical Framework, 4) Alternative versions of the Logical Framework, 5) The Editor’s concerns (about uses of the Logical Framework), 6) Online survey on views and usage of the Logical Framework

THEORY OF CHANGE REVIEW – A report commissioned by Comic Relief
This review was commissioned to capture staff and partners’ experiences in using theory of change; to identify others in development that are using theory of change and analyse their different approaches; and to draw together learning from everyone to inform what Comic Relief does next.