Many evaluations and logic models only focus on intended outcomes and impacts – but positive or negative unintended results can be important too.

Use these options before a program is implemented to identify possible unintended outcomes and impacts, especially negative impacts (that make things worse not better) that should also be investigated and tracked.

Make sure your data collection remains open to unintended results that you have not anticipated by including some open-ended questions in interviews and questionnaires, and by encouraging reporting of unexpected results.

Once you have identified possible negative consequences use options from the ‘DESCRIBE‘ component to gather information about them if and when they occur.  Make sure your data collection remains open to the unintended and unanticipated by including some open-ended questions in interviews and questionnaires, and by encouraging reporting of unexpected results.

Options

• Key informant interviews:asking experienced people to identify possible negative impacts, based on their experience with similar programs. Program critics can be especially useful.
• Negative programme theory: identifying ways in which program activities might produce negative impacts rather than their intended impacts.
• Risk assessment:identifying the potential negative impacts, their likelihood of occurring and how they might be avoided.
• Six Hats Thinking about unintended results:promoting holistic and lateral thinking in decision-making and evaluation.
• Unusual events reporting:making sure that unforseen events, incidents or outcomes are recorded.

Synonyms:
logic model, programme logic, programme theory, theory of change, causal model, results chain, intervention logic, program logic, program theory

A programme theory explains how an intervention (a project, a programme, a policy, a strategy) is understood to contribute to a chain of results that produce the intended or actual impacts.

It can include positive impacts (which are beneficial) and negative impacts (which are detrimental). It can also show the other factors which contribute to producing impacts, such as context and other projects and programmes.

Different types of diagrams can be used to represent a programme theory.  These are often referred to as logic models, as they show the overall logic of how the intervention is understood to work.

Why is it done?

Programme theory can be used to provide a conceptual framework for monitoring, for evaluation,  or for an integrated monitoring and evaluation framework.

A programme theory can be a very useful way of bringing together existing evidence about a programme, and clarifying where there is agreement and disagreement about how the programme is understood to work, and where there are gaps in the evidence.

It can be used for a single evaluation, for planning cluster evaluations of different projects funded under a single program, or to bring together evidence from multiple evaluations and research.

When is it done?

A programme theory is often developed during the planning stage of a new intervention.  It can also be developed during implementation and even after a programme has finished. ​ When an evaluation is being planned, it is useful to review the programme theory and revise or elaborate it if necessary. .

How is it developed?

A programme theory can be developed by programme staff, by an external evaluator, by programme designers, or collaboratively with the community.

How is it represented?

The diagrams used to represent a programme theory (usually referred to as logic models) can be drawn in different ways.

Sometimes they are shown as a series of boxes (inputs->processes->outputs->outcomes->impacts), sometimes they are shown in a table, sometimes they are shown as a series of results, with activities occurring alongside them rather than just at the start. These different types are shown as different options on this page (below).

Options

Processes for developing a programme theory:

• Articulating mental models:talking individually or in groups with key informants (including programme planners, service implementors and clients) about how they understand an intervention works.
• Backcasting: working backward from a desirable future, to the present in order to determine the feasibility of the idea or project.
• Five Whys:asking questions in order to examine the cause-and-effect relationships that create underlying problems.
• Generic action theories
• Generic change theories
• Group model building:building a logic model in a group, often using sticky notes.
• Previous research and evaluation:using the findings from evaluation and research studies that were previously conducted on the same or closely related areas.
• SWOT Analysis: reflecting on and assessing the Strengths, Weaknesses,Opportunities and Threats of a particular strategy in order to discover how it can best be implemented.

Ways of representing programme theory in a logic model:

• Tiny Tools Results Chain:mapping both positive and negative possible impacts from an intervention
• Logframe: designing, executing and assessing projects by considering the relationships between available resources, planned activities, and desired changes or results.
• Outcomes hierarchy(also known as a theory of change or an outcomes chain): showing a series of outcomes leading up to the final impacts of a project.
• Realist matrix:focusing on one of the steps in an outcomes chain and then identifying the mechanism involved in producing the outcome and the contexts within which this mechanism operates.
• Results chain(also known as a ‘pipeline model’): showing a programme as a series of boxes inputs-> activities-> outputs -> outcomes -> impacts
• Triple column: showing an outcomes hierarchy in the central column

Advice

Advice for choosing between options for representing programme theory

Consider the format that will be familiar to the people who will be using the logic model. Many development organisations expect to see a logframe.

• Results chainlogic models are most appropriate when all the activities are at the beginning of the process, and less useful when there are a series of activities throughout participants’ passage through a programme.
• Logframeshave a very restrictive format, with only 4 levels. Doing separate logframes for different components can improve their usefulness.
• Realist matricesonly focus on one step in the chain.  While they provide better insight into this step, there either need to be a series of them or they need to be used along with other types of logic models to cover the entire process.

Advice for good practice when developing, representing or using programme theory

See our guide to what might be considered inadequate, adequate and good practice.

Approaches

A number of approaches include recommendations about how to develop a logic model as part of undertaking an evaluation:

• Collaborative Outcomes Reporting: using a collaborative approach to developing a logic model.
• Outcome Mapping: focusing on identifying ‘boundary partners’ whose work is not under the control of a programme but who are critically important to its success in achieving its objectives.
• Participatory Impact Pathways Approach: focusing particularly on identifying the networks of actors involved and how these networks need to change in order for the objectives to be achieved.
• Realist evaluation:a form of theory-driven evaluation that seeks to understand what works for whom, where and why, taking into account how context makes a difference to programme results

Resources

• Learning for Sustainability website:  Annotated list of resources about developing and using a theory of change.
• Funnell, S., & Rogers, P. (2011). Purposeful program theory:Effective use of theories of change and logic models. San Francisco, CA: John Wiley & Sons.

It is helpful to develop an initial description of the project, program or policy as part of beginning an evaluation. Checking this with different stakeholders can be a helpful way of beginning to identify where there are disagreements or gaps in what is known about it.

An overview of what’s being evaluated can include information on:

• The rationale: the issue being addressed, what is being done, who is intended to benefit
• The scale of the intervention, budget and resources allocated and stage of implementation
• The roles of partner organizations and other stakeholders involved in implementation
• The implications of contextual factors – geographic, social, political, economic and institutional circumstances can create opportunities or challenges
• Significant changes that have occurred over time – because of changes in contextual factors or lessons learnt

Options

• Existing project description:using existing project descriptions of what is being evaluated.
• Peak experience description: describing a time when the project/program/policy worked particularly well.  This option is part of the “Appreciative Inquiry” approach to evaluation.
• Thumbnail description: briefly describing the project/program/policy

Approaches

• Appreciative Inquiry: a group process which allows an organisation to understand the best elements of itself in order to create a better future.

This cluster of evaluation tasks develops an initial description of the program and how it is understood to work. This can be used to:

• engage stakeholders in the task “understand and engage stakeholders” from the ‘Manage‘ cluster of tasks
• guide choices about what data to collect in the ‘Describe‘ cluster of tasks
• inform testing of causal links when planning how to ‘Understand Causes‘

Tasks

There are three tasks involved in developing an initial program description and how it is understood to work:

1. Develop an initial description

Developing an initial description of the project/program/policy.

2. Develop program theory/logic model

It is usually helpful to illustrate how your project/program/policy is understood to work by developing a causal model that links activities with intended outcomes. You can then use the program theory or logic model to inform monitoring and evaluation planning.

3. Identify potential unintended results

It is useful to think about possible negative impacts (that make things worse not better) and how they can be identified before a program is implemented.

Source:
Define. (n.d.). Retrieved January 26, 2017, from http://betterevaluation.org/en/plan/define