EFX: Evaluation support for FAIR and X4L Projects

Toolkit Navigation

- Toolkit Introduction
- Project Logic
- The Purpose of Evaluation
- Evaluation Goals and Questions
- Indicators of Achievement
- Evaluation Methods
- Analysis of Data
- Presentation of Results
- Timing Evaluation Activity
- Administrative Considerations
- Ethical Considerations in Evaluation
- General Links

Project Logic

Articulating and documenting the 'logic' of your project

The aim of this section of the EFX web Toolkit is to provide enough of a description of our approach to understanding implicit theories of change to allow project teams to carry out their own exercise in 'surfacing' the assumptions embedded in their own work. More detail will be found in the EDNER Project Issue Paper 7.

The point of carrying out such an exercise is to reveal such assumptions, so that differences within and around the project team can be aired, consensus improved and the internal logic of the project enhanced. The approach described here builds on the work of John Nash (Stanford Learning Lab), Leo Plugge and Anneke Eurelings (until recently at the University of Maastricht), and Helge Stromdahl (Royal Institute of Technology, Stockholm) who are among the first people we know to have tried out 'theory-based evaluation' or 'theory-anchored evaluation' in the field of learning technologies (see Nash et al, 2000; Stromdahl & Langerth-Zetterman, 2001 - references are on the General Links page).

Surfacing a project's theory of change involves the following steps:

Each member of the project team (and ideally each important stakeholder) should write down their vision of the project's 'outcome of interest'. One way to do this is to get each person to write a document in response to a 'History of the Future' Exercise.
Copies of the documents thus produced should be given to all members of the project team (and ideally each important stakeholder) and a meeting held to identify common elements, identify key differences and work towards consensus about a definition of the main outcome or outcomes of interest. This should be written down.
At the same or a later meeting, project team members and stakeholders - usually with the aid of a facilitator - should try to create a logic map of the project's theory of change. (For more about how to use logic models to help a project tell its story about change see McLaughlin & Jordan, 1998.) An example of a project logic map is given in Figure 1
The logic map in Figure 1 consists of project outputs (ellipses, given on the right of the map), project activities or inputs (rectangles, given on the left of the map) and intermediate goals (rounded rectangles, in the centre of the map). The definition of the project outcomes begins (and often ends) with agreement about the main outcome(s) of interest. The inputs are the 'big ideas' brought together at the outset of the project. They may reflect resources and activities in the real world or theoretical constructs. The intermediate goals represent states of affairs that bridge between inputs and outputs. Their creation may well be the principal work of the project.
Once an agreed project logic map has been produced, explain the directed links between the inputs and the intermediate goals and between the intermediate goals and the outputs. (It may help to number these links on the map and write paragraphs about each link in an accompanying document.) The work involved in explaining these links is what 'brings to the surface' the project's implicit theory of change.

Over a period, revisit the map, adding detail as appropriate and amending elements in light of experience, changing circumstances etc. Be sure to retain a copy of each main version of the map and its accompanying documentation.

Project Logic Diagram