EFX: Evaluation support for FAIR and X4L Projects

Toolkit Navigation

- Toolkit Introduction
- Project Logic
- The Purpose of Evaluation
- Evaluation Goals and Questions
- Indicators of Achievement
- Evaluation Methods
- Analysis of Data
- Presentation of Results
- Timing Evaluation Activity
- Administrative Considerations
- Ethical Considerations in Evaluation
- General Links

Evaluation Goals and Questions

It is useful to begin an evaluation plan by stating which of the project's goals and objectives are to be subject to evaluation. This does, of course, assume that the goals and objectives have been stated reasonably clearly! However, rather than spend an inordinate amount of time trying to define them in the abstract, it is usually helpful to focus thinking about a project or a programme on outcomes and impacts (see the Project Logic page) - except where the focus is deliberately on evaluating processes and outputs for a particular purpose. Bear in mind the 'systems' model:

Inputs leading to Processes leading to Outputs leading to Outcomes leading to Impacts

The focus of evaluation is on 'What is happening?' or 'What happened?', so the questions you ask need to be quite carefully focussed to produce useful results. A particular issue is being careful to establish clearly cause and effect - see G. E Dallal's paper on this topic.

Some useful formative evaluation approaches are to explore:

Variations which have occurred between the project proposal, the agreement of a project plan and the actual work carried out.
Who is involved in the project. It is especially useful to explore the actual engagement of the different stakeholder groups.
What does the project team see as the main lessons learned to date (including the 'negative' lessons, the things that didn't work as expected).
Which environmental factors are getting in the way. For example, the lack of an agreed technical standard might be hampering development decisions.
How adequate are the resources available.

While a summative evaluation might examine:

The extent to which the original aim and concept was realised.
The lessons which have been learned, for example on the feasibility of different approaches.
Whether the products of the project/programme are scalable and sustainable.
Whether the products and outcomes represent good value for money.
The relationship of the project/programme to others which have been undertaken elsewhere, perhaps in parallel. In other words, is this the optimum solution to the problem?
The process issues which have emerged which might be applied to future projects and programmes.

At the detailed level, it is helpful to try to formulate precise 'research questions' that permit a precise answer. For example, if you are interested in processes:

Ask "Has each milestone been passed satisfactorily?", because you can answer 'Yes' or 'No' (and then go on to elaborate)
Don't ask "Is the project on track?", because the answer is likely to be somewhere between 'Yes' and 'No', and the elaboration will get in the way.