Evaluation

Rate this page

Rating statistics for this page

3.1 out of 5 from 42 votes

Breakdown

13 votes

6 votes

6 votes

7 votes

10 votes

As well as finding out what evaluation means and who is responsible for it, this page also provides information on:

Self-evaluation

External evaluation

Evaluation approaches

Ethical considerations in evaluation

What is evaluation?

Monitoring information may be reported on a quarterly or even a monthly basis. Evaluation – taking stock and making judgements – is likely to be carried out less frequently.

Evaluation often uses monitoring information that has been produced over a period of time. However, this is not always available, and evaluation may use one-off methods of enquiry. Often the best combination is bringing together regular monitoring data with additional information gathered against specific key questions.

An organisation should think about evaluation from the start, so that monitoring can be carried out with evaluation in mind.

Who does the evaluation?

Who carries out the evaluation may partly depend on the amount of money you have available for it. There are other issues to consider, such as whether your organisation values building skills of analysis and reflection internally or whether it values involving users.

• Self-evaluation is when an organisation uses its own expertise to carry out evaluation.
• External evaluation is when an outside evaluator is brought in to carry out an evaluation for the organisation.

These might be combined, depending on costs, the amount of time required and available within the organisation, the type of evaluation required and the skills involved.

Self-evaluation

With self-evaluation, monitoring and evaluation is built into the everyday life of the organisation and is an essential part of organisational management and development.

Why do self-evaluation?

• It can build on and complement internal reporting systems.
• It can be more easily integrated with other work processes.
• It can tap into internal knowledge more easily.
• Control and use of the information stays within the organisation.
• It encourages greater learning and a more reflective approach.
• It is more likely to lead to change.
• The process can develop new skills in those involved.
• It may be cheaper.

In a 2007 online survey (of nearly 800 respondents) eighty-one per cent of third sector respondents reported changing how things were done as a result of monitoring and evaluation. Sixty-five per cent had changed products or services as a result of monitoring and evaluation findings.

For more information read First Steps in Evaluation from the Charities Evaluation Services.

External evaluation

External evaluation may be appropriate when an organisation lacks the skills or time to carry out an evaluation itself or wishes to carry out a more in-depth evaluation than its own resources would permit. Sometimes funders will require an external evaluation of a project as a condition of their funding.

Why have an external evaluation?

• External evaluators can bring an external perspective.
• The scale and scope of the evaluation may make it more suitable for experienced evaluators.
• Funders may request an external evaluation and it may be more credible to an external audience.
• It should involve less time commitment internally.

Managing external evaluation

A number of decisions need to be made when you are planning an external evaluation:

• How will you look for and select an evaluator?
• How will the evaluator work with the organisation?
• How much data collection will be done internally and when will internal monitoring data be available?
• When will the report be needed?
• Who is the primary audience for the report?

Evaluation approaches

There is a significant difference in approaches to evaluation. Some stakeholders may value a ‘scientific’ approach, looking for hard measurement and objective evidence. Others may value evaluation that encompasses description and different perspectives.

Evaluating aims and objectives: This approach focuses on monitoring and evaluating an organisation’s progress and effectiveness in delivering planned outputs, outcomes and impacts. Information is gathered against selected indicators.

Case study evaluation:  allows you to examine a particular individual, event or activity in greater detail. This approach may be particularly appropriate when evaluating certain types of projects or for certain audiences. If you have a number of case studies or stories to tell, you will then be able to compare and contrast them and pull out any common themes. You may want to let participants tell their story in their own words. If so, you will need to tape an interview or get permission to quote a diary record or other form of self-report.

Appreciative enquiry: This approach focuses on what has worked well, often telling stories in order to learn about a project and its outcomes.

User-led evaluation: In a user-led evaluation control passes to the users themselves or to user-representatives. This may suit your organisation if your work is particularly focused on building the capacity of users and involving them in the delivery of services. However, it is important to provide enough training and support to users who are leading the evaluation.

Ethical considerations in evaluation

There are important ethical considerations when carrying out monitoring and evaluation. For example:

• Storing information in accordance with data protection guidance
• Obtaining informed consent
• Ensuring that you don’t breach confidentiality
• Making sure that an individual or organisation is not harmed by how you work or what you report
• Observing copyright and intellectual property when using secondary materials (such as articles and other organisation’s data)

Useful links

Resources for self-evaluation

The following provide information on evaluation models and approaches, and research methods and tools:

• Monitoring and evaluation resource guide - (CES) 
• Evaluation Support Scotland
• Proving and Improving - a quality and impact toolkit for social enterprise.

Resources for managing external evaluation

• What to include in an evaluation brief (download from CES)
• How to cost an evaluation (download from CES)
• Getting the best from external evaluation (pdf from Evaluation Support Scotland)

 Further reading

• Ethical Guidelines for research (pdf 179.48kb)
• Good Practice Guidelines for Evaluators
• Practical Monitoring and Evaluation: a guide for voluntary organisations (CES).
  

Have your say

Do you always evaluate your work, projects and activities? What systems do you use to do this most effectively?

Share your knowledge on the Improving performance forum.