Your evaluation can only be as effective as the arrangements you put in place to collect the information, no matter how good your planning has been. Careful management of the data collection and analysis process will ensure that you obtain the right information to properly monitor the progress of your funding stream or project and evaluate its effectiveness.
Process
In Step 5 you’ll put in place and manage the process of collecting, analysing and reporting your measurement and evaluation (M&E) data.
1. arrange how you will collect measurement and evaluation data
The data collection arrangements you need to put in place should reflect the decisions you made at Step 4 about what data you need to collect and the tools you plan to use to collect it.
For all of the data that you plan to collect you’ll need to consider who, when and how you will:
Develop the data collection tools
Collect the data
Collate (bring together) the data
Analyse and report the data
All data collection tools and processes must comply with data protection legislation. The regulations are due to change in May 2018.
TIP: Throughout this step, think carefully about how you can maximise your data collection and response rates. Our guide offers some practical suggestions.
2. Develop the Data Collection Tools
Make clear whose role it is to develop each of the tools you will use to collect your measurement and evaluation data. You may not always need to create these from scratch. There might be existing tools (e.g. surveys, registration forms) that you or others have already developed that you can use. However, you’ll need to check to make sure these will provide all the data you need - and adapt them if necessary.
The types of data collection tools that you typically need to develop include:
TIP: Make sure whoever develops your data collection tools is fully aware of the decisions that have been made in Steps 1-4. The tools must provide the data you need to measure your outcome indicators. For example, if one of your outcomes focuses on men aged 65+, you will need to collect gender and age information for your participants.
3. Collect the Data
You will need to decide and make clear who is responsible for collecting all of the data you need for your measurement and evaluation. For example, if some of your data will come from a participant registration form, it should be clear whose job it is to give people the forms, make sure they fill them in and collect the forms.
You’ll need to think about and agree:
What data you need to collect before the initiative begins (‘baseline’ data)
When you will collect data while the funding stream or project is in progress
How often you will collect data
What data you need to collect at the end of your project (‘endline’ data)
If and when you will collect any follow-up data
TIP: It's essential that you collect your baseline data before people start to take part in project activities. If you don't you won't be able to show the project's true impact.
‘Baseline’ and ‘Endline’ are terms used in measurement and evaluation to describe data that is collected at particular points in the course of the project or intervention:
Baseline data is collected before any participant activities get underway
Endline data is collected after all participant activities have been completed
These are important because collecting the same information at the start and end of any initiative or intervention is one of the best ways to uncover any impact or changes that you have brought about.
To work out if you should collect baseline and endline data, you need to think about your outcome indicators and what you are trying to prove by measuring them. If one of the things you want to do is show the impact that your work is having, you will likely benefit from collecting baseline and endline data.
Recommended Action Decide on the arrangements for developing and implementing your data collection tools and document this in the 'Process, Roles and Responsibilities' template.
4. Collate the Data
Depending on the data collection tools you have used, the data you collect may not all be in one place, making it difficult to analyse or report on it. For example, you might have collected hand written registration forms, which are of limited use until the information they hold is combined together in one place - such as in a spreadsheet, table or database.
Depending on the tools you have used, this could be an automated process (e.g. if you have an online registration system), or it might require a more time consuming task involving transferring information from physical forms or surveys into a spreadsheet or database.
Before you start collecting data you need to think about:
Who will collate the data and when this needs to be done
What they will need to do this (e.g. a tablet or computer)
How often they will need to do this
How long the process will take
5. Analyse and Report the Data
Once you have collected and collated your data, it will be possible to analyse it and produce reports. Before you start this process make a list of all the data you have collected that you think it will be useful to report on. This might include things such as:
Number of participants
Demographics of participants
Number of sessions / activities
A summary of questionnaire responses
Retention and drop-out rates
It is a good idea to plan out any reports you intend to produce. To do this you will need to consider what data you should include and how you want to present the report - for example, will it just be a written report, or does it need to include charts and data tables. Think about who the report is intended for and the types of information they are likely to find most valuable.
The analysis and reporting you do should:
Fulfil any stakeholder requirements (as identified in Step 2)
Provide you with useful information that helps you make decisions
Identify things that enable you to make improvements to your funding stream or project
Show you where things might be going wrong so you can fix them
Give updates on the impact you are having (i.e. progress towards your outcome indicators)
You should make clear who is responsible for analysing your data, who is responsible for reporting the findings and how often this needs to be done. Sport England funding recipients typically have to report back to Sport England every six months, although specific requirements vary by funding stream.
At Sport England you need to make clear:
What the funding recipient will be expected to do
What Sport England will do
Where relevant, what the independent evaluation supplier will do
At a lower level, each of these organisations will then need to identify who specifically will carry out each part of the process for their individual projects.
INSIGHT TEAM SUPPORT
The Sport England Insight Team is available to advise colleagues how to put in place data collection and reporting arrangements.
Recommended Action Decide on the arrangements or requirements for data analysis and reporting and document this in the 'Process, Roles and Responsibilities' template.
Recommended Outputs
Process, Roles & Responsibilities
Before moving on, make sure you have agreed and communicated who has responsibility for all of the tasks involved in collecting, collating, analysing and reporting your M&E data.
purpose
Process
