develop and implement your measurement tools
Why is this step important?
- Supports the practical arrangements that will enable you to collect the data you need for your evaluation
- Ensures that clear processes and roles are agreed for collecting, reporting and analysing your data
What does this step involve?
Put in place arrangements to collect and report M&E data:
- Develop data collection tools
- Collect the data
- Collate the data
- Analyse and report the data
Click to read more...
Tools and Resouces
- Process, Roles & Responsibilities
- Designing and using registration forms
- Maximising your response rates
Click to see all tools and resources...
Your evaluation can only be as effective as the arrangements you put in place to collect the information, no matter how good your planning has been. Careful management of the data collection and analysis process will ensure that you obtain the right information to properly monitor the progress of your funding stream or project and evaluate its effectiveness.
In Step 5 you’ll put in place and manage the process of collecting, analysing and reporting your measurement and evaluation (M&E) data.
1. arrange how you will collect measurement and evaluation data
The data collection arrangements you need to put in place should reflect the decisions you made at Step 4 about what data you need to collect and the tools you plan to use to collect it.
For all of the data that you plan to collect you’ll need to consider who, when and how you will:
- Develop the data collection tools
- Collect the data
- Collate (bring together) the data
- Analyse and report the data
All data collection tools and processes must comply with data protection legislation. The regulations are due to change in May 2018.
TIP: Throughout this step, think carefully about how you can maximise your data collection and response rates. Our guide offers some practical suggestions.
2. Develop the Data Collection Tools
Make clear whose role it is to develop each of the tools you will use to collect your measurement and evaluation data. You may not always need to create these from scratch. There might be existing tools (e.g. surveys, registration forms) that you or others have already developed that you can use. However, you’ll need to check to make sure these will provide all the data you need - and adapt them if necessary.
The types of data collection tools that you typically need to develop include:
- Registration forms (read our guide)
- Membership records
- Attendance records
- Survey questionnaires
- Interview guides
- Activity feedback forms
TIP: Make sure whoever develops your data collection tools is fully aware of the decisions that have been made in Steps 1-4. The tools must provide the data you need to measure your outcome indicators. For example, if one of your outcomes focuses on men aged 65+, you will need to collect gender and age information for your participants.
3. Collect the Data
You will need to decide and make clear who is responsible for collecting all of the data you need for your measurement and evaluation. For example, if some of your data will come from a participant registration form, it should be clear whose job it is to give people the forms, make sure they fill them in and collect the forms.
You’ll need to think about and agree:
- What data you need to collect before the initiative begins (‘baseline’ data)
- When you will collect data while the funding stream or project is in progress
- How often you will collect data
- What data you need to collect at the end of your project (‘endline’ data)
- If and when you will collect any follow-up data
TIP: It's essential that you collect your baseline data before people start to take part in project activities. If you don't you won't be able to show the project's true impact.
Decide on the arrangements for developing and implementing your data collection tools and document this in the 'Process, Roles and Responsibilities' template.
4. Collate the Data
Depending on the data collection tools you have used, the data you collect may not all be in one place, making it difficult to analyse or report on it. For example, you might have collected hand written registration forms, which are of limited use until the information they hold is combined together in one place - such as in a spreadsheet, table or database.
Depending on the tools you have used, this could be an automated process (e.g. if you have an online registration system), or it might require a more time consuming task involving transferring information from physical forms or surveys into a spreadsheet or database.
Before you start collecting data you need to think about:
- Who will collate the data and when this needs to be done
- What they will need to do this (e.g. a tablet or computer)
- How often they will need to do this
- How long the process will take
5. Analyse and Report the Data
Once you have collected and collated your data, it will be possible to analyse it and produce reports. Before you start this process make a list of all the data you have collected that you think it will be useful to report on. This might include things such as:
- Number of participants
- Demographics of participants
- Number of sessions / activities
- A summary of questionnaire responses
- Retention and drop-out rates
It is a good idea to plan out any reports you intend to produce. To do this you will need to consider what data you should include and how you want to present the report - for example, will it just be a written report, or does it need to include charts and data tables. Think about who the report is intended for and the types of information they are likely to find most valuable.
The analysis and reporting you do should:
- Fulfil any stakeholder requirements (as identified in Step 2)
- Provide you with useful information that helps you make decisions
- Identify things that enable you to make improvements to your funding stream or project
- Show you where things might be going wrong so you can fix them
- Give updates on the impact you are having (i.e. progress towards your outcome indicators)
You should make clear who is responsible for analysing your data, who is responsible for reporting the findings and how often this needs to be done. Sport England funding recipients typically have to report back to Sport England every six months, although specific requirements vary by funding stream.
Not Sport England staff?
At Sport England you need to make clear:
- What the funding recipient will be expected to do
- What Sport England will do
- Where relevant, what the independent evaluation supplier will do
At a lower level, each of these organisations will then need to identify who specifically will carry out each part of the process for their individual projects.
INSIGHT TEAM SUPPORT
The Sport England Insight Team is available to advise colleagues how to put in place data collection and reporting arrangements.
Decide on the arrangements or requirements for data analysis and reporting and document this in the 'Process, Roles and Responsibilities' template.