step 4

select data collection methods and tools

why is this step important?

  • Determines the data you need to collect and how to collect it
  • Helps you select the most appropriate methods and tools to measure your outcomes
  • Ensures consistent data collection supporting accurate impact measurement, benchmarking and comparisons

what does this step involve?

  • Decide what data you're going to collect
  • Choose the methods and tools you'll use to collect your data
  • Work out how you're going to collect the data

      Click to read more...

Tools and resources

Outputs

  • Measurement & Evaluation Plan

helpful resources

  • Data Collection Guide
  • 'Accessible Evaluation' Guidance and Case Studies
  • 'Standard Questions' Banks for Adults and Children

      Click to see all tools and resources...

purpose

Clearly defining the data that you need to collect will enable you to effectively measure the impact of your funding stream or project and ensure that the same things are always measured in the same way. This way, you can have clear and consistent benchmarks (points of reference) and make fair comparisons.

Process

In Step 4 you will decide what data you need to collect and select the best methods and tools to use to collect it.

1. decide what you are going to collect

You need to collect data that will tell you if your funding stream or project is working as expected. To do this you will need to think about what information will be most effective to:

  • inform how things are going and what could be improved (process evaluation)
  • show whether the intended outcomes are being or have been achieved (impact evaluation)

The data you collect for impact evaluation should be directly linked to the ‘outcome indicators’ that you defined and agreed in Step 2.

You will get the best understanding of impact if you:

  • look at and compare different kinds of information from a range of sources
  • collect a mixture of quantitative (numbers) and qualitative (descriptive) information

TIP: Qualitative data can include images, videos and quotes as well as written accounts.

Think about whether there is any existing data that you could use to support your process evaluation and impact measurement. This might give you some of the information you need and save you having to collect it yourself. One example is Sport England's Active Lives Survey, which collects and publishes data on sport and physical activity participation in England.

For Sport England investment, funding recipients will need to collect some standard data and report it back to Sport England. What they need to report will depend on the level of evaluation decided for that funding stream or project (see Step 3). Sport England will produce investment guides setting out measurement and evaluation requirements for specific funding streams.

Recommended Actions
1. Decide what data you need to collect for your process evaluation.
2. Decide what data you need to measure the impact of your funding stream or project.
3. Identify any existing data that you plan to use as part of your measurement and evaluation (M&E) approach.

2. choose your data collection methods and tools

There are lots of different ways you can collect data. You’ll need to choose the data collection methods and tools that are most suited to your funding stream or project based on the requirements you have set out in Steps 1, 2 and 3. If possible, speak to your funders, stakeholders, and your target audience and involve them in choosing suitable methods. This is particularly important if your audience has communication or accessibility needs. Examples of data collection methods include surveys, interviews, focus groups and participant observation. 

We have developed an information pack which gives an introduction to the most commonly used data collection methods and tools, such as 'Surveys and Questionnaires':

This information pack will help you to decide on the best way to get the data that you need to collect - to download it click on the image above or use the links on the right.

We also provide guidance on designing evaluation that is accessible for audiences with communication or accessibility needs.

Sport England’s Question Banks

Sport England wants to work with other organisations to standardise the questions that are used to obtain some commonly collected data, such as the characteristics of project participants and how physically active people are. We have developed an adult question bank that sets out the standard questions that you should use when collecting this data from adults and another question bank to be used to collect data from children.

Sport England expects these standard questions to be used when this type of information needs to be collected, in all projects that involve our investment.

The questions are suitable for most adults and children, but you may need to adapt them for particular groups such as adults or children who might have difficulty understanding a question (e.g. where there are language barriers or people with impairments such as learning difficulties). See our Accessible Evaluation resources for more guidance on this.

Funding recipients will also need to think about the tools and methods to use to collect any additional data required to measure progress of outcome indicators, obtain information for learning, or to report to other funders, partners and stakeholders.

Recommended Actions
1. Choose the questions that are relevant for your M&E approach from the Sport England Question Banks.
2. Decide and document the methods and tools you will use to collect the data you need in the 'M&E Plan' template.

3. decide how to collect the data

The final thing you need to decide on are the processes and systems you will use to collect and store the data. There are some bespoke systems that are designed to support the collection of M&E data, however, simple tools (such as Excel) are often all you need.

Just make sure you factor in time and resource for any manual data entry required - transcribing interviews or entering survey responses from printed forms into a spreadsheet can be a time consuming task.

TIP: It's easy to let a system that’s available dictate what you collect and end up without the information you’re looking for. Be sure to find the best system for collecting the data you need.

The key is to think about how much and what type of data you need to collect and what you will need to do with it. This will help you decide which data collection processes and systems will best suit the needs of your funding stream or project.

If you plan to use existing data sources, such as the Active Lives Survey, spend time working out exactly how you plan to use this data and what you hope it will tell you. Most national surveys are designed to track very large populations and therefore may not be suitable for measuring the impact of a funding stream or project on particular groups or local areas. In addition, most official statistics are only released once a year, so they won’t give a short-term view of impact or change.

Sport England funding recipients currently have to report measurement data to us in a standard format, usually via our online portal. Sometimes they also have to use other systems and templates, which means manually keying the information in as the data doesn’t transfer automatically into Sport England’s records. We’re seeking to develop systems that will allow this to be automated in the future.

Recommended Action
Use the 'M&E Plan' template to document how you plan to collect the data that will enable you to measure all of your outcome indicators.

4. think about ethics and data protection

While you are deciding what tools and methods to use for data collection you will also need to think about ethics and data protection. Start by identifying any potential ethical considerations or challenges, and how you might overcome them. For example:

  • Obtaining informed consent from research participants. If your research is with children, young people or vulnerable adults then informed consent must come from a parent or guardian
  • Telling research participants how their data will be used
  • Protecting research participants’ anonymity and confidentiality
  • Ensuring that research subjects can exercise their rights in relation to their personal data (to access, correct, withdraw or delete the data)
  • Minimising any risk of harm to the research participants and researchers
  • Avoiding deceptive practices by providing honest and complete information to participants about what to expect during the research.

The General Data Protection Regulation (GDPR) came into effect on 25th May 2018. Everyone is responsible for using personal data carefully and needs to follow strict rules or ‘Data Protection Principles’. The requirements of the GDPR are encompassed in six Data Protection Principles stating that personal data shall be:

  • processed fairly, lawfully and in a transparent manner
  • collected for specified, explicit and legitimate purposes
  • used in a way that is adequate, relevant and limited to what is necessary
  • accurate and where necessary up to date
  • kept in a form that allows identification of individuals for no longer than necessary
  • processed in a manner that ensures the appropriate security of the personal data, including against unauthorised or unlawful processing.

Guidance on Personal Data

 

External Resources:

Recommended Action
Identify any ethical or data protection considerations relating to your evaluation, and set out the necessary steps for dealing with them.

recommended outputs

Output icon

M&E plan

Before you move on you should be clear about what data you need to collect and the methods and tools you will use. Creating an 'M&E Plan' will help with your planning to ensure you are collecting the data you need to measure your indicators.

M&E Plan TemplateOpens in new window

Framework overview

Evaluation Framework

Welcome to the Sport England Evaluation Framework.

To help us tailor the content on this site to meet your needs, can you tell us which of the following best describes your role?

I work for Sport England I work for A partner organisation
You can change your settings at any time using the menu in the site footer