Herramientas de usuario

Herramientas del sitio


cursos:cursocochrane:seleccion

# 4: Selecting studies and collecting data

## Selecting studies for inclusion ### Selection process

#### Keeping records Throughout the selection process, you should keep careful records about the studies excluded from the review. In presenting the results of your review, you will be required to:

  • give a summary of the total number of records identified in your search
  • identify the number excluded at each stage of the screening process
  • provide reasons for any articles excluded when assessed in full text
  • present a PRISMA flow diagram.

#### The PRISMA flow diagram Like other elements of your review, your search and selection process should be clearly reported. The PRISMA statement on reporting in systematic reviews includes a flow diagram to help you do this, and your selection process should be documented in sufficient detail to complete the flow diagram.

The PRISMA flow diagram maps out the number of records identified, included and excluded at each stage of the screening process. The flow diagram also includes space to provide a brief summary of the reasons for excluding any records.

See the link in the Resources list to find out more about PRISMA.

#### Working through the flow diagram Let’s walk through the stages you should go through when completing the PRISMA flow diagram.

### Minimizing bias in selection Selecting which studies to include or exclude from your review is perhaps the most important decision you will make. Including or excluding a study can change the magnitude of effect or even the overall significance of a meta-analysis. In making these judgements, it’s essential to take steps to minimize any possible bias.

#### Pre-specified inclusion criteria Each record identified through your search should be carefully compared against your inclusion criteria, and must meet all of these to be included in your review. This is why it is crucial to have clear, pre-specified criteria from your protocol to help you make these decisions as objectively as possible. The PICOS format for defining eligibility criteria is described in module 2: Writing the review protocol.

Having a written summary or checklist for authors to have on hand when screening search results can be helpful – whether on paper or electronically as a form or spreadsheet. As soon as a study fails to meet one of the criteria, it can be excluded and there’s no need to consider it further.

#### Considering study design as inclusion criterion Study design is commonly part of your inclusion criteria, and many Cochrane systematic reviews of interventions include only randomized controlled trials (RCTs). Cochrane Crowd is a citizen science platform that can help you learn about and practice identifying RCTs through real time screening of abstracts. There’s a useful tutorial available, and all of your contributions help Cochrane identify real RCTs for inclusion in Cochrane's Central Register of Controlled Trials (CENTRAL).

You might like to try completing the Cochrane Crowd tutorial, and then challenge yourself to screen a few real abstracts.

Cochrane Crowd is open to anyone and you can find the link in the Resources list for this module. If you used a Cochrane account to access these training modules, you can use the same account to log in to Cochrane Crowd. If not, you can set up a login and get screening in a few minutes.

## Collecting data

### What to look for In a broad sense, we consider ‘data’ to be any information about (or deriving from) a study, including details of methods, participants, setting, context, interventions, outcomes, results, publications and investigators.

#### Sources of data Most systematic reviews obtain the majority of their data from study reports, which include journal articles, books, dissertations, conference abstracts, trial registry entries and web sites. Another source of data can be correspondence with investigators, from which additional details and even individual patient data can sometimes be obtained.

Regardless of the source of data, you will need to collect a wide range of information about each study:

  • everything you will want to report, and analyse in your review;
  • everything your readers will want to know about your included studies.

Things to consider The following items describe the key considerations about different kinds of data you will be collecting from the included studies. You can also download a PDF Collecting data: items to consider.

### Data collection forms Once you have decided what you need to collect, creating a form will help you systematically look for and collect the data, keep an organized record of what you found, and also track anything of importance that was not reported in the study.

TIP: paper based with a copy of the paper

### Collecting outcome data The best strategy when collecting data is to collect whatever outcome measures are reported in the study. This means you have the most comprehensive picture possible, and you can compare the data across all your included studies before selecting the best analysis approach to take.

#### Mind the details Aside from the results themselves, quite a lot of detail is needed for good reporting of outcomes. Read the following excerpt from the Methods section of a fictitious study and then answer the questions below.

#### Methods 65 participants who complained of day time drowsiness were recruited from a metropolitan area in Melbourne. They were all aged between 20 and 55 years and regularly drank coffee (> 2 cups per day). Participants were excluded if they had scores of less than 50 on a VAS measure of fatigue. After consenting to the study they were randomized to two groups.

One group received a café latte with 100 mg of caffeine added, and the other received an identical-tasting decaffeinated cafè latte. Outcome measures were fatigue (measured on a VAS scale) and irritability (measured on the Irritability Negative Affectivity Subscale (INAS)). Changes from baseline to 30 minutes and six hours post treatment on the VAS fatigue and INAS total scores were compared between treatment groups. If a headache occurred within 24 hours of intervention, we considered it as an adverse effect.

Methods 65 participants who complained of day time drowsiness were recruited from a metropolitan area in Melbourne. They were all aged between 20 and 55 years and regularly drank coffee (> 2 cups per day). Participants were excluded if they had scores of less than 50 on a VAS measure of fatigue. After consenting to the study they were randomized to two groups.

One group received a café latte with 100 mg of caffeine added, and the other received an identical-tasting decaffeinated cafè latte. Outcome measures were fatigue (measured on a VAS scale) and irritability (measured on the Irritability Negative Affectivity Subscale (INAS)). Changes from baseline to 30 minutes and six hours post treatment on the VAS fatigue and INAS total scores were compared between treatment groups. If a headache occurred within 24 hours of intervention, we considered it as an adverse effect.

Check your understanding Can you answer the questions below, based on the information above? You can free-type your answers. Type ‘Unclear’ if you think a piece of information has not been reported in the excerpt. Enter your answers into the spaces below then select Submit. What was considered an adverse effect? What were the time points for measurement of fatigue and irritability? What were the upper and lower limits of the INAS scale? How many participants had their fatigue assessed six hours after the treatment?

What was considered an adverse effect? You should collect information about the definitions used for each outcome, such as diagnostic criteria, or thresholds for definitions.

In this case, a headache was considered an adverse effect if it occurred within 24 hours of intervention.

What were the time points for measurement of fatigue and irritability? It’s important to collect details on the timing of each measure, as each study is likely to report measures at varying time points.

In this case, the outcome was measured 30 minutes and six hours post treatment.

What were the upper and lower limits of the INAS scale? In the example above, it is not clear what are the limits of the scale.

Whenever a study is reporting results measured on a scale, you should try to collect the data in as much detail as possible:

Limits of the scale – e.g. is your pain scale a 0-10, 1-10 or 0-100? The direction of benefit – does a higher score mean better quality of life, or worse? Is the study using a modified version or subsection of the scale? Has the scale been validated? What difference is big enough to be detectable or meaningful to study participants? E.g. on a 10-point pain scale, changes of less than 1.5 points may not be enough to be important. This level of detail is necessary in the context of a systematic review, which compares results across multiple studies. If studies are using different definitions, time points, or scales, you will need to know. Also remember to use meaningful descriptions of outcomes or scales for readers who may not recognize abbreviations or technical terms.

How many participants had their fatigue assessed six hours after the treatment? Based on this excerpt above, we know how many people were enrolled in the study, but not how many were assessed six hours after the treatment. It is common that participant numbers change throughout the study, so keep track of how many people were in each group, and were measured for each outcome at each time point. These changes affect your analysis, and will help you assess the risk of bias for each study.

### Dealing with challenges Your included studies may not provide the data you expected, or any usable data at all. How should you approach these challenges?

An example: blood transfusion This is an example of data collected for a real review, measuring blood transfusion as an outcome. You can see almost every study reported the data in a different way.

Definitions The various studies reported the outcome in several different ways:

the volume of blood transfused; the number of standard units transfused; the volume of blood adjusted for the weight of the patient; the number of patients who had any blood transfused. Some studies may have reported the result in more than one way, appearing more than once on this table.

Statistics Within each of those outcome definitions, the numbers reported also differed:

most reported means, but one reported median; some reported standard errors, some standard deviations; quite a few did not label the numbers they reported. One study reported the results in a graph, meaning the review author would have to measure the graph to work out the numerical results.

What to do? It’s difficult to summarize all these studies together, as they’re not all measuring quite the same thing. You will have to make a choice as to how you handle the complexity:

you may have a preferred measure that you consider more valid than another, or you could choose to report the most common measure; in some cases, you can convert one measure to another (e.g. converting pounds to kilograms, or a standard error to a standard deviation – see module 6: Analysing the data). You should contact the authors if their reporting is unclear, or request more detailed information that may help you report consistent data from each study.

Incompatible results After collecting all the results from your included studies, you may realize that some reported results are not compatible with the majority of studies, and cannot be added to your meta-analysis with the results of the other studies. What should you do with those results?

### Data types and formats

In collecting outcomes from your included studies, you will find data in a number of different types, and formats. Understanding these categories is essential to conducting and interpreting any subsequent meta-analysis.

#### Data types: an overview The following is an overview of the common types of data you may encounter.

Dichotomous Description

There are only two categories (also called binary data).

Examples

  • Awake or asleep.
  • In pain or not in pain.
  • Pregnant or not pregnant.

Data to collect

  • Number of participants with ‘events’ (or in each category).
  • Total number of participants in the experimental group.
  • Total number of participants in the control groups.

It’s not always clear whether the total number of people for whom the outcome was measured is the same as the total sample size at any given time point. Where possible, collect the actual number of people who were measured for each outcome, at each time point. If that information is not available, you should contact the authors to request more information. If that is not possible, you can use the total sample size for the group, but bear in mind that this is not ideal and should be made clear in the interpretation.

Continuous Description

Data that are measured on a scale, rather than counted or categorized. In truly continuous data, the scale consists of equal intervals of agreed value.

Examples

Temperature, height, weight.

Data to collect

  • Mean value of the outcome measurements in each intervention group. (If medians and interquartile ranges are reported, that may indicate that the data is skewed. See module 6: Analysing the data for more information on how to deal with skewed data.)
  • Standard deviation of the outcome measurements in each intervention group.
  • Number of participants on whom the outcome was measured in each intervention group.

Try to get the actual number of people in each group who were measured for this particular outcome at each time point if you can, although you may only be able to find the total number of people randomized to each group.

For further information see Cochrane Handbook Chapter 7.7.3.

Ordinal Description

Categories with a meaningful order, but not true continuous data. Gaps between the categories may not be equal. Depending on the number of categories we can have ‘short’ or ‘long’ ordinal scales. Consequently, the analytical options differ for the two groups.

Examples

Short: Degree of burns (I-IV), school grades (A to F or 5 to 1).

Long: Quality of life scales, pain scales of at least 10 points (commonly treated as continuous data).

Data you need to collect

Data in any form in which they are given in the study report. Analysis depends on whether the data will be converted to dichotomous categories, treated as continuous, or analysed directly as ordinal data. For more information, see module 6: Analysing the data.

See also Cochrane Handbook Chapter 7.7.4.

Counts and rates Description

Outcomes that may happen to each individual more than once can be counted.

Counts are the number of times an event happened to an individual.

Rates occur if counts are measured along with the time over which they are observed.

Examples

  • Number of tick bites per person in a day.
  • Number of falls per person in a year.

Data to collect

Data in any form in which they are given in the study report. For more information on how to analyse time to event data, see module 6: Analysing the data.

See also Cochrane Handbook Chapter 7.7.5.

Time to event Description

This type of data is produced when the study measures the time taken before each participant experiences the outcome of interest.

Examples

  • Time to discharge from hospital.
  • Time to recurrence of migraine.
  • Time to death.

Data you need to collect

Data in any form in which they are given in the study report. For more information on how to analyse time to event data, see module 6: Analysing the data.

See also Cochrane Handbook Chapter 7.7.6.

## Software tools

  • Covidence
  • EPPI reviewer
  • Plot Digitizer

## Protocol considerations You should publish a protocol before you look into the available studies as this reduces the impact of biases, and allows peer review of the planned methods.

### Considerations for your protocol Let's look at what to include in your protocol when considering study selection and data collection.

Study selection

You should include:

  • whether two authors will independently assess studies;
  • process of assessment (e.g. screening abstracts, then full text);
  • how disagreements will be managed;
  • any other methods used to select the studies (including the use of software).

Data collection

You should include:

  • data categories to be collected;
  • whether two authors will independently collect data;
  • piloting and use of instructions for data collection form;
  • how disagreements will be managed;
  • what attempts will be made to obtain or clarify data from study authors;
  • processes for managing missing data.

MECIR standards

MECIR are Cochrane's methodological standards for developing protocols, reviews and updates.

The following MECIR items are relevant when planning the study selection and data collection in your protocol:

  • PR22 – Inclusion decisions;
  • PR 23 – Data collection process;
  • PR24 – Requests for data;
  • PR25 – Data items;
  • PR26 – Missing data.
  • A detailed description of these items and the full list of MECIR standards is available in Resources.
cursos/cursocochrane/seleccion.txt · Última modificación: 2017/12/21 06:00 por 127.0.0.1