Evaluation of a Systematic Review of the Effects of Garlic on Serum Lipids

In evaluating a systematic review, it is important to be thorough and complete in assessing the study and methods used. The standard issues to address include the research question, research strategies, sample used in the study, appraisal of quality, data extraction methods, and data analysis including data from qualitative and quantitative research methods. This paper provides a systematic review of a meta-analysis of the effects of garlic consumption on blood serum lipids.

Ried, Tobin, and Fakler (2013) conducted a meta-analysis of the effect of garlic on blood lipids. They studied primary trials which conducted research, with conflicting results, of the "effect of garlic preparations on total cholesterol, low-density lipoprotein cholesterol, high density lipoprotein cholesterol, and triglycerides" (p. 282). The following will detail the standard issues of a systematic review as they relate to this meta-analysis.

Research Question

The first step in a systematic review or meta-analysis is to identify and define what is being studied. "Data cannot be meaningfully collected and integrated until there is a clear sense of what question is being addressed" (Polit & Beck, 2012, p. 655). In their systematic review, Ried et al. (2013) start with a statement that "garlic preparations have been linked to cardiovascular benefits" (p. 282). While not in the form of a question, this serves as the research statement. However, the research statement is very broad. Polit and Beck (2012) state that "the broader the question, the more complex and costly the meta-analysis becomes" (p. 656). Ried et al. (2013) continue with a brief overview and explanation of the concepts and phenomena described in the most current research and results (p. 282). This topic is definitely an appropriate study for the nursing field as garlic preparations are cheaper, require less patient e-monitoring, and have less side effects than medications for cardiovascular health.

Research Strategies

The authors of this systematic review do a good job of reporting their criteria, databases, and keywords. They utilized the Medline and Coch databases as well as Google Scholar to find "randomized, placebo-controlled human trials investigating the effect of garlic on cholesterol" published between the years of 1955 and 2011 (Ried et al., 2013, p. 283). These databases as well as the research criteria are appropriate for this study, and the authors included a list of six keywords which were utilized in the search. To supplement their efforts, the authors also did a "hand-search" to manually review reference lists of published studies and articles to ensure relevance of studies utilized. This widens the available data as "computerized indexing systems were less sophisticated before the mid-1990s" (Polit & Beck, 2012, p. 658). Using this method, the authors were able to expand the scope of research. Lastly, the results in Figure 1 provide a flow diagram to explain the strategy behind selecting trials (Ried et al., 2013, p. 284).

Research Sample

Because of the vast amount of research that is available for certain topics, it is important to plan criteria for sample selection. This keeps the sample size from being too large, and it also ensures credibility. Polit and Beck (2012) explain that when specifying exclusion and inclusion criteria for research, "sampling criteria typically cover substantive, methodological and practical elements. The criteria must stipulate specific variables" including specific outcomes, populations, methodology, and study design (p. 656-657). In this systematic review, the authors specified that to be included in the meta-analysis, trials must be studying adults and be two weeks or longer in duration, contain a true placebo control group, and test garlic as a single active substance (Ried et al., 2013, p. 283). Studies were excluded if required data such as baseline levels or standard deviations were missing. Authors of recent trials were contacted to attempt to obtain missing information.

Out of 63 studies assessed, the authors used 39 studies with a total of 2,298 adult participants (Ried et al., 2013, p. 284). The authors provide a comprehensive explanation and rationale for excluded studies. Tables 1 details the characteristics of the 39 studies and trials included in the systematic review. Table 2 explains the reason for exclusion of the 24 trials and studies which were rejected. The reasons for rejection included insufficient data, combination preparation rather than pure garlic preparation, no placebo control, length of study, population, and low compliance (Ried et al., 2013, p. 285-286). Explanations for exclusion are provided, particularly that renal patients and pregnant participants might be taking medication that lowers cholesterol and skews the data.

Data Extraction and Quality Appraisal

Data was extracted following by three independent investigators following Cochrane Collaboration guidelines. Using three investigators enhances the integrity of the findings as it "allows for an assessment of interpreter agreement, which should be sufficiently high to persuade readers of the review that the recorded information is accurate" (Polit & Beck, 2012, p. 660). A wide range of data was extracted from the studies including number of subjects, study design (parallel or crossover), type and dosage of garlic studied, duration of trial, and cholesterol levels of the control group and the treatment group.

It is important to evaluate the quality of the studies selected for a systematic review in order to determine how much confidence should be held about the results. Polit and Beck (2012) cite the Cochrane Handbook which recommends using a component approach to appraisal rather than a global scale. In this method, individual components of studies such as randomization, blind subjects, and attrition are assessed for quality (p. 659). This is a more reliable method of assessment than a global assessment scale which measures strength of evidence and is not always a valid method of quality appraisal. In the systematic review on the effects of garlic, the authors used a component approach to appraisal based on Cochrane guidelines. They assigned two investigators to independently assess "on the basis of randomization, blinding, loss to follow-up, funding sources, and compliance. High quality was given a score of 1 and low quality a score of 0 for each of the five items" (Ried et al., 2013, p. 283). This helped with selection of studies to include and which to exclude based on low quality. The authors also state that disagreement was resolved by discussion between all authors of the systematic review. This method did not merely measure results but measured component quality in each study.

Data Analysis

General Analysis

The authors of the systematic review utilized a meta-analysis conducted using the Cochrane Program Review Manager version 5.1.6 in order to pool and integrate the data. The data was thoroughly analyzed according to the "effect of garlic on total cholesterol (TC), LDL cholesterol, HDL cholesterol, and triglycerides" and sub-grouped by study or trial duration (Ried et al., 2013, p. 283). Adequate figures and tables were included to explain the findings including figures detailing the findings of the effects of garlic on total serum cholesterol, LDL cholesterol, HDL cholesterol, and triglycerides. Tables explaining sub-group analysis, sensitivity analysis, and side effects are provided. The authors thoroughly explain the findings in text as well as tables and figures.

Quantitative Analysis

The greatest benefit of using a meta-analysis to assess quantitative data is that a meta-analysis is completely objective. It is a statistical analysis that does not take personal opinion or viewpoint into consideration. Polit and Beck (2012) explain that "the essence of a meta-analysis is that information from various studies is used to develop a common metric, the effect size. Effect sizes are averaged across studies, yielding aggregated information" (p. 654). In the systematic review on garlic, the authors conducted a meta-analysis of each independent variable. This was an appropriate choice as the data studied is objective, not subjective, in nature. Additionally, the authors found a high level of heterogeneity across the studies and trials. Figure 2 shows a forest plot detailing the heterogeneity in the effect of garlic on total serum cholesterol (Ried et al., 2013, p. 288). Because some of the trials studied were weighted higher than others based on quality of study, the weighted average of the pooled effect estimate places more emphasis on the results of the studies which were of higher quality.

Ried et al. (2013) followed standard procedures for computing the effect size estimates for each individual outcome studied. For total cholesterol and LDL cholesterol, they found a larger effect size in trials of longer duration than those shorter than 12 weeks in duration (p. 288-289). For the other two results of HDL cholesterol and triglycerides, they found a much smaller effect size (p. 292). Sensitivity analysis of trials was also utilized to influence effect size.

Because the systematic review found high levels of heterogeneity, the authors chose to use a generic inverse variance model and a randomized effects model. A random effects model "assumes that each study estimates different, yet related, true effects and that the various effects are normally distributed around a mean effect size value. A random effects model takes both within- and between-study variation into account" (Polit and Beck, 2012, p. 662-663). Additionally, random effects models produce greater confidence of results in studies with high heterogeneity. In the systematic review, there were clinical or methodological differences which produced heterogeneity such as the study design and the administration of the garlic product. Subgroup analyses in the systematic review included splitting the effect size information into gender, length of study, and administration of garlic. Hence, the random effects model was the appropriate choice.

It is nearly impossible to identify all relevant studies; therefore publication bias must be recognized and addressed. Polit and Beck (2012) recommend using a funnel plot to display effects from individual studies and precision of studies (p. 664-665). In the systematic review, Ried et al. (2013) utilized a funnel plot to investigate the effects of garlic on cholesterol levels and found no publication bias (p. 296).

Qualitative Analysis

A metasynthesis can be defined as the process of examining findings, discovering the features, and combining phenomena into a whole. Polit and Beck (2012) list the most useful metasynthesis types for the nursing field as meta-ethnography, meta-study, qualitative metasynthesis, critical interpretive synthesis, grounded formal theory, and thematic synthesis (p. 667). While the authors of the systematic review on the effects of garlic do not use any of these specific titles, they do provide a comprehensive discussion session which summarizes the findings from studying the trials. This is the metasummary. Ried et al. (2013) mention using sub-group analyses including only non-industry sponsored trials when comparing the findings (p. 294), but this is the only technique discussed in the interpretation of data. Data was interpreted based on the mean difference in total cholesterol, LDL cholesterol, HDL cholesterol, and triglycerides levels as well as noted side effects. The sub-group analysis and sensitivity analysis were the methods used to compute effect sizes. Table 4 in the systematic review displays the findings within the sub-group analysis according to type of garlic, duration of trial, mean baseline of total cholesterol and LDL cholesterol, and industry findings. The sensitivity analysis notes that excluded trials included: trials of low quality and high loss to follow-up, trials involving patients on lipid-lowering medication, and trials with alternative active arms (Ried et al., 2013, p. 290-291).

The authors provide a thorough discussion of the findings including overall reduction in cholesterols and triglycerides, comparison to other cholesterol-lowering medications, and comparison between types of garlic used in the trials. Effect sizes are thoroughly discussed and clinical relevance is explained. However, the authors do not call this a metasynthesis or discuss the phases of a metasynthesis.

Discussions and Conclusions

The authors of the systematic review provide a discussion section which details the conclusions drawn about the data collected. They condense the data findings into easily understood results about the exact numbers in reduction of total serum cholesterol and LDL cholesterol as well as overall findings. They explain that "in the present meta-analysis, subgroup meta-analysis by type of garlic preparation suggests aged garlic extract to be more effective than garlic powder or garlic oil in reducing total serum cholesterol. Garlic powder seemed to be more effective for reducing LDL cholesterol, while garlic oil appeared to be more effective for increasing HDL cholesterol" (Ried et al., 2013, p. 296-297). They go on to explain that this systematic review includes the largest number of trials investigating the effect of garlic preparations on blood lipids and that it is the first systematic review to report on side effects.

Ried et al. (2013) detail the limitations of the systematic review. "Due to high heterogeneity in meta-analysis, resulting effect sizes should be interpreted with some caution…. The presence of additional factors may explain the high variability between studies. Available data were insufficient to undertake subgroup analysis by dosage of active ingredient, likely a confounding factor" (p. 297).

Applicability to the field of nursing was explained as the findings relate to cardiovascular health. "The observed 8% reduction in total serum cholesterol with garlic in subjects with elevated cholesterol is of clinical relevance and is associated with a 38% reduction in risk of coronary events at age 50. Similarly, the 9% reduction in LDL cholesterol levels may contribute to a 6% reduction in risk of adverse coronary and vascular events" (Ried et al., 2013, p. 294).

Conclusion

This systematic review is helpful for the nursing community as it provides a thorough meta-analysis and discussion of the effects of garlic on serum lipid levels. The systematic review provides a solid research statement and describes the methods and techniques utilized to choose sample, obtain data, and perform the meta-analysis. Most importantly, there is clinical relevance which can be applied to the nursing field.

References

Polit, Denise F., & Cheryl Tatano Beck. Systematic Reviews of Research Evidence: Meta-Analysis, Metasynthesis, and Mixed Study Review. Resource manual for nursing research: generating and assessing evidence for nursing practice. 9th ed. Philadelphia: Wolters Kluwer Health/lippincott Williams & Wilkins, 2012. 653-676.

Ried, K., Toben, C., & Fakler, P. (2013). Effect of garlic on serum lipids: An updated meta-analysis. Nutrition Reviews, 71(5), 282-299.