Critical Appraisal of Quantitative Evidence

Critical Appraisal of Quantitative Evidence

Critically appraise and evaluate a Level I study. (Meta-analysis attached) This study MUST study your suggested intervention and MUST demonstrate the positive patient outcome that you have identified to build support for your proposed practice change.

For the written descriptions, discuss/follow the outline and components of the Evaluation Table in 2 paragraphs.Critical Appraisal of Quantitative Evidence

Theoretical/Conceptual Framework

Windsor, Jemal & Alessi (2015) conducted a meta-analysis that compared Cognitive Behavioral Therapy (CBT)   impact in reducing substance use among non-Hispanics and non-Hispanics. The intervention used in the management of participants with substance use disorder was based on the cognitive behavioral theory which focused on how rational the thought patterns for individuals were and the association between personal behavior, thoughts and feelings (Windsor, Jemal & Alessi, 2015). The application of cognitive behavioral theory did not include internal mental processes but how problematic or adaptive human behavior could be developed, sustained or eliminated through external reinforcement among Hispanics and non-Hispanics

Purpose of the Study

According to Windsor, Jemal & Alessi (2015), Cognitive behavioral therapy is the most effective intervention used in the management of substance use disorders. However, they noted that most CBT trials predominantly included white samples as compared to blacks. Windsor, Jemal & Alessi (2015) suggested that clinicians and researchers need to be cautious when generalizing the effects of CBT in Hispanics and Blacks population. Therefore, the researchers sought to find out if CBT could be generalized in reducing substance use in ethnoracial minority groups. Therefore, the key research question was:  Can the use of CBT in the management of substance use disorders be generalized among ethnoracial minorities?    Critical Appraisal of Quantitative Evidence


Design and Method

            The researchers selected articles from scientific databases and only identified studies which examined the impact of CBT in the reduction of substance use among adults using keywords such as cognitive behavioral therapy, drug, substance, abuse, addiction, and dependence (Windsor, Jemal & Alessi, 2015). The search yielded 322 articles, studies which had been reviewed by three raters to ensure that they met the inclusion criteria. The final meta-analysis included 15 reports which described 16 studies and contributed 126 effects sizes with the population N=3784 individuals. Based on the articles found, the researchers compared the groups which were receiving CBT (experimental group) and the group which received comparison treatment (12 step treatment) and followed up post-treatment(Windsor, Jemal & Alessi, 2015).

Sample and Setting

The study was a meta-analysis which included 15 research reports that described 126 effect sizes and 16 studies with a specified population of 3784 participants who had taken part in randomized controlled trials.  Of these studies, 4 had been published between 1997 and 2000 and included in the non-Hispanic White subgroup while the remainder of the studies were those published between 2000 and 2012 and included Black Hispanics(Windsor, Jemal & Alessi, 2015).

Major Variables

The dependent variable was cognitive behavioral therapy and the independent variables as mentioned by the researchers included:  race, employment, gender, sample size, substance type, number of CBT sessions and treatment format (individual, group or both) (Windsor, Jemal & Alessi, 2015).Critical Appraisal of Quantitative Evidence

Measurement of Major Variables

Data Analysis

The analysis used descriptive statistics to compare the effect sizes of substance use using comprehensive meta-analysis software. To assess the effectiveness of CBT between Blacks and Hispanics, the researchers divided the studies into two subgroups: one that had more than 70% of the sample who self-identified as non-Hispanic Whites and those which had more than 70% of the sample which identified as Black Hispanics. Based on the racial breakdown for all studies, a meta-regression analysis was done with an aim of examining the impact gender, race and CBT sessions to analyze the effect it had on size (Windsor, Jemal & Alessi, 2015).

Study Findings

            The overall findings revealed that 16 peers reviewed journals were used in the meta-analysis, 37% which had specified their percentage sample of Black Hispanics and only 4 studies which had a sample of 70% as Blacks/ Hispanics. All studies were randomized controlled trials which compared the level of effectiveness of CBT with other treatments on substance use outcomes. Data were examined using statistical analyses of t-tests, ANOVA, GEE, GLM and ANCOVA (Windsor, Jemal & Alessi, 2015).

In the first analysis where 13 studies were used to analyze the overall effect size of CBT vs. comparison groups on substance use outcomes, statistical significance was not attained (g < .01; 95% CI: -.09 – .10; p = .95) and  all studies generally had a low heterogeneity (I2 = .00).  The Blacks/Hispanic studies had a mean effect size which were weak and didn’t attain statistical significance (g = .03; 95% CI: -.25 – .31; p = .83) and had a low heterogeneity (I2 = 17.28).  For non-Hispanic Whites (NHW), the mean effect size was weak and didn’t reach statistical significance (g < -.01; 95% CI: -.10 – .10; p = .98) with a low heterogeneity (I2 = 0.00). The final difference in the effect sizes was g = .02 (Q = .044; p =.83) revealing that there was no significant variation in size between NHW and BH studies in the Comparison versus CBT group.Critical Appraisal of Quantitative Evidence

The pre and posttest comparison on the general effect size of CBT on substance use outcomes was strong g = 1.19 (95% CI: 1.05 – 1.33; p = .00) and had a moderate heterogeneity (I2 = 56.18). For the B/H studies, the mean effect size was strong g (g =.86) with a low heterogeneity (95% CI: .62 – 1.10; p < .001; I2 = 0.00). For the NHW studies, the mean effect size was also strong g = 1.36 with a moderate heterogeneity (95% CI: 1.19 – 1.53; p < .001; I2 = 42.12). Using the random effects analysis, the differential effect size was g =.50 (Q= 11.04; p < .001), an indication that  the NHW studies  had an effect size which was higher  as compared to BH  studies  in the pre and posttest comparison.

Based on the random effects meta-regression analysis,   no significant effects were revealed with regards to study type, gender, format and number of CBT sessions. However, race (slope: -0.49; 95% CI: -.79 – -.20; p < .001), employment (slope: -0.01; 95% CI: .01 – -.02; p < .001), sample size (slope: .002; 95% CI: .00 –.00; p < .001), and study type (slope: -1.15; 95% CI: -1.39 – -.92; p < .001) were undeniably associated with effect size.

Implications to Practice

            The study was a Level I evidence whose major strength was that the randomization facilitated statistical analysis. However, a major limitation of this study is that only a few of the studies used in the meta-analysis met the inclusion criteria and none of them gave special analysis to Hispanics/Blacks. Due to lack of generalizability, the findings cannot be generalized As a result; it was difficult to separate to all populations. Besides, since the studies were limited, analyzing the effect CBT had in reducing substance use was difficult.Critical Appraisal of Quantitative Evidence


Based on the findings of this study, it was found that CBT is a highly useful intervention for the management and reduction of substance use in the Black/Hispanic population. The nature of the change faced by  substance users can be viewed from three dimensions: a) cognitive, where clients are taught how they can identify and change distorted thinking patterns, b) behavioral, where clients acquire skills  through training  to improve their ability to cope with different life’s situations, and c) experientially, where clients are assisted  to  naturally  experiment  and test  the extent that individual  beliefs  on specific events  can be termed as rational.


            In the clinical setting when managing adolescent and adult clients with substance use, clinicians should use CBT in either, individual, group or both forms as they have proven to be highly effective in reducing substance use.


Windsor, L. C., Jemal, A., & Alessi, E. J. (2015). Cognitive behavioral therapy: A meta-analysis of race and substance use outcomes. Cultural Diversity and Ethnic Minority Psychology21(2), 300.Critical Appraisal of Quantitative Evidence