HOOFDSTUK 4. THE EFFECTIVENESS OF INSTITUTIONAL … 4.pdf1 HOOFDSTUK 4. HOOFDSTUK 4. THE...

VU Research Portal

De professionele jeugdzorgwerker

de Swart, J.J.W.

2011

document versionPublisher's PDF, also known as Version of record

Link to publication in VU Research Portal

citation for published version (APA)de Swart, J. J. W. (2011). De professionele jeugdzorgwerker: Kenmerken van jeugdzorgwerkers in relatie totkwaliteit van de jeugdzorg.

General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.

• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal ?

Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.

E-mail address:[email protected]

Download date: 22. Mar. 2021

https://research.vu.nl/en/publications/2f3126b5-bfdb-4a0d-8f49-de5c38614205

1

HOOFDSTUK 4. HOOFDSTUK 4. HOOFDSTUK 4. HOOFDSTUK 4.

THE EFFECTIVENESS OF INSTITUTIONAL YOUTH CARE OVER THE PAST

THREE DECADES: A META-ANALYSIS1

1 De Swart, J. J. W., Van den Broek, H., Stams, G. J. J. M., Asscher, J. J., Van der Laan, P. H., Holsbrink - Engels, G. A, & Van der Helm, G. H. P. (2011). The effectiveness of institutional youth care. Manuscript submitted for publication.

2

AbstractAbstractAbstractAbstract

A meta-analysis was conducted of 27 controlled studies, including 17.038 youth, to examine the

effectiveness of institutional youth care over the past three decades. We compared institutional

Evidence-Based Treatment (EBT) with non-institutional EBT, institutional Care As Usual (CAU)

with non-institutional CAU, institutional CAU with non-institutional EBT, and institutional EBT with

institutional CAU. Only the last comparison yielded a significant and small-to-medium effect (d =

.36), which not only shows that institutional youth care can be equally effective as non-

institutional youth care, but also demonstrates that it seems more effective to provide youth

with EBT during their stay in the institution. Furthermore, moderator analyses indicated that only

cognitive behavior therapy showed a significant and medium effect (d = .50), whereas skills

training and care as usual showed no effect. Age, gender, type of outcome measure

(delinquency, skills, problem behavior, and overall problems) and study design characteristics

(randomized, matched or non-matched controls) did not influence the results.

keywords: keywords: keywords: keywords: institutional care, group homes, evidence-based treatment, incarcerated, meta-

analysis, effectiveness

IntroductionIntroductionIntroductionIntroduction

Although there are only a small percentage of children in institutional care, about three to eight

percent of the children who seek treatment according to Hair (2005), the interest in the

effectiveness of institutional youth care has grown substantially over the years (Lyons, Woltman,

Martinovich, & Hancock, 2009). Institutional youth care is carried out in 24-hour group living

facilities that are not licensed as hospitals and offer care or mental health treatment for

children and adolescents with serious behaviour problems, including law violations. The daily

living environment is often used as a therapeutic means (Bates, English, & Kouidou-Giles, 1997;

Hair, 2005). There is a wide variety of institutional services, with juveniles living in groups of

different sizes, different treatment methods (French & Cameron, 2002) and specialized services

for different types of problem behaviour (Barth, 2002).

The increased interest in the outcomes of institutional youth care can be explained by

two factors. First, scientific knowledge about effective interventions in youth care in general and

in juvenile justice increased considerably and promoted evidence-based working in

(institutional) youth care as well (Lyons, Woltman, Martinovich, & Hancock, 2009). Second,

institutional youth care is the most drastic and the most expensive type of youth care (Barth,

2002; Hoagwood, Burns, Kiser, Ringweisen & Schoenwald, 2001; Knorth, Harder, Zandberg &

Kendrick, 2008), which resulted in putting even more pressure on this sector to demonstrate

effective outcomes than in non-institutional youth care (Lyons et al., 2009).

3

Equivocal results of institutional care interventions (Scherrer, 1994), high costs and the

general scepticism that institutional care has a negative effect on children and society turned

institutional care into a treatment of ‘the last resorts’ (Bates et al., 1997; Frensch & Cameron,

2002; McLeigh & O’Neill Bridell, 2011). However, the question is whether institutional youth

care is indeed less effective when compared to non-institutional care. Anglin (2004) even

argued that for specific juveniles, institutional youth care can be more effective than non-

institutional care, as institutional youth care may be the preferred option for juveniles who need

a structured and (highly) supervised environment. Moreover, for those juveniles for whom there

is no alternative to non-institutional youth care, an important question is whether there is a way

to treat effectively within institutional care.

A number of meta-analyses examined the effectiveness of institutional youth care in

general (Garret, 1984; Knorth et al., 2008; Scherrer, 1994). These meta-analyses yielded

positive effect sizes, which indicated that institutional care can be an effective intervention.

Although all meta-analyses reached the same general conclusion that institutional youth care

can be effective, there was quite some variability in effect sizes due to differences in outcome

measures, sample characteristics (age, gender), and study design characteristics. In the study

of Garret (1984) the effect size for the outcome measure ‘recidivism’ was only small, but

positive, whereas the effect size for ‘academic improvement’ was rather large. Scherrer (1994)

did not find any significant differences between outcome measures. Knorth et al. (2008)

concluded that treatment was much more effective for externalizing and general problem

behavior than for internalizing problems. Garret (1984) showed that age of the juvenile was

inversely related to the effectiveness of treatment. The explanation given for this result is that

younger juvenile delinquents are more open to change and less rooted in their deviant behavior

(see also: Garrido & Morales, 2008). Garret (1984) demonstrated that females were more

positively affected by treatment than males, whereas Scherrer (1994) did not find a significant

difference between male or female programs, but concluded that programs which mixed males

and females together were more effective. Garret (1984) examined how different research

designs affected differences in effect sizes. Non-controlled studies with a pre- and post-test

yielded a large overall effect size, but dropped to a small effect size in controlled studies.

We considered it timely to conduct a new meta-analysis, 17 and 27 years after the

comprehensive meta-analyses that were performed by Garret (1084) and Scherrer (1994),

respectively. Knorth et al. (2008) conducted a (selective) meta-analysis of the outcomes of

institutional child and youth care, but only examined non-controlled studies that were published

between 1990 and 2005, without performing moderator analyses due to lack of (statistical)

information in the primary studies. Knorth et al. could therefore not explain the heterogeneity of

the overall mean effect sizes for internalizing, externalizing and total behaviour problems.

Knorth et al. discussed the controlled studies in a (selective) narrative review. They concluded

4

that institutional care shows the most promising short term outcomes when treatment is based

on behaviour-therapeutic methods or family involvement, whereas there is not much evidence

of positive long term outcomes.

The present meta-analysis examines the effectiveness of institutional youth care over

the past three decades, and only focuses on the quasi (experimental) studies that were

published in this period. Heterogeneity of effect sizes is examined by means of analyses of

factors (moderators) that may impact on the outcome of effectiveness studies, including sample

characteristics (age, gender, cultural background of the children), characteristics of youth care

workers (e.g., education, age, and sex), type of setting (secure versus non-secure), study design

characteristic (e.g., experimental versus quasi-experimental, time to follow up), study quality,

outcome (delinquency, behaviour problems, skills), judicial context of the intervention, type of

intervention (skills, cognitive behaviour therapy or care as usual), and type of comparison

(comparing treatment within institutional care or between institutional and non-institutional

care). This final moderator was neglected in previous meta-analyses of institutional youth care,

but seems of crucial importance for weighting the effectiveness of institutional youth care. It is

plausible to suggest that the degree to which youth care is effective depends on the particular

comparison that is made. In the present meta-analysis, institutional evidence-based treatment

(EBT), which is structured and manual-guided treatment that is based on empirical evidence

(Weisz, Jensen-Doss, & Hawley, 2006), is compared to institutional care as usual (CAU), that is,

regularly group care. We also compared institutional EBT to non-institutional EBT, institutional

CAU with non-institutional CAU, such as regular foster care, and institutional CAU with non-

institutional EBT, such as multi systemic treatment.

MethodMethodMethodMethod

Selection of studies

The present study included all published studies since 1980 that address the effectiveness of

institutional youth care. First, we searched in the following electronic data-bases: psychINFO,

MEDLINE, ERIC, Picarta, International Bibliografy of Social Sciences, Adlib, Sciencedirect,

Springerlink and Google Scholar, using the keywords in various combinations: child, youth

adolescent, inpatient, hospit* Institutional youth care, institutional care, comparison, group

care, group homes, incarcerated. Second, we inspected the reference sections of systematic

reviews and meta-analyses concerning institutional youth care in order to find more studies that

had not been included yet. This search yielded 118 studies of which 27 studies (N= 17.038

youth) eventually met the inclusion criteria of our meta-analysis.

5

Inclusion criteria

Criteria for inclusion of studies into the meta-analysis were as follows: (1) the experimental

group had to receive CAU or EBT in an institutional setting; (2) studies had to include control

groups (youth receiving CAU or regular group care in an institutional setting, or youth receiving

EBT or CAU in non-institutional settings); (3) studies had to be published in a peer reviewed

journal (4) the participants had to be enrolled in the study from 1980 on; (5) Studies had to

provide at least post test scores; (6) the average age of the youth in institutional care had to be

under age 21. We decided to exclude studies if institutional treatment was based on external

control and coercion, such as boot camps, instead of therapeutic interventions that engage

juveniles in a supportive and constructive process of change (Lipsey, 2009a). Wilson,

Mackenzie and Mitchell (2008) showed in a comprehensive meta-analysis that correctional boot

camps were not effective. The literature search resulted in 27 articles (see the Appendix A) that

met the inclusion criteria.

Coding the studies

Each study included in the meta-analysis was coded for publication characteristics, sample

characteristics, the continent where the study was conducted, study design characteristics,

study quality, judicial context of the intervention, type of setting, type of intervention, the

particular outcome measure, and type of comparison that was made. No studies could be coded

for length of stay, IQ, and characteristics of youth care workers due to lack of information.

Publication characteristics were publication year and impact factor of the journal.

Sample characteristics were gender (male versus mixed sample), mean age of the youngsters,

and the percentage of juveniles with an ethnic minority background. The continent where the

study was carried out was coded as North America or Western Europe. The study design was

coded as Randomized Controlled Trial (RCT), quasi-experimental design with a matched control

group, and quasi-experimental design with a non-matched control group (a study was

considered matched if the experimental and comparison group were comparable on at least

gender, age, and cultural background). The methodological quality of the studies was assessed

using a study quality index (QI) (Downs & Black, 1998), which evaluates the study quality (is

their sufficient information provided to enable the reader to make an unbiased judgement

concerning the results of the study); external validity; internal validity (bias and confounding)

and power, with a maximal QI score of 32. QI scores of > 20 were considered good, 11 to 20

moderate, and < 11 poor. All studies proved to be of moderate quality (M = 15,48, SD = 1,847).

We coded the nature of the particular comparison that was made in each study by

contrasting institutional EBT with non-institutional EBT, institutional CAU (i.e., regular group care)

with non-institutional CAU (i.e. foster care), institutional CAU with non-institutional EBT, and

institutional EBT with institutional CAU. In addition, we coded the type of setting (secure versus

6

non-secure) and whether or not institutional youth care was being delivered in a judicial-

penitentiary context. . . . The type of intervention was coded as skills, cognitive behaviour therapy

(CBT) and care as usual, and the outcome measure was coded as delinquency, behaviour

problems, skills and a mixed category of miscellaneous problems.

Calculation and analysis of effect sizes

Cohen’s d was calculated to index the effectiveness of institutional youth care on the basis of

differences between juveniles receiving institutional and non-institutional care and difference

between evidence-based treatment (EBT) and care as usual (CAU) in institutional care settings.

If available, follow-up data were used instead of post-test data. Follow-up time was included as

both a categorical moderator (post-test or follow-up assessment) and a continuous moderator

(number of months between end-of-treatment and follow-up assessment). To account for initial

group differences that may impact on post-test group differences, Cohens’s d was adjusted for

pre-test group differences, if such information was available, by subtracting the pre-test effect

size from post-test effect size. Subsequently, a moderator was created in order to examine

whether adjustment for initial group differences affected the combined overall effect size, and if

so, to which degree.

In most instances, Cohen’s d was calculated on the basis of mean scores and standard

deviations. In some cases, however, the calculation of Cohen’s d was based on reported test

statistics, such as F, p or t values, or on differences between percentages. The reported values

were transformed into Cohen’s d with an effect size computation program (Wilson, 2001 a, b).

After having calculated the effect sizes for each individual study, the overall combined

effect size was calculated and moderator analyses were conducted using SPSS macros (Hox,

2002; Wilson, 2001), based on the random effect model. In random effect models, significance

testing is based on the total amount of studies in the meta-analysis and results can be

generalized to the population of studies from which the current set of studies was drawn

(Rosenthal, 1995).

Homogeneity analysis was conducted at p<0.05 with SPPS-macros (Hox, 2002; Wilson,

2001) to establish whether the sample of the studies was homogeneous, that is, whether effect

sizes were constant across studies, or if the differences between effect sizes could have been

the result of another cause then subject-level sampling error. If so, differences in effect sizes

between studies may be explained by study characteristics (Lipsey & Wilson, 2001, pp. 115-

119). In case of heterogeneity, an analysis of variance (ANOVA) for the categorical variables can

be performed to examine which moderators significantly account for variability in effect sizes

among the studies included in the meta-analysis. To investigate the effects of continuous

moderator variables, regression analyses can be performed. Outlying effect sizes were identified

7

on the basis of z values larger than 3.3 or smaller than -3.3 (p < .005; Tabachnick & Fidell,

2001).

Publication bias

In this meta-analysis we only used published studies in order to have some under limit of study

quality. However, this choice may increase the risk that studies with non-significant findings,

which are less likely to be published, are not included in the meta-analysis. This phenomenon

has been designated as the ‘file drawer problem’ (Rosenthal, 1995). A way to find out if the

meta-analytic results may be affected by publication bias is to examine a funnel plot of the

distribution of effect sizes. Each individual study’s effect size is plotted on the horizontal axis

against its sample size, standard error or precision (the reciprocal of the standard error) on the

vertical axis. The distribution of effect sizes should be shaped as a funnel if no publication bias

is present, since the more numerous studies with small sample sizes are expected to show a

larger variation in the magnitude of effect sizes than the less numerous studies with large effect

sizes. A violation of funnel plot symmetry reflects publication bias, that is, a selective inclusion

of studies showing positive or negative outcomes (Sutton, Duval, Tweedie, Abrams, & Jones,

2000). Funnel plot asymmetry can be tested by regressing the standard normal deviate, defined

as the effect size divided by its standard error, against the estimate’s precision (the inverse of

the standard error), which largely depends on sample size (see Egger, Smith, Schneider, &

Minder, 1997). If there is asymmetry, the regression line does not run through the origin and the

intercept significantly deviates from zero.

ResultsResultsResultsResults

The Appendix A provides an overview of all 27 studies (N= 17,038 youth) included in this meta-

analysis, with the effect size of each individual study and the respective study characteristics. In

line with generally accepted principles for the interpretation of effect sizes, an effect size of d =

.20 was considered small, an effect size of d = .50 was considered medium, and an effect size

of .80 was considered large (Cohen, 1988).

The individual study effect sizes ranged from d = -.690 to d = 1.806. Possible

publication bias was examined by testing funnel plot asymmetry. The standard normal deviate

was regressed against the estimate’s precision. As the intercept did not significantly deviate

from zero (t = 1.405, p = .172), there was no indication of funnel plot asymmetry and therefore

no indication of publication bias.

Because the set of effect sizes proved to be heterogeneous, Q (26) = 1355.838, p <

0.001, a moderator analysis was conducted in order to explain the variation in effect sizes

among studies (Mullen, 1989). As can be derived from Table 1, a series of ANOVA’s yielded two

significant moderators: comparison – Q (3) = 10.7234, p < .05 – and type of intervention – Q

8

(2) = 17.211 p < .0001. A significant and positive small-to-medium effect size of d = 0.337 was

found for the comparison between evidence-based treatment and care as usual (general group

care) in institutional settings, whereas other comparisons yielded negative but non-significant

effect sizes. This was also true when the comparison moderator was dichotomized into a set of

studies contrasting institutional EBT with institutional CAU or general group care (d = 0.337, p <

.001) and a set of studies contrasting residential care with non-residential care (d = -0.199, ns.):

Q (1, 25) = 10.530, p < .01. Furthermore, only CBT yielded a significant effect size of d = 0.520,

whereas skills training and care as usual had no significant effect. In addition, we analyzed the

continuous moderators (see Table 2), which only yielded a significant effect size for publication

year (Beta = -.453, t = -2,723, p < 0.01).

Table 1. Categorical Moderator Analyses (ANOVA’S): Cohen’s d, 95% CI, Q Statistics

*p < .05. ** p < .01. *** p < .001

Moderator variables

N

K

d

95% confidence interval

Q statistic between studies

Q statistic within studies

Overall 17038 27 0.129 -0.02 to 0.259 1355.838*** Gender 2.554 Male 4334 10 0.329* 0.033 to 0.626 14.261 Mixed 12704 17 0.026 -0.199 to 0.251 14.048 Continent 1.617 North America 11985 21 0.200 -0.007 to 0.408 27.755 Western Europe 5053 6 -0.084 -0.470 to 0.302 0.425 Criminal justice intervention 2.116 Yes 4757 13 0.271* 0.011 to 0.533 18.903 No 11620 12 -0.006 -0.276 to 0.264 8.881 Mixed 661 2 0.124 -0.534 to 0.781 0.594 Incarcerated 3.288 Yes 4198 12 0.321 0.055 to 0.588 16.000 No 12840 15 -0.008 -0.244 to 0.228 12.453 Comparison 10.723* ebt-inst vs. ebt-noninst 65 1 -0.338 -1.164 to 0.488 0.000 cau-inst vs. ebt-noninst 1493 6 -0.196 -0.525 to 0.133 1.921 cau-inst vs. cau-noninst 10060 3 -0.161 -0.620 to 0.298 0.350 ebt-inst vs. cau-inst 5420 17 0.337*** 0.138 to 0.537 26.736* Design 0.147 Randomized 1469 9 0.099 -0.224 to 0.422 6.078 Matched 14368 9 0.186 -0.136 to 0.509 13.475 Non-matched 1201 9 0.126 -0.202 to 0.455 8.622 Adjusted for pre-test differences 0.011 no 13543 13 0.148 -0.123 to 0.418 18.451 yes 3495 14 0.128 -0.133 to 0.389 9.757 Follow-up 0.054 yes 16411 20 0.150 -0.067 to 0.368 25.240 no 627 7 0.100 -0.272 to 0.471 2.956 Outcome 0.130 Delinquency 12555 9 0.100 -0.226 to 0.419 16.984* Behavior 2829 6 0.177 -0.220 to 0.573 6.057 Skills 1003 4 0.180 -0.303 to 0.663 1.100 Mixed 651 8 0.133 -0.214 to 0.480 3.225 Type of intervention 17.211*** Skills 1158 8 0.017 -0.252 to 0.286 6.960 CBT 4327 10 0.520*** 0.281 to 0.759 19.382* CAU 11553 9 -0.184 -0.430 to 0.061 0.783

9

Table 2. Continuous moderator analyses: Standardized Regression Coefficients (Beta), t-Values, and Significances (p)

Moderator variables

N

K

Beta

t

p Time Follow Up 16411 20 -.036 0.167 .868 Publication year 17038 27 -.453 -2.723 .007 Impact factor journal 17038 27 .033 -0.176 .860 Study quality 17038 27 -.106 -0.564 .573 Ethnic minority 13795 23 -.148 -0.729 .466 Age juveniles 17008 26 .246 1.321 .187

All significant moderators were included in a multiple regression analysis in order to examine

their unique contribution to the overall combined effect size. Comparison and type of

intervention were transformed into a set of dummy variables in order to be able to include these

moderators as dichotomous variables in the multiple regression analysis. We found a significant

regression equation – Q (5, 21) = 36.34, p < .001 – with two significant moderators, namely

CBT versus skills and care as usual (b = .426, t = 3.253, p < .01) and the comparison that

contrasts EBT with CAU in institutional settings versus all other possible comparisons (b = .269,

t = 2.058, p < .05).

The results of the multiple regression analysis indicate that the effect of year of

publication on the overall combined effect size is an artifact explained by an association

between the comparison that is made and treatment type on the one hand and year the

publication on the other hand. The multiple regression results also indicate that applying

evidence-based treatment in institutional settings, in particular cognitive behavior therapy,

uniquely accounts for the effectiveness of institutional youth care.

DiscussionDiscussionDiscussionDiscussion

The first purpose of this study was to replicate previous meta-analyses of effectiveness of

institutional youth care and to test the hypotheses about the effectiveness of residential care,

but only including studies with a (quasi)experimental design in contrast to some of these earlier

reviews. A total of 27 published studies (N = 17,038 youth) were included in this meta-analysis.

As the overall combined effect size proved to be heterogeneous, we conducted moderator

analyses that yielded two significant moderators uniquely explaining the effectiveness of

institutional youth care: type of comparison and type of intervention.

We compared institutional Evidence-Based Treatment (EBT) with non-institutional EBT,

institutional Care As Usual (CAU) with non-institutional CAU, institutional CAU with non-

institutional EBT, and institutional EBT with institutional CAU. Only the last comparison,

institutional EBT versus institutional CAU, yielded a significant and small-to-medium effect (d =

.34), which not only shows that institutional care can be equally effective as non-institutional

care, but also demonstrates that it is more promising to provide youth with EBT during their stay

in the institution.

10

This conclusion supports some of the findings of previous meta-analyses, showing that

youngsters react positively to treatment in institutional care and in general improve in their

psychological wellbeing (Garret, 1984; Knorth et al., 2008; Scherrer, 1994). Our meta-analysis

also supports the conclusion of the meta-analysis of evidence-based youth psychotherapies

versus usual clinical care by Weisz et al. (2006) in which EBT proved to be more effective than

CAU (d = 0.30), independent of treatment modality. The results of the narrative review by Knorth

et al. (2008), suggest that non-institutional evidence-based treatment might perform better than

institutional group care (CAU). The present meta-analysis included six studies comparing non-

institutional EBT with institutional group care, yielding a small and negative but non-significant

effect size for institutional group care (CAU) when compared to non-institutional EBT. However,

more research is needed to examine if this effect-size is still non-significant over a larger

number of studies.

Moderator analyses indicated that only cognitive behavior therapy (CBT) had a medium

effect (d = .50), whereas skills training and care as usual had no effect. In other meta-analyses

and narrative reviews, CBT also proved to be an effective intervention (Armelius &Andreassen,

2006; Garrido & Morales, 2007; Weisz et al., 2006), even though Armelius and Andreassen

found a rather small effect for CBT in comparison with CAU and no significant difference

between CBT and alternative evidence-based treatment.

It might come as a surprise that social skills training did not have a significant effect.

However, there is an abundant body of research showing that the effect of social skills training

for juveniles, in general, tends to be small. Meta-analyses by Ang and Hughes (2002), Cook et

al. (2008), Durlak, Weissberg, and Pachan (2010), Losel and Beelmann (2003), Schneider

(1992) and Quinn, Kavale, Mathur, Rutherford and Forness (1999) report small to medium

effects. Moreover, these effects generally do not persist long after the intervention has ended.

As social skills training usually does not take place in a naturalistic setting, the effects of social

skills training have limited generalizability (Maag, 2005, 2006). If social skills training is

delivered in institutional settings, it may be the case that the social skills cannot be generalized

to other settings outside the institution.

Age, gender, study design characteristics (randomized, matched versus non-matched

controls) and type of outcome measure (delinquency, skills, problem behavior, and overall

problems) did not influence the effectiveness of institutional youth care in our meta-analysis.

The conclusion of Knorth et al. (2008) that institutional care would be especially effective for

teaching specific skills could therefore not be confirmed. More recent studies showed smaller

effect sizes. This effect, however, disappeared after controlling for type of treatment and the

EBT-CAU comparisons in the multiple regression analysis. Earlier studies more often inspected

the effects of CBT than recent studies did.

11

Unfortunately, we were not able to test a number of relevant moderators, such as IQ,

length of stay, and characteristics of youth care workers, because there was not sufficient

information available to code these moderators. It is particularly remarkable that even recent

studies do not report on intelligence of the sample under study or examine intelligence as a

moderator of treatment effects, as there is a vast body of research showing that treatment

should be responsive to the offender’s cognitive abilities and learning style (Andrews & Bonta,

2010; Koning & Collin, 2007; Van Nieuwenhuijzen, Orobio de Castro, Wijnroks, & Vermeer,

2009). Although length of stay seems important to consider from the perspective of possible

institutionalization effects, which dates back to the seminal work of Goffman (1961), it is

difficult to examine the unique effect of length of stay on treatment outcomes in residential

youth care, as it might be confounded with a host of client and treatment characteristics that

cannot be accounted for in a single study. It is therefore imperative that studies describe length

of stay of the sample under study in order that future meta-analyses of institutional youth care’s

effectiveness can adequately examine the impact of length of stay.

The studies used for this meta-analysis do not report on characteristics of youth care

workers. This is remarkable, because it is assumed that characteristics of group care workers,

such as professionalism, education and training and relationship skills constitute an important

moderator of positive treatment outcomes (e.g. Bickman et al., 2004; Duncan, Miller, Wampold,

& Hubble, 2009; Van der Helm, Boekee, Stams, & van der Laan, 2011).

There are some limitations of this meta-analysis that need to be mentioned. To preserve

some minimum level of study quality we only included published studies, which might increase

the risk of publication bias. Analyses of publication bias, however, by means of a funnel-plot of

the distribution of effect sizes across studies, proved that there was no indication of publication

bias. A second limitation was that, although we decided to use published studies only, we could

not prevent that all studies included in this meta-analysis were of moderate quality. On the other

hand, there were no published studies that should be considered as low quality studies. A third

limitation is that the overall effect size for CBT was based on a heterogeneous set of effect

sizes. But homogeneity might not be a realistic expectation, because of the diverse populations

that are found within institutional youth care.

In spite of these limitations, this meta-analysis provides us with new and better insights

in the effectiveness of institutional youth care than any of the previous meta-analyses. First, we

demonstrated that evidence-based treatment delivered in institutional youth care does not have

to be less effective than evidence-based treatment in non-institutional youth care. Notably, we

replicated the findings from the meta-analysis of non-institutional youth care by Weisz et al.

(2006), showing that evidence-based treatment performs better than regular group care (i.e.

care as usual) provided in institutional settings, which has often been designated as treatment

of ‘the last resorts’. Nevertheless, the present meta-analysis also showed that providing regular

12

group care to youngsters who tend to have serious behavioral and developmental problems

does not seem to have a positive effect and might even increase problems. Therefore, it

remains important to always first consider non-institutional natural interventions, including

foster care, as an alternative to regular group care, in particular in the case of young children.

However, if non-institutional care is not a realistic option – for example, when children need a

structured and (highly) supervised environment – it seems imperative to offer these juveniles

evidence-based treatment. In the end, the question is not whether institutional or non-

institutional care should be preferred, but how to optimize the effects of EBT for children who

receive institutional and non-institutional youth care.

The availability of aftercare can be considered important for the generalization of

treatment effects after institutional care. A recent meta-analysis showed that aftercare can have

a small and positive short-term effect (James, Stams, Ascher, De Roo, & Van der Laan, 2011).

Aftercare, however, is currently not common in institutional youth care, but it is plausible to

suggest that aftercare that builds on positive outcomes of institutional care may considerably

increase the short and long term effects of evidence-based treatment delivered in institutional

youth care.

13

LiteratureLiteratureLiteratureLiterature

* Abrams, L.S., Shannon, S.K., & Sangalang, C. (2007). Transition services for incarcerated

youth: a mixed methods evaluation study. Child and Youth Services Review, 30, 746-

757.

Andrews, D.A., & Bonta, J. (2010). The Psychology of Criminal Conduct (Fifth Edition). New

Providence, NJ: Matthew Bender & Company Inc.

Ang, P. & Hughes, J. N. (2001). Differential benefits of skills training with antisocial youth based

on group composition: a meta-analytic investigation. School Psychology Review, 31,

164-185.

Anglin, J. (2004). Creating "well functioning" residential care and defining its place in a system of

care. Child and Youth Care Forum, 33, 175-192.

Armelius, B-Å & Andreassen, T.H. (2007). Cognitive-behavioral treatment for antisocial behavior

in youth in residential treatment. Cochrane Database of Systematic Reviews 2007,

Issue 4, Art.No. CD005650, DOI: 10.1002/14651858. CD005650.pub2.

* Armstrong, T.A. (2003). The effect of moral reconation therapy on the recidivism of youthful

offenses. Criminal Justice and Behavior, 30, 668–687.

Barth, R.P. (2002). Institutions vs. foster homes: the empirical base for the second century of

action. Chapel Hill, N.C.: School of Social Work, Jordan Institutes for Families.

* Barth, R.P., Greeson, J.K., Green, R.L., Hurley, S., & Sisson, J. (2007). Outcomes for youth

receiving intensive in-home therapy or residential care: a comparison using propensity

scores. American Journal of Orthopsychiatry,77, 497-505.

Bates, B.C., English, D.J., & Kouidou-Giles, S. (1997). Institutional treatment and its alternatives:

a review of the literature. Child & Youth Care Forum, 26, 7-51.

* Berger, L.M., Bruch, S.K., Johnson, E.I., James, S., & Rubin, D. (2009). Estimating the “impact”

of out-of-home placement on child well-being: approaching the problem of selection

bias. Child Development, 80, 1856-1876.

Bickman, L., Vides de Andrade, A.R., Lambert, E.W., Doucette, A., Sapyta, J., Boyd, A.S.,

Rumberger, D.T., Moore-Kurnot, J., McDonough, L.C., & Rautkis, M.B. (2004). Youth

Therapeutic Alliance in Intensive Treatment Settings. The Journal of Behavioral Health

Services & Research, 31, 134-148.

* Brugman, D., Bink, M.D., Nas, C.N., & Van den Bos, J.K. (2007). Kunnen delinquent jongeren

elkaar helpen in hun sociale ontwikkeling? Effecten peer-hulpprogramma EQUIP op

denkfouten en recidive. Tijdschrift voor Criminologie, 47, 153-169.

* Caldwell, M.F., & Van Rybroek, G.J. (2001). Efficacy of a decompression treatment model in

the clinical management of violent juvenile offenders. International Journal of Offender

Therapy and Comparative Criminology, 45, 469-477.

14

* Caldwell, M.F., & Van Rybroek, G.J. (2005). Reducing violence in serious juvenile offenders

using intensive treatment. International Journal of Law and Psychiatry, 28, 622-636.

* Cann, J., Falshaw, L., & Friendship C. (2003). Understanding ‘What Works’: Accredited

Cognitive Skills Programmes for adult men and young Offenders. Youth Justice, 5, 165-

179.

* Chamberlain, P., & Reid, J. B. (1998). Comparison of two community alternatives to

incarceration for chronic juvenile offenders. Journal of Consulting and Clinical

Psychology, 66, 624−633.

Cook, C.R., Gresham, F.M., Kern, L., Barreras, R.B., Thornton, S & Crews, S.D., (2008). Social

skills training for secondary students with emotional and/or behavioral disorders: A

review and analysis of the meta-analytic literature. Journal of Emotional and Behavioral

Disorders, 16, 131-144.

Dishion, T.J., McCord, J., & Poulin, F. (1999). When interventions harm. American Psychologist,

54, 755-764.

Downs S. H., & Black, N. (1998). The feasibility of creating a checklist for the assessment of the

methodological quality both of randomised and non-randomised studies of health care

interventions. Journal of Epidemiological and Community Health, 52, 377-384.

Duncan, B.L., Miller, S.D., Wampold, B.E., & Hubble, M.A. (2009). The heart and soul of Change.

Delivering What Works in Therapy. Washington D.C.: American Psychological Association.

Durlak, J.A., Weissberg, R.P, & Pachan, M. (2010). A meta-analysis of after-school programs that

seek to promote personal and social skills in children and adolescents. American

Journal of Community Psychology, 45, 294-309.

Egger, M., Smith, G.D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a

simple graphical test. British Medical Journal, 315, 629-634.

French, K.M. & Cameron, G. (2002). Treatment of choice or a last resort? A review of

institutional mental health placements for children and youth. Child and Youth Care

Forum, 32, 307-339.

* Frensch, K., Cameron, G., & Preyde, M. (2009). Community adaptation of youth accessing

residential programs or a home-based alternative: School attendance and academic

functioning. Child & Youth Care Forum. 38, 287-303.

Garret, C.J. (1984). Meta-analysis of the effects of institutional and community residential

treatment on adjudicated delinquents. University of Colorado at Boulder: Unpublished

dissertation.

Garrido, V., & Morales, L.A. (2007). Serious (violent and chronic) juvenile offenders: a

Systematic Review of treatment effectiveness in secure corrections. In: The Campbell

Collaboration Reviews of Intervention and Policy Evaluations (C2-RIPE), July 2007.

Philadelphia, Pennsylvania: Campbell Collaboration.

15

Goffman, E. (1961). Asylums: Essays on the social situation of mental patients and other

inmates. New York: Doubleday.

* Gordon, J.A., Moriartry, L.J., & Grant, P.H. (2000). The impact of a juvenile residential

treatment center on minority offenders. Journal of Contemporary Criminal Justice, 16,

194-208.

* Guerra, N.G., & Slaby, R.G. (1990). Cognitive mediators of aggression in adolescent offenders:

2. Intervention. Developmental Psychology, 26, 269−277.

Hair, H. J. (2005). Outcomes for children and adolescents after residential treatment: A review

of research from 1993-2003. Journal of Child and Family Studies, 14-4, 551-575.

* Hawkins, J.D., Jenson, J.M., Catalano, R.F., & Wells, E.A. (1991). Effects of skills training

intervention with juvenile delinquents. Research on Social Work Practice, 1, 107-121.

* Hengeler, S.W., Rowland, M.D., Randall, J., Ward, D.M., Pickrel, S.G., Cunningham, P.B., Miller,

S.L., Edwards, J., Zealberg, J.J., Hand, L.D., & Santos, A.B. (1999). Home-based

multisystemic therapy as an alternative to the hospitalization of youths in psychiatric

crisis: clinical outcomes. Journal of the American Academy of Child & Adolescent

Psychiatry, 38, 1331-1339.

* Henggeler, S.W., Rowland, M. D., Halliday-Boykins, C., Sheidow, A. J., Ward, D. M., & Randall,

J. (2003). One-year follow-up of multisystemic therapy as an alternative to the

hospitalization of youths in psychiatric crisis. Journal of the American Academy of Child

and Adolescent Psychiatry, 42, 543−551.

Hoagwood, H., Burns, B.J., Kiser, L., Ringeisen, H., & Schoenwald, S.K. (2001). Evidence-based

praktijk in kinderen en adolescenten diensten voor geestelijke gezondheidszorg.

Psychiatric Services, 52, 1179-1189.

Hox, J. (2002). Multilevel Analysis. Techniques and Applications. London: Lawrence Erlbaum

Associates, Publishers.

James, C., Stams, G.J.J.M., Ascher, J.J., De Roo, A.R., & Van der Laan, P.H. (2011)

Aftercare programs for reducing recidivism among juvenile and young adult

offenders: A Meta Analytic Review. Manuscript submitted for publication.

* Katz, L.Y., Cox, B.J., Gunasekara, S., & Miller, A.L. (2004). Feasibility of dialectical

behavior therapy for suicidal adolescent inpatients. Journal of the American

Academy of Child and Adolescent Psychiatry, 43, 276−282.

* Kazdin, A.E. (1987). Treatment of antisocial behavior in children: current status and future

directions. Psychological Bulletin, 10, 187-203.

Koning, N.D. de, & Collin, P.J.L. (2007). The treatment of children and adolescents with a

psychiatric disorder and mental retardation. Kind en Adolescent, 28, 215-229.

16

Knorth, E.J., Harder, A.T., Zandberg, T., & Kendrick, A.J. (2008). Under one roof: A review and

selective meta-analysis on the outcomes of institutional child and youth care. Children

and Youth Services Review, 30, 123-140.

* Kolko, D. J., Loar, L. L., & Sturnick, D. (1990). Inpatient social–cognitive skills training groups

with conduct disordered and attention deficit disordered children. Journal of Child

Psychology and Psychiatry, 31, 737−748.

* Leve, L.D., & Chamberlain, P. (2005). Association with delinquent peers: Intervention effects

for youth in the juvenile justice system. Journal of Abnormal Child Psychology, 33,

339−347.

Losel, F. & Beelmann, A. (2003). Effects of child skills training on preventing antisocial behavior:

A systematic review of randomized evaluations. Annals of the American Academy of

Political and Social Science, 587, 84-109.

Lyons, J.S., & McCulloch, J.R. (2006). Monitoring and managing outcomes in institutional

treatment: Practice-based evidence in search of evidence-based practice. Journal of the

American Academy of Child and Adolescent Psychiatry, 45, 247-251.

Lyons, J.S., Woltman, H., Martinovich, Z., & Hancock, B. (2009). An outcomes perspective of

residential treatment in the system of care. Residential Treatment For Children & Youth,

26, 71-91.

Lipsey, M.W. (2009). The primary factors that characterize effective interventions with juvenile

offenders: a meta-analytic overview. Victims and Offenders, 4,124-147.

Lipsey, M.S. & Wilson, D.B. (2001). Practical Meta-Analysis. Applied Social Research Methods

Series (vol. 49). Thousand Oaks, CA Sage publications.

Maag, J.W. (2005). Social skills training for youth with emotional and behavioral disorders and

learning disabilities: Problems, conclusions, and suggestions. Exceptionality, 13, 155-

172.

Maag, J.W. (2006). Social skills training for students with emotional and behavioral disorders: A

review of reviews. Behavioral Disorders, 32, 4-17.

* Mattejat, F., Hirt, B.R., Wilken, J., Schmidt, M. H., & Remschmidt, H. (2001). Efficacy of

inpatient and home treatment in psychiatrically disturbed children and adolescents.

European Child and Adolescent Psychiatry, 10, 71−79.

* McCrae, J.S., Bethany, R.L., Barth, R.P., & Rauktis, M.E. (2010). Comparing three of well-being

outcomes for youth in group and nonkinship foster care. Child Welfare, 89, 229-249.

McLeigh, J.D., & O’Neill Briddell, L. (2011). Where Have All the Children Gone? The Effects of the

Justice System on America’s Children and Youth. The Community, 1, 334-341.

* Moody, E. E. (1997). Lessons from pair counseling with incarcerated juvenile delinquents.

Journal of Addictions and Offender Counseling, 18, 10−25.

Mullen, B. (1989). Advanced basic meta-analysis. New Jersey: Laurence Erlbaum Associates.

17

Quinn, M.M., Kavale, K.A., Mathur, S.R., Rutherford, R.B. & Forness, S.R. (1999). A meta-

analysis of social skills interventions for students with emotional or behavioral disorders.

Journal of Emotional or Behavioral disorders, 7, 54-64.

* Rohde, P., Jorgerson, J. S., Seeley, J. R., & Mace, D. E. (2004). Pilot evaluation of the coping

course: A cognitive behavioral intervention to enhance coping skills in incarcerated

youth. Journal of the American Academy of Child and Adolescent Psychiatry, 43,

669−676.

Rosenthal, R. (1995). Writing meta-analytic Reviews. Psychological Bulletin, 118, 183-192.

* Ryan, J.P., Marshall, J.M., Herz, D., & Hernandez, P.M. (2008). Juvenile delinquency in child

welfare: investigating group home effects. Children and Youth Services Review,

30, 1088-1099.

Scherrer, J.L. (1994). A Meta-analysis of the effectiveness of residential treatment programs for

children and adolescents. University of Illinois at Chicago: unpublished dissertation.

Schneider, B.H. (1992). Didactic Methods for enhancing children’s peer relations: a quantative

review, Clinical Psychology Review, 12, 363-382.

* Scholte, E. M., & Van der Ploeg, J. D. (2003). Effectiveness of residential treatment methods

for youngsters with severe behavioral problems: Findings from a one year follow-up

study. International Journal of Child and Family Welfare, 6, 85−197.

Stams, G.J., Brugman, D., Deković, M., Van Rosmalen, L., Van der Laan, P.H., & Gibbs, J.C.

(2006). The moral judgment of delinquents: a meta-analysis. Journal of Abnormal Child

Psychology, 34, 692-708.

Sutton, A.J., Duval, S.J., Tweedie, R.L., Abrams, K.R., & Jones, D.R. (2000). Empirical

assessment of effect of publication bias on meta-analyses, British Medical Journal,320,

1574-1577.

Tabachnick, B.G. & Fidell, L.S. (2001). Using multivariate statistics. New York: Harper & Row.

* Thompson, R.W., Smith, G. L., Osgood, D.W., Dowd, T.P., Friman, P. C., & Daly, D.L. (1996).

Residential care: A study of short- and long-term educational effects. Children and Youth

Services Review, 18, 221−242.

Van der Helm, G.H.P., Boekee, I., Stams, G.J.J.M., Van der Laan, P.H. (in press). Fear is the key:

keeping the balance between flexibility and control in a Dutch youth prison. Journal of

Children’s Services.

Van der Helm, G.H.P., Klapwijk, M., Stams, G.J.J.M., & Van der Laan, P.H. (2009). ‘What Works’

for juvenile prisoners: The role of group climate in a youth prison. Journal of Children’s

Services, 4, 36-48.

Van Nieuwenhuijzen, M., Orobio de Castro, B., Wijnroks, L., & Vermeer, A. (2009). Social

problem-solving and mild intellectual disabilities: Relations with externalizing behavior

18

and therapeutic context. American Association on Intellectual and Developmental

Disabilities, 114(1), 42-51.

Weisz, J.R., Jensen-Doss, A., & Hawley, K.M. (2006). Evidence-based youth psychotherapies

versus usual clinical care. A meta-analysis of direct comparisons. American Psychologist,

61, 671-689.

* Wilmshurst, L.A.(2002). Treatment Programs for Youth with Emotional and Behavioral

Disorders: An Outcome Study of Two Alternate Approaches. Mental Health Services

Research, 4, 85-96.

Wilson, D.B. (2001). Effect size determination program.

Wilson, D.B., MacKenzie, D.L., & Mitchell, F.N. (2005). Effects of correctional boot camps on

offending. New York: Campbell Collaboration.

19

Appendix Appendix Appendix Appendix AAAA Study characteristics

Study Publication year

Impact factor journal

Study quality rating

Worldpart of origin

Comparison Design Intervention residential

N Sex Avarage Age

Ethnic minority

Crime justice intervention

Open or incarcerated

d

Abrams et al. 2008 1,17 14 North America

TreatRes – Groupcare

non matched

skills 61 boys 15,90 73,77% yes incarcerated -0.563

Armstrong 2003 1,88 16 North America


RCT skills 212 boys 20,21 68,87% yes incarcerated 0.005

Barth et al. 2007 2,21 14 North America

GroupCare – TreatNotres

matched control

TAU 786 mixed 11,42 39,35% no open -0.146

Berger et al. 2009 3,63 16 Western Europe

GroupCare - NoTreatNoRes

matched control

TAU 1710 mixed 9,78 52,00% no open -0.054

Brugman et al.

2007 0,00 15 Western Europe


non matched

skills 77 boys 16,40 unknown yes incarcerated -0.083

Caldwell & Rybroek

2005 1,10 14 North America


matched control

moderate 248 boys 16,00 62,00% yes incarcerated 0.537

Caldwell & Rybroek



matched control

CBT 30 boys unknown unknown yes incarcerated 1.806

Cann et al. 2005 0,00 16 Western Europe


matched control

CBT 3068 boys 17,60 unknown yes incarcerated 0.113

French, Cameron & Preyde



non matched

TAU 210 mixed 11,55 1,00% no open 0.027

Gordon 2000 1,05 19 North America


RCT CBT 480 boys 16,62 42,71% yes open 0.260

Guerra & Slaby



matched control

CBT 120 mixed 17,20 60,00% yes incarcerated 0.345

Hawkins et al.



RCT CBT 150 mixed 15,75 49,65% yes incarcerated 0.610

Henggeler et al.



RCT TAU 116 mixed 13,00 66,00% no open -0176

Henggeler et al.



RCT TAU 160 mixed 12,90 67,00% no open -0.149

Katz et al. 2004 4,98 16 North America


non matched

CBT 62 mixed 15,40 27,40% no open 0,410

Kazdin 1987 4,46 15 North America


CBT 37 mixed 10,90 23,20% no open 1.246

Kolko et al. 1990 4,98 16 Western Europe


matched control

CBT 56 mixed 10,40 50,32% no open -0,317

Leeman et al.



RCT skills 54 boys 16,00 33,33% yes incarcerated 0.629

Leve & Chamberlain



RCT TAU 153 mixed 14,88 21,57% yes open -0.690

Mattejat et al.



RCT TAU 68 mixed 11,90 unknown no open -0.039

20

Mc Crae 2010 0,67 18 North America


matched control

TAU 124 mixed 12,20 67,70% no open -0.069

Moody 1997 0,00 11 North America


non matched

skills 28 boys 14,20 89,29% yes incarcerated 0.266

Rohde 2004 4,98 14 North America


RCT CBT 76 boys 16,30 41,70% yes incarcerated 0.469

Ryan et al. 2008 1,17 18 North America


matched control

TAU 8226 mixed 8,50 83,00% no open -0.354

Scholte & Van der Ploeg



non matched

skills 74 mixed 14,90 35,00% mixed incarcerated -0.142

Thompson 1996 1,12 15 North America


unknown skills 587 mixed 14,55 29,60% mixed open 0.375

Wilmshurst 2002 0,00 17 North America

TreatRes - TreatNotRes

non matched

skills 65 mixed 11,12 5,00% no open -0.338

HOOFDSTUK 4. THE EFFECTIVENESS OF INSTITUTIONAL … 4.pdf1 HOOFDSTUK 4. HOOFDSTUK 4. THE...

Documents

Transcript of HOOFDSTUK 4. THE EFFECTIVENESS OF INSTITUTIONAL … 4.pdf1 HOOFDSTUK 4. HOOFDSTUK 4. THE...