Screening for Posttraumatic Stress Symptoms in Young Refugees

[Full title: Screening for Posttraumatic Stress Symptoms in Young Refugees: Comparison of Questionnaire Data with and without Involvement of an Interpreter]

International Journal of Environmental Research and Public Health Brief Report Screening for Posttraumatic Stress Symptoms in Young Refugees: Comparison of Questionnaire Data with and without Involvement of an Interpreter Lauritz Rudolf Floribert Müller * ,† , Johanna Unterhitzenberger † , Svenja Wintersohl, Rita Rosner and Julia König Citation: Müller, L.R.F.; Unterhitzenberger, J.; Wintersohl, S.; Rosner, R.; König, J. Screening for Posttraumatic Stress Symptoms in Young Refugees: Comparison of Questionnaire Data with and without Involvement of an Interpreter Int. J Environ. Res. Public Health 2021 , 18 , 6803. https://doi.org/10.3390/ ijerph 18136803 Academic Editor: Paul B. Tchounwou Received: 30 April 2021 Accepted: 23 June 2021 Published: 24 June 2021 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations Copyright: © 2021 by the authors Licensee MDPI, Basel, Switzerland This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/) Department of Psychology, Catholic University Eichstätt-Ingolstadt, Ostenstraße 25, 85072 Eichstätt, Germany; johanna.unterhitzenberger@ku.de (J.U.); svenja.wintersohl@gmx.de (S.W.); rita.rosner@ku.de (R.R.); julia.koenig@ku.de (J.K.) * Correspondence: lauritz.mueller@ku.de; Tel.: +49-842-193-21265 † Joint first authors Abstract: Background : The substantial number of young refugees who have arrived in Europe since 2015 requires rapid screening to identify those in need of treatment. However, translated versions of screening measures are not always available, necessitating the support of interpreters. The Child and Adolescent Trauma Screen (CATS) is a validated questionnaire for posttraumatic stress symptoms Here, we report on the psychometric properties of the CATS in a sample of young refugees as a function of interpreter involvement Methods: A total of N = 145 ( M age = 16.8, SD = 1.54; 93% male) were assessed with the CATS, with half of the screenings conducted with and half without interpreters. Post hoc analyses included calculating internal consistency using Cronbach’s α . We used confirmative factor analysis to investigate the factor structure Results: The CATS total scale showed good reliability ( α = 0.84). Differences in psychometric properties between the interpreter vs. the no interpreter group were minor and tended to be in favor of the interpreter group. Results of a confirmatory factor analysis were acceptable after the exclusion of items with low item-scale correlations Conclusions: The sample and the administration of the assessment represent the situation of young refugees in Germany, where resources are low and translated versions not always available. The CATS may be a helpful screening tool for clinicians working with young refugees, even when administered with an interpreter. Limitations include the post hoc design of the analysis without randomization of participants and the lack of a third comparison group using translated questionnaire versions Keywords: refugee; adolescent; assessment; screening; PTSD; interpreter 1. Introduction Asylum-seeking youth, and especially unaccompanied asylum-seeking youth, have a high risk for posttraumatic stress disorder (PTSD; [ 1 , 2 ]). A brief and valid screening should constitute the first step of recently proposed stepped-care models to meet the psychological needs of young refugees, which are currently being tested in this group [ 3 ]. To this end, measures must be validated for culturally diverse samples. The Child and Adolescent Trauma Screen (CATS; [ 4 ]) is a short self-report questionnaire with items corresponding to PTSD criteria in the Diagnostic and Statistical Manual for Mental Disorders, 5 th edition (DSM-5; [ 5 ]). It has been validated in an international (Western) sample with good reliability and validity and has since been used repeatedly with refugee youth [ 1 , 6 , 7 ]. However, self-report screening measures, including the CATS, are not available for all languages of interest in the refugee context, and, moreover, many asylum-seeking youth are illiterate and cannot use translated versions even when they exist [ 8 ]. Accordingly, in reallife care settings, screenings are often conducted with non-native language versions even Int. J. Environ. Res. Public Health 2021 , 18 , 6803. https://doi.org/10.3390/ijerph 18136803 https://www.mdpi.com/journal/ijerph

Int. J. Environ. Res. Public Health 2021 , 18 , 6803 2 of 7 when the respondents’ language proficiency is low or must be supported by interpreters In this context, untrained lay interpreters, who might be unfamiliar with mental health services, are often employed [ 9 ]. In principle, it is conceivable that the involvement of (lay) interpreters could change both screening results and therapeutic processes for a variety of reasons, including omission of cultural taboos in the translation process, role difficulties that might lead interpreters to take a directive rather than a neutral role, and lack of professional understanding regarding mental health services [ 10 ]. In terms of therapy effectiveness, the few existing post hoc findings on interpreter involvement present a mixed picture [ 11 , 12 ], whereby the latest study found that the group with interpreters involved showed poorer results than the comparison group without interpreters [ 12 ]. However, there are very few quantitative findings on the effects of employing an interpreter in mental health settings in general and, to our knowledge, no studies on the effects on screening or diagnostic results in particular Accordingly, this post hoc study aims (1) to investigate the internal consistency of the CATS in a sample of young refugees resettled to Germany using German language versions with vs. without the involvement of an interpreter; (2) to compare symptom severity, item values and item-scale correlations between the two groups; and (3) to examine the factor structure of the CATS in the total sample 2. Materials and Methods 2.1. Procedure and Participants Data from two studies were used with a total of N = 145. Study 1 assessed the mental health of n = 98 minor refugees resettled in southern Germany in an interview-like setting [ 1 ]. Study 2 was a pilot study [ 6 ] examining trauma-focused cognitive behavioral therapy [ 13 ] with n = 47 unaccompanied refugee minors. Overall, participants were, on average, M = 16.73 years of age ( SD = 1.54). Most participants were unaccompanied ( n = 115, 78.8%) and boys ( n = 135, 93.1%), reflecting the population structure in Germany, where most unaccompanied refugee minors are male (for further sociodemographic information, see Supplementary Material Table S 1) Both studies were approved by the IRB of the Catholic University Eichstätt-Ingolstadt (2016/23 and 2015/02/16) and participants gave their written informed consent to participate in the respective study 2.2. Measure The Child and Adolescent Trauma Screen (CATS) assesses posttraumatic stress symptoms (PTSS) according to DSM-5 in youngsters aged 7 to 17 years [ 4 ]. Participants indicate whether they have experienced 15 potentially traumatic events (yes/no) and specify the currently most distressing event. Then, they rate 20 items on PTSS on a 4-point Likert scale (0— never to 3— almost always ). The sum score is between 0 and 60, with scores ≥ 21 indicating clinically relevant PTSS. The four subscales—intrusion, avoidance, negative alterations in cognition and mood (NACM), and hyperarousal—can be formed, which refer to the PTSD diagnostic criteria in the DSM-5. The CATS has been validated in English, German, and Norwegian language versions. We used the German version. In half of the sample, n = 72 (50%), interpreters supported diagnostics (for one Study 1 participant, information on interpreter involvement was missing) 3. Data Analysis We used SPSS version 25 (IBM, Armonk, USA) for all analyses. We examined descriptive statistics and internal consistency (Cronbach’s α ) for the CATS total score and symptom subscales, and for cases with and without involvement of an interpreter. We computed χ 2 -statistics to examine differences in internal consistency between the two subgroups using cocron [ 14 ]. A confirmative factor analysis was applied to investigate the previously confirmed DSM-5 factor structure using AMOS for SPSS version 25. For RM-

Int. J. Environ. Res. Public Health 2021 , 18 , 6803 3 of 7 SEA, values ≤ 0.06 (marginal: 0.07–0.08) [ 15 ] and for CFI and TLI, values ≥ 0.95 (marginal: 0.90–0.94) indicate a good fit [ 16 ]. 4. Results 4.1. Internal Consistency The CATS showed a good internal consistency with Cronbach’s α = 0.84 in the total sample. Internal consistency was slightly, but not significantly, higher when assessed with an interpreter ( n = 72; α = 0.85) than without one ( n = 72; α = 0.82; χ 2 (1) = 0.684, p = 0.408) The four symptom clusters differed in their internal consistency and showed rather poor results (see Table 1 ). Internal consistencies of the four subscales were consistently, but not significantly, lower in the group without an interpreter, with avoidance even having a negative value in this group Table 1. Internal consistencies (Cronbach’s α ) of the four subscales in the full sample and the two subsamples Full Sample Interpreter No Interpreter Group Difference ( n = 145) ( n = 72) ( n = 72) χ 2 (1) p CATS total 0.838 0.852 0.818 0.684 0.408 Intrusion 0.727 0.748 0.699 0.370 0.543 Avoidance 0.310 0.432 − 0.029 1.813 a 0.178 NACM 0.659 0.656 0.654 <0.001 0.983 Hyperarousal 0.585 0.606 0.580 0.052 0.821 Note. CATS = Child and Adolescent Trauma Screen. NACM = Negative alterations in cognition and mood a As it was not possible to compute the comparison with a negative value of α , α = 0 was substituted in the group without interpreter 4.2. Symptom Scores and Item Analyses Participants from Study 1 had lower average CATS scores, M = 22.40, SD = 9.35, than participants from Study 2, M = 26.87, SD = 10.35, F (143) = 2.61, p = 0.010. CATS scores did not vary with interpreter involvement in either subsample. In Study 1, cases without interpreters had a mean CATS score of M = 21.56, SD = 9.07, and cases with interpreters of M = 23.91, SD = 9.78, T (96) = -1.200, p = 0.233, and in Study 2, cases without interpreters had a mean CATS score of M = 29.33, SD = 7.14 and with interpreters M = 26.51, SD = 11.03, T (44) = 0.727, p = 0.471 Means and standard deviations, as well as corrected item-scale correlations of the 20 CATS items, are given in Table 2 . Five items in the sample without interpreters, and three items in the sample with interpreters had item-scale correlations ≤ 0.30, resulting in five items (8, 10, 12, 16, and 17) in the overall sample with item-scale correlations ≤ 0.30 4.3. Factorial Validity The four-factor CFA model proposed by the DSM-5 and found by Sachser et al. (2017) did not show a very good fit. While RMSEA was good, at 0.06 (90% CI 0.04–0.07), the CFI was, at 0.86, below the acceptable range. Removing five items with item-scale correlations at or below r = 0.30 resulted in a slightly worse, but still acceptable, RMSEA of 0.07 (90% CI 0.05–0.09), and improved CFI to 0.89 (see Table 2 ).

Int. J. Environ. Res. Public Health 2021 , 18 , 6803 4 of 7 Table 2. Item means, standard deviations and corrected item-scale correlations in the full sample and the two subsamples Full Sample ( n = 145) Interpreter ( n = 72) No Interpreter ( n = 72) Item M SD ISC M SD ISC M SD ISC 1. Upsetting thoughts or pictures about what happened that pop into your head 1.61 0.96 0.45 1.67 0.99 0.42 1.56 0.93 0.48 2. Bad dreams reminding you of what happened 1.37 0.96 0.40 1.38 0.94 0.45 1.35 0.98 0.37 3. Feeling as if what happened is happening all over again 0.99 1.05 0.47 1.19 a 1.11 0.50 0.81 a 0.96 0.41 4. Feeling very upset when you are reminded of what happened 1.73 0.95 0.52 1.85 0.94 0.54 1.63 0.94 0.48 5. Strong feelings in your body when you are reminded of what happened (sweating, heart beating fast, upset stomach) 1.37 1.03 0.50 1.44 1.05 0.49 1.28 1.02 0.52 6. Trying not to think about what happened. Or to not have feelings about it 2.00 0.90 0.37 2.14 0.92 0.52 1.88 0.85 0.17 7. Staying away from anything that reminds you of what happened (people, places, things, situations, talks) 1.05 1.11 0.38 1.32 b 1.17 0.33 0.79 b 0.99 0.40 8. Not being able to remember part of what happened 0.70 0.89 0.13 0.74 0.87 0.15 0.67 0.92 0.10 9. Negative thoughts about yourself or others. Thoughts like I will not have a good life, no one can be trusted, the whole world is unsafe 1.25 1.12 0.51 1.29 1.14 0.60 1.22 1.10 0.39 10. Blaming yourself for what happened. Or blaming someone else when it is not their fault 0.67 0.87 0.30 0.69 0.87 0.33 0.65 0.87 0.25 11. Bad feelings (afraid, angry, guilty, ashamed) a lot of the time 1.43 0.94 0.56 1.49 1.03 0.57 1.38 0.85 0.55 12. Not wanting to do things you used to do 1.14 1.14 0.30 1.17 1.15 0.27 1.11 1.15 0.33 13. Not feeling close to people 0.86 0.98 0.43 0.94 1.05 0.43 0.78 0.91 0.42 14. Not being able to have good or happy feelings 1.15 1.00 0.57 1.24 1.05 0.61 1.07 0.95 0.52 15. Feeling mad. Having fits of anger and taking it out on others 1.03 0.96 0.45 1.06 1.01 0.43 1.01 0.93 0.48 16. Doing unsafe things 0.33 0.64 0.29 0.26 0.60 0.44 0.40 0.66 0.16 17. Being overly careful (checking to see who is around you) 1.23 1.12 0.17 1.19 1.11 0.14 1.26 1.14 0.22 18. Being jumpy 1.11 1.05 0.43 1.03 1.01 0.39 1.19 1.10 0.52 19. Problems paying attention 1.12 1.00 0.52 1.32 c 1.06 0.59 0.93 c 0.91 0.41 20. Trouble falling or staying asleep 1.72 1.13 0.55 1.85 1.16 0.58 1.57 1.09 0.53 Note. The subscales are composed as follows: Intrusion (items 1, 2, 3, 4, 5), avoidance (items 6, 7), NACM (items 8, 9, 10, 11, 12, 13, 14), and hyperarousal (items 15, 16, 17, 18, 19, 20) M = mean; SD = standard deviation; ISC = item-scale correlations. ISCs ≤ 0.30 are in bold print a significant difference between means, T (139.1) = − 2.251, p = 0.026 b significant difference between means, T (138.1) = − 2.915, p = 0.004 c significant difference between means, T (138.8) = − 2.364, p = 0.019.

Int. J. Environ. Res. Public Health 2021 , 18 , 6803 5 of 7 5. Discussion We reported on the post hoc psychometric evaluation of the CATS as a function of interpreter involvement in a sample of severely traumatized refugee youngsters from 17 countries. The CATS previously showed good psychometric properties with translated versions and samples from Germany, Norway, and the US [ 4 ]. In our sample, the overall internal reliability was lower, but still good and did not differ between the subgroup that involved an interpreter and the one that did not. In our sample, however, avoidance, hyperarousal, and NACM showed questionable to very poor alphas, with a negative α for avoidance in the group without an interpreter. It must be kept in mind, however, that this subscale has only two items dealing with different forms of avoidance and is, therefore, not ideal to begin with from a factor analytic point of view [ 17 ]. Although the internal consistencies of the subscales tended to be consistently better in the group with interpreters, some of them were still outside the acceptable range. On the one hand, this could indicate that the young refugees were unfamiliar with psychological test formats [ 18 ], that language difficulties might have occurred, or that culturally divergent concepts were captured by the questionnaires. On the other hand, the sometimes low reliability even in the interpreter group may also stem from the difficulty of translating almost simultaneously in the screening situation and the resulting quality of translation for culturally disputable constructs Accordingly, translated questionnaire versions, in which the translation process is more elaborate due to iterative improvements, could contribute to increasing the reliability in the recording of the subscales [ 19 ]. Five items had low corrected item-scale correlations of 0.30 or below, and this problem was more pronounced in the group not supported by an interpreter. The four-factor model showed an acceptable fit only after removing items with bad item characteristics and even then, fit indices were worse than those reported in the original study [ 4 ]. Given the heterogeneity of our sample including refugee youth from 17 countries, this deviation from the original study, which examined more homogeneous national samples, may not, however, be too surprising 5.1. Limitations A key limitation of this analysis is its retrospective nature and the fact that the variable of interest is difficult to manipulate experimentally. Study clinicians decided whether to employ an interpreter for screening according to their perception of participants’ German language skills and preferences. However, most studies on the impact of interpreters on mental health services share this limitation. While sample 1 offered voluntary participation in a screening study, sample 2 was a service use sample, which might have resulted in selection biases affecting this analysis. Furthermore, a third group using translated versions was lacking and the subsamples were too small so as to conduct CFAs for both groups separately 5.2. Implications Overall, the psychometric differences between the two groups were small and tended to favor the assistance of interpreters (internal consistency, item-scale correlations). Considering the multilingual population of young refugees in Germany, where translated versions of screening measures are often lacking, the common practice to employ lay interpreters does not seem to be disadvantageous, at least in terms of reliability 6. Conclusions The CATS is a reliable screening instrument for PTSS in culturally diverse refugee youth samples, including its use with the involvement of interpreters, making it a feasible screening tool for professionals working in this field (available at https://ulmeronlineklinik.de/course/view.php?id=1701 , accessed on 24 June 2021). Assessments should be followed by a (semi-)structured interview to ascertain clinically relevant symptoms or diagnostic status. From a scientific point of view, these findings should be verified

Int. J. Environ. Res. Public Health 2021 , 18 , 6803 6 of 7 by means of a validation of the CATS with a gold standard clinical interview. Given the paucity of quantitative research findings on the influence of interpreters on processes in mental health services, which has been emphasized by a variety of research groups [ 10 , 20 ], this study can be understood as a starting point for further rigorous studies, despite the limitations due to the post hoc design Supplementary Materials: The following are available online at https://www.mdpi.com/article/10 .3390/ijerph 18136803/s 1 , Table S 1: Characteristics of study sample Author Contributions: L.R.F.M. undertook the data collection in Study 1 and was responsible for drafting the manuscript. J.U. was principal investigator in both studies and was responsible for drafting the manuscript and analyzing the data. S.W. undertook the data collection in Study 2 and was responsible for the integration of datasets. R.R. supervised both studies and revised the manuscript. J.K. was involved in all issues with regard to data analysis and manuscript drafting. All authors have read and agreed to the published version of the manuscript Funding: This work was supported by the funding program proFor+ of the Catholic University Eichstätt-Ingolstadt (ID: 2316). Publication of this work was supported by the German Research Foundation (DFG) within the funding programme Open Access Publishing Institutional Review Board Statement: This is an additional analysis of two pre-studies. The studies were conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of the Catholic University Eichstätt-Ingolstadt (protocol code 2016/23 and 2015/02/16) Informed Consent Statement: Informed consent was obtained from all subjects involved in the study. Written informed consent has been obtained from all participants to publish this paper Data availability statement: The datasets generated and analyzed during this study are not publicly available due to sensitive and potentially identifying participant information but are available from the corresponding author on reasonable request Acknowledgments: We would like to thank Margret Lang and Karl Büter and all participants Conflicts of Interest: The authors declare no conflict of interest. The sponsors had no role in the design, execution, interpretation, or writing of the study Abbreviations CATS Child and Adolescent Trauma Screen CFA Confirmatory factor analysis CFI Comparative fit index DSM-5 Diagnostic and Statistical Manual for Mental Disorders, 5 th edition NACM Negative alterations in cognition and mood RMSEA Root mean square error of approximation PTSD Post-traumatic stress disorder PTSS Post-traumatic stress symptoms SPSS Statistical Package for the Social Sciences TLI Tucker–Lewis index References 1 Müller, L.R.F.; Büter, K.P.; Rosner, R.; Unterhitzenberger, J. Mental health and associated stress factors in accompanied and unaccompanied refugee minors resettled in Germany: A cross-sectional study Child Adolesc. Psychiatry Ment. Health 2019 , 13 , 8 [ CrossRef ] [ PubMed ] 2 Von Werthern, M.; Grigorakis, G.; Vizard, E. The mental health and wellbeing of Unaccompanied Refugee Minors (URMs) Child Abus. Negl 2019 , 98 , 104146. [ CrossRef ] [ PubMed ] 3 Rosner, R.; Sachser, C.; Hornfeck, F.; Kilian, R.; Kindler, H.; Muche, R. Improving mental health care for unaccompanied young refugees through a stepped-care approach versus usual care+: Study protocol of a cluster randomized controlled hybrid effectiveness implementation trial Trials 2020 , 21 , 1–13. [ CrossRef ] [ PubMed ] 4 Sachser, C.; Berliner, L.; Holt, T.; Jensen, T.K.; Jungbluth, N.; Risch, E. International development and psychometric properties of the Child and Adolescent Trauma Screen (CATS) J. Affect Disord 2017 , 210 , 189–195. [ CrossRef ] [ PubMed ]

Int. J. Environ. Res. Public Health 2021 , 18 , 6803 7 of 7 5 American Psychiatric Association Diagnostic and Statistical Manual of Mental Disorders [DSM-5] , 5 th ed.; American Psychiatric Association: Washington, DC, USA, 2013. [ CrossRef ] 6 Unterhitzenberger, J.; Wintersohl, S.; Lang, M.; König, J.; Rosner, R. Providing manualized individual trauma-focused CBT to unaccompanied refugee minors with uncertain residence status: A pilot study Child Adolesc. Psychiatry Ment. Health 2019 , 13 , 22 [ CrossRef ] [ PubMed ] 7 Pfeiffer, E.; Sukale, T.; Müller, L.R.F.; Plener, P.L.; Rosner, R.; Fegert, J.M. The symptom representation of posttraumatic stress disorder in a sample of unaccompanied and accompanied refugee minors in Germany: A network analysis Eur. J. Psychotraumatol 2019 , 10 , 1675990. [ CrossRef ] [ PubMed ] 8 Jakobsen, M.; Demott, M.A.M.; Heir, T. Prevalence of psychiatric disorders among unaccompanied asylumseeking adolescents in Norway Clin. Pr. Epidemiol. Ment. Health 2014 , 10 , 53–58. [ CrossRef ] [ PubMed ] 9 Gartley, T.; Due, C. The Interpreter Is Not an Invisible Being: A Thematic Analysis of the Impact of Interpreters in Mental Health Service Provision with Refugee Clients Aust. Psychol 2017 , 52 , 31–40. [ CrossRef ] 10 Rousseau, C.; Measham, T.; Moro, M.-R. Working with Interpreters in Child Mental Health Child Adolesc. Ment. Health 2011 , 16 , 55–59. [ CrossRef ] [ PubMed ] 11 d’Ardenne, P.; Ruaro, L.; Cestari, L.; Fakhoury, W.; Priebe, S. Does Interpreter-Mediated CBT with Traumatized Refugee People Work? A Comparison of Patient Outcomes in East London Behav. Cogn. Psychother 2007 , 35 , 293–301. [ CrossRef ] 12 Sander, R.; Laugesen, H.; Skammeritz, S.; Mortensen, E.L.; Carlsson, J. Interpreter-mediated psychotherapy with trauma-affected refugees—A retrospective cohort study Psychiatry Res 2019 , 271 , 684–692. [ CrossRef ] [ PubMed ] 13 Cohen, J.A.; Mannarino, A.P.; Deblinger, E Treating Trauma and Traumatic Grief in Children and Adolescents , 2 nd ed.; Guilford Publications: New York, NY, USA, 2017 14 Diedenhofen, B.; Musch, J. cocron: A Web Interface and R Package for the Statistical Comparison of Cronbach’s Alpha Coefficients Int. J. Internet Sci 2016 , 11 , 51–60 15 MacCallum, R.C.; Browne, M.W.; Sugawara, H.M. Power analysis and determination of sample size for covariance structure modeling Psychol. Methods 1996 , 1 , 130–149. [ CrossRef ] 16 Hu, L.-T.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives Struct. Equ. Model 1999 , 6 , 1–55. [ CrossRef ] 17 Kline, R.B Principles and Practice of Structural Equation Modeling ; Guilford Publications: New York, NY, USA, 2005 18 Ehntholt, K.A.; Yule, W. Practitioner review: Assessment and treatment of refugee children and adolescents who have experienced war-related trauma J. Child Psychol. Psychiatry 2006 , 47 , 1197–1210. [ CrossRef ] [ PubMed ] 19 Bhui, K.; Mohamud, S.; Warfa, N.; Craig, T.J.; Stansfeld, S.A. Cultural adaptation of mental health measures: Improving the quality of clinical practice and research Br. J. Psychiatry 2003 , 183 , 184–186. [ CrossRef ] [ PubMed ] 20 Cecchet, S.J.; Calabrese, D. Interpreter-Mediated Therapy for Refugees: A Need for Awareness and Training Grad. Stud. J. Psychol 2011 , 13 , 12–16.