The Effectiveness of a Leadership Subject Using a Hybrid Teaching Mode during...

[Full title: The Effectiveness of a Leadership Subject Using a Hybrid Teaching Mode during the Pandemic: Objective Outcome and Subjective Outcome Evaluation]

Citation: Chai, W.; Li, X.; Shek, D.T.L The Effectiveness of a Leadership Subject Using a Hybrid Teaching Mode during the Pandemic: Objective Outcome and Subjective Outcome Evaluation Int. J. Environ Res. Public Health 2022 , 19 , 9809 https://doi.org/10.3390/ ijerph 19169809 Academic Editor: Iv á n Herrera-Peco Received: 17 May 2022 Accepted: 5 August 2022 Published: 9 August 2022 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations Copyright: © 2022 by the authors Licensee MDPI, Basel, Switzerland This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/) International Journal of Environmental Research and Public Health Article The Effectiveness of a Leadership Subject Using a Hybrid Teaching Mode during the Pandemic: Objective Outcome and Subjective Outcome Evaluation Wenyu Chai, Xiang Li and Daniel T. L. Shek * Department of Applied Social Sciences, The Hong Kong Polytechnic University, Hong Kong, China * Correspondence: daniel.shek@polyu.edu.hk Abstract: Positive youth development (PYD) is an innovative approach to protect students from mental health problems and promote their positive and holistic development. Although there are many studies on the beneficial effects of PYD programs on youth in high school and community contexts, it is not clear whether subjects adopting PYD principles can promote positive development for university students. Moreover, it is unclear whether such subjects are effective under COVID-19, where subjects are commonly taught via the “hybrid” mode (i.e., face-to-face plus online teaching). The present study examined students’ changes in the PYD, wellbeing, and desired graduate attributes after they had taken a leadership subject utilizing PYD principles taught by the “hybrid” mode ( N = 630). Adopting the one-group pre-test and post-test design (i.e., objective outcome evaluation), we found that students showed significant positive improvement in PYD indicators, wellbeing, as well as desired graduate attributes. Additionally, students had high satisfaction with the course design and teaching staff, and perceived many benefits from this subject based on the subjective outcome evaluation conducted at the end of the subject. Results also showed that students’ satisfaction with the curriculum significantly and positively predicted their positive change in PYD indicators, indicating the convergence of subjective outcome evaluation and objective outcome evaluation. The results highlight the positive impacts of the hybrid mode leadership subject with PYD principles in higher education Keywords: positive youth development; university students; leadership program; higher education; hybrid mode teaching; COVID-19 pandemic 1. Introduction 1.1. Challenges Faced by the Higher Education Sector In recent years, there has been increasing concern for the positive and holistic development of university students due to both internal and external challenges faced by higher education [ 1 – 3 ]. Regarding internal challenges, the transition to university is a significant challenge as students need to deal with multifaceted adjustment issues, such as new academic and social environments, high independency, different learning modes, being away from home, peer pressure, and academic and financial demands [ 4 – 7 ]. These adjustment difficulties would also expose students to a higher risk of mental health problems [ 4 ]. In fact, research consistently showed high prevalence rates of mental health problems (e.g., depression, anxiety, and psychological distress) among university students, which were substantially higher than the general population [ 8 – 11 ]. Poor mental health would lead to other negative consequences, such as an increase in problem behaviour, as well as a decrease in physical health conditions, academic performance, and life satisfaction [ 12 – 14 ]. Therefore, it is important to develop effective preventive programs for university students. One possible direction is to promote the developmental assets of young people because research indicated that psychological strengths and positive qualities Int. J. Environ. Res. Public Health 2022 , 19 , 9809. https://doi.org/10.3390/ijerph 19169809 https://www.mdpi.com/journal/ijerph

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 2 of 17 could promote students’ well-being (e.g., self-esteem, self-efficacy, resilience, optimism, social emotional competence), and buffer the negative impact of different challenges and pressures on student development [ 4 , 15 , 16 ]. In addition to internal challenges, higher education is also facing challenges posed by the changing economic and societal conditions. One major social change is the transformation from a manufacturing economy to a service economy in recent decades [ 17 ]. Although the manufacturing economy focuses on standardized production and requires semi-skilled or skilled manual workers, the service economy depends on knowledge, information, innovation, and requires holistic qualities of employees [ 18 , 19 ]. To produce high-quality services, intrapersonal and interpersonal competences, such as effective communication, problem solving, critical thinking, and creativity, are significantly required [ 17 , 20 ]. In addition, knowledge explosion and rapid advancement of science and technology make work and life “uncertain” and “ever-changing” [ 18 ], which also requires high adaptability, flexibility, lifelong learning, as well as capacities to deal with challenges such as resilience and emotional competence [ 18 , 21 ]. Hence, it is important for higher education to find effective means to develop these intrapersonal and interpersonal qualities and to promote holistic development of university students 1.2. Positive Youth Development (PYD) Approach To find effective means to prevent students from developing mental health problems and promote their holistic development, the positive youth development (PYD) framework is a promising approach. As an innovative paradigm in psychology and youth studies, PYD focuses on the development of strengths and potentials of youths, which is different from the pathological orientation of traditional psychological research and youth studies [ 22 – 24 ]. Different theories and models of PYD were proposed by scholars. Lerner and his colleagues [ 25 ] conceptualized PYD as a developmental process achieved through positive interaction between individual and his/her environments. If the environment is constructed to promote the development of strengths of adolescents, PYD occurs which could be indicated by 5 qualities, namely “competence”, “confidence”, “connection”, “character”, and “caring” [ 25 ]. Catalano and his colleagues [ 23 ] proposed 15 indicators of PYD after studying different PYD programs in the United States. These 15 indicators include “cognitive competence”, “social competence”, “emotional competence”, “clear and positive identity”, “resilience”, “spirituality”, and prosocial norms. Benson and his colleagues [ 22 ] proposed the concept of developmental assets under the framework of PYD which includes internal and external assets. The internal assets refer to internal positive qualities or competences including “commitment to learning”, “positive values”, “social competencies”, and “positive identity”; while the external assets refer to quality relationships received from the environment including “support”, “empowerment”, “boundaries”, and “expectations” from parents, peers, and community, and “constructive use of time” [ 26 ] (p. 29). Although there are differences among these perspectives, they all share a common idea that intervention programs should be developed to promote the development of positive qualities and strengths to protect students from problems and foster their positive development and thriving [ 27 ]. In line with the theoretical postulations, empirical findings showed that higher levels of PYD predicted lower levels of mental health problems indexed by depression, anxiety, emotional problems, hopelessness, and suicidal ideation [ 28 – 30 ]. Higher levels of PYD also predicted lower levels of problem behaviour, such as delinquency and internet addiction [ 31 , 32 ], and higher levels of healthy functioning, academic performance, and academic wellbeing [ 33 , 34 ]. Taken as a whole, both theoretical models and empirical findings suggest that PYD is a potentially effective approach to foster holistic development of university students to help them meet both internal and external challenges.

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 3 of 17 1.3. PYD and Wellbeing of University Students Among different correlates and outcomes of PYD, research consistently showed that PYD attributes were positively linked to subjective wellbeing, such as life satisfaction [ 35 ] and thriving [ 26 ]. For example, research on Chinese adolescents showed that PYD positively predicted life satisfaction among secondary school students in Hong Kong in both cross-sectional and longitudinal studies [ 36 , 37 ]. In addition, PYD indicators, such as the five “Cs” and developmental assets, also significantly and positively predicted youth thriving [ 26 , 38 ]. Thriving refers to an ideal status, manifestation, exemplar, outcome, or process of positive youth development [ 22 , 39 , 40 ]. It is manifested by adolescents’ active participation, commitment, and contribution to their community and society [ 39 ]. Scholars argued that adolescents’ development of the PYD indicators, such as the five “Cs” or developmental assets, would finally contribute to thriving [ 21 , 40 ]. 1.4. PYD Programs and Related Research Along with its theoretical development and empirical studies, PYD models have been adopted and incorporated in many programs in educational and social service sectors There is also a body of literature showing positive effects of these programs [ 41 , 42 ]. For example, a systematic review of 23 school-based PYD programs showed that these programs had significant and positive impact on students’ development of different psychological strengths and wellbeing [ 41 ]. Particularly, the review showed that curriculaand lecturebased PYD programs promoted students’ social and emotional skills and other aspects of positive development. In addition, a meta-analysis on the effects of 82 school-based social emotional learning interventions based on the PYD approach revealed that these interventions significantly promoted students’ social and emotional competence, as well as other indicators of wellbeing [ 43 ]. Furthermore, a meta-analysis on 24 outside-school PYD interventions showed that, regardless of program features, these programs had a significant and positive impact on adolescents’ psychological adjustment and academic performance although with a small effect size [ 44 ]. However, these programs had no effect on promoting positive social behaviours and reducing problem behaviours. Moreover, several studies showed that PYD programs promoted the PYD attributes and life satisfaction, as well as reduced depression and problem behaviours in Chinese secondary school students [ 32 , 45 ]. 1.5. Research Gaps There are several research gaps in the existing literature in this field. First, most of the existing studies mainly focused on PYD programs and their effects in high school and community contexts, with very few studies on university students, who are an important group of youths or emergent adults. Second, there is inadequate research on the application of PYD programs in leadership programs (such as credit-bearing subjects) in a non-Western context. Based on a review of research on PYD from 1995 to 2020 [ 46 ], one major research gap identified was the dominating position of Western studies in PYD research. In another review of PYD studies in the Asian context, it was found that there were very few validated PYD programs in Asian communities with rigorous evaluation design [ 47 ]. Therefore, there is an urgent need for research on PYD programs and their effects in the Asian context Third, there are a few studies on the effects of educational programs based on the PYD framework under the COVID-19 which adopt the hybrid and online modes of teaching [ 48 – 51 ]. The pandemic has changed the traditional format of teaching and learning in many higher education institutions. To cope with challenges of the pandemic, many institutions have to adjust their teaching and learning approach from the traditional face-to-face mode to either pure online mode or a hybrid mode which allows students to continue their study [ 52 , 53 ]. Particularly, hybrid-mode teaching has become a widely adopted practice in higher education during the COVID-19 pandemic. Although there are different definitions of hybrid-mode teaching, it, in a broader sense, refers to an integration of online learning (synchronized/asynchronized) with face-to-face teaching [ 54 ]. In a narrow sense, scholars argued that hybrid-mode teaching focuses more on conducting online synchronized teach-

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 4 of 17 ing and face-to-face teaching simultaneously in the same class session to accommodate the needs of students in different locations [ 55 ], which has been named as “HyFlex” teaching However, there are conflicting perceptions on the effects of hybrid-mode teaching Some scholars argued that the hybrid-mode teaching had a lot of benefits, such as allowing high flexibility, more student choices, and catering different needs [ 55 ]. However, some scholars argued that the hybrid-mode teaching could have negative effects on student learning, such as enlarged differences in students’ learning experience and undermining class and group cohesion and students’ sense of belonging [ 54 ]. Unfortunately, there are limited empirical studies on the effect and impact of hybrid-mode teaching [ 51 ]. Some recent studies showed that students encountered challenges in online learning, such as their lack of readiness, as well as physical and psychological preparation for the online learning [ 56 ]. Nevertheless, there are also studies showing that students performed better during the pandemic due to the increased self-autonomy in learning strategies [ 57 ] and they preferred the flexibility in online and hybrid learning [ 52 ]. Under the pandemic, it is urgent to understand the effects of PYD program adopting hybrid-mode on students’ development Fourth, there is a need to understand the impacts of PYD programs based on rigorously designed evaluation studies [ 47 ]. Particularly, there is a need to include both objective outcome evaluation and subjective outcome evaluation to provide a more comprehensive understanding of the program effects and there is a need to advance our understanding of the association between objective outcome evaluation and subjective outcome evaluation. Subjective outcome evaluation is an important and commonly used evaluation method in educational, health care, and social service fields [ 58 , 59 ]. It provides important information on the perceptions, feelings, and viewpoints of program participants who are important stakeholders and beneficiaries of intervention programs. However, this evaluation method was criticized by some scholars as being biased and not related to objective evaluation outcomes [ 60 ]. Additionally, there are inconsistent empirical findings. Some studies found no or a weak correlation between subjective measures and objective measures [ 61 , 62 ], while some studies showed that subjective outcome evaluation results were significantly correlated with objective outcome evaluation results [ 63 – 65 ]. Therefore, there is a strong need to further test the relationship between subjective outcome evaluation and objective outcome evaluation findings to advance understanding in this area to determine the effectiveness of educational and intervention programs 1.6. A Leadership Program Based on PYD Framework in a Local University in Hong Kong Promotion of holistic development among university students is an urgent task for higher education institutions in Hong Kong. Similar to the international landscape, there is a high prevalence of mental health problems among Hong Kong university students. For example, in a recent survey on 1200 undergraduate students from the eight public universities in Hong Kong, 68.5% students reported mild to severe depression symptoms associated with mild to severe anxiety symptoms [ 66 ]. In another recent study, the prevalence rate of depression among university students in Hong Kong was much higher (41.0%) than those in mainland China (16.8%) and Macau (35.2%) [ 67 ]. In addition, the dominance of the service economy in Hong Kong raises requirements for university graduates with strong intrapersonal and interpersonal competences [ 68 , 69 ]. Against this background, a leadership program named “Tomorrow’s Leaders” (TL) was developed based on the PYD framework in one public university in Hong Kong to facilitate students’ positive and holistic development. The TL program is a three-credit compulsory subject for all the firstyear undergraduates enrolled in the university. It aims to equip students with important concepts and theories of PYD qualities (i.e., intrapersonal and interpersonal qualities) that are essential for being future leaders of the society. Specifically, the subject has six intended learning outcomes: (1) “understand and integrate theories, research and concepts on the basic qualities (particularly intrapersonal and interpersonal qualities) of effective leaders”; (2) “develop self-awareness and self-understanding”; (3) “demonstrate self-leadership in pursuit of continual self-improvement”; (4) “apply intrapersonal and interpersonal skills in

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 5 of 17 daily lives”; (5) “appreciate the importance of intrapersonal and interpersonal qualities in effective leadership”; and (6) “recognize and accept their responsibility as professionals and citizens to the society and the world”. These intended learning outcomes are also aligned with the desired graduate attributes of the university, such as life-long learning, problem solving, critical thinking, and ethical leadership. With reference to the objectives, the content of the subject focuses on 9 essential intrapersonal and interpersonal qualities delivered in 13 lectures including “self-leadership”, “social emotional competence”, “resilience and stress-coping”, “morality”, “spirituality”, “law-abiding leadership”, “cultural competence and global citizenship”, “effective communication”, and “team building”. For each lecture, there are group discussion, reflective exercises, role play, case discussion to facilitate students’ experiential learning, active learning, and self-reflection to link and apply the learned quality in daily life and future profession. In addition, three assessment strategies (i.e., class participation, group project, and term paper) are used to help students consolidate their learning experience Since its implementation from 2012 to 2019, the subject was delivered in face-to-face mode. Different evaluation studies showed that the subject was effective in promoting students’ development in PYD qualities and wellbeing [ 70 – 72 ]. However, due to the COVID-19 pandemic outbreak in Hong Kong, most of the universities adopted either pure online teaching or hybrid teaching mode. In accordance with the university policy, the TL subject transformed its teaching and learning from face-to-face mode to a hybrid-mode in Semester 1 of the 2021/22 academic year. Specifically, for each lecture, the original three-hour face-to-face lecture was changed to a two-hour lecture plus one-hour nonsynchronized before-class online self-regulated learning. For the two-hour lecture, students could attend the lecture either in face-to-face approach or online approach (i.e., HyFlex mode) based on their preferences and availability 1.7. The Present Study Although the traditional face-to-face mode TL subject was shown to be effective in promoting students’ development in PYD qualities and wellbeing, whether the subject is still effective by adopting the hybrid-mode teaching under the COVID-19 is unknown. Hence, the present study aimed to examine the effects of the hybrid-mode TL subject. With reference to the above literature review, three research questions and related hypotheses are proposed: 1 Do students change after taking the TL subject with PYD focus via the hybrid mode of teaching and learning? Based on the past studies [ 70 , 71 ], we expected that students would show positive changes in the outcome indicators, including PYD attributes (Hypothesis 1 a), well-being (Hypothesis 1 b) and desired graduate attributes (Hypothesis 1 c); 2 Are students satisfied with their learning experience, including perceptions of course design, teaching staff, and benefits of the subject delivered via the hybrid mode? With reference to past studies [ 73 , 74 ], we hypothesized that a majority of the students would have positive perceptions of the course design, teacher, and benefits of the subject (Hypothesis 2); 3 Is students’ satisfaction with their learning experience associated with the objective outcome evaluation measures? With reference to the literature [ 63 , 75 , 76 ], we expected that students’ satisfaction would be significantly related to their outcomes over time (Hypothesis 3) in objective outcome evaluation 2. Materials and Methods 2.1. Participants and Procedure The one-group pre-test and post-test design was adopted in the present study. This method is widely used in different research fields, such as clinical health, education, social work, and program evaluation [ 77 – 79 ]. Many studies have adopted this method to evaluate the effects of intervention or education programs including those studies published in higher-ranking journals, such as International Journal of Environmental Research and Public

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 6 of 17 Health [ 77 ], Research on Social Work Practice [ 78 ], and Assessment and Evaluation in Higher Education [ 7 ]. For example, Gil and Kim [ 77 ] used the one-group pre-test and post-test design to evaluate the effects of an online self-help intervention on preventing depressive symptoms. Son [ 79 ] used this design to examine the effects of an undergraduate course on nursing students’ empathy and caring for the elderly. Additionally, Chan and Holosko [ 78 ] evaluated the effectiveness of a youth media practice program on self-esteem and critical thinking using a one-group pre-test and post-test design Scholars argued that the method is useful and effective to understand the effects of intervention programs under several situations. First, one-group pre-test and post-test design would be adopted if it is difficult or unfeasible to form a control group [ 80 , 81 ]. This is a major reason for most evaluation and intervention studies that adopted this research design. For example, Kurki and his colleagues [ 82 ] adopted a one-group pre-test and post-test design to examine the effects of a digital mental health literacy program on the wellbeing of medical students due to the reason that “only one group is available” [ 82 ] (p. 10) and it is not feasible to divide students to control group and experimental group. In another study, Siu and his colleagues [ 83 ] examined the effects of two intervention studies on stress management using a one-group pre-test and post-test design due to the limit of budget and time to recruit a control group. As TL subject is a compulsory subject offered to nearly all the first-year students at this university as a requirement for graduation, it is practically infeasible to assign students to a control group who would not take the subject. Additionally, as the number of participants is large, it is quite difficult and expensive to recruit students who did not take this subject from another university as the control group. Second, one threat to the internal validity of the method is that there might be other factors influencing the results such as maturation and history (other events). However, this threat would be reduced if the duration of study is shorter [ 80 ]. As the duration of the present study was one semester (13 weeks), the possible influence of other factors (especially natural maturation) may be reduced The participants were recruited from the undergraduates taking the hybrid-mode TL subject in Semester 1 of the 2021/22 academic year. In the first lecture, all the students were invited to complete an online objective outcome evaluation questionnaire (pre-test) Formal consent was obtained from the participants through an online consent form. At the end of the semester, the students were invited to complete the same questionnaire again (post-test) and a subjective outcome evaluation questionnaire. A total of 630 students completed both preand post-test questionnaires and the subjective outcome evaluation questionnaire. Most of the participants (83.5%) were 18 years old or above and 48.2% of them were female. In addition, 62.0% of the participants were born in Hong Kong and 32.4% of them were born in mainland China. The detailed demographic information of the participants is shown in Table 1 . Table 1. Demographic information of participants ( n = 630) Demographic Variables Frequency Percent Gender Male 326 51.8% Female 303 48.2% Age 16 or below 1 0.2% 17 102 16.3% 18 338 54.2% 19 92 14.7% 20 or above 91 14.6% Place of Birth Hong Kong 390 62.0% Mainland China 204 32.4% Other places 35 5.6%

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 7 of 17 Table 1. Cont Demographic Variables Frequency Percent School/Faculty Faculty of Business 1 0.2% School of Design 4 0.6% Faculty of Humanities 59 9.4% Faculty of Construction and Environment 57 9.1% Faculty of Engineering 228 36.2% Faculty of Hotel and Tourism Management 47 7.5% Faculty of Health and Social Sciences 106 16.9% Faculty of Applied Science and Textiles 127 20.2% 2.2. Measures 2.2.1. Chinese Positive Youth Development Scale (CPYDS) The PYD qualities were measured using a short version “Chinese Positive Youth Development Scale” (CPYDS). Developed based on the PYD attributes identified by Catalano and his colleagues [ 23 ], CPYDS measures the positive youth development of Chinese adolescents [ 84 ] The scale possesses good psychometrical properties based on different validation studies [ 84 , 85 ] The short version CPYDS adopted in the present study contains 34 items assessing 12 PYD qualities (primary factors of the scale). The 12 qualities are “resilience” (e.g., “When I face difficulty, I will not give up easily”), “social competence” (e.g., “I know how to communicate with others”), “emotional competence” (e.g., “When I am unhappy, I can appropriately show my emotions”), “cognitive competence” (e.g., “I know how to find the causes of and solutions to a problem”), “behavioural competence” (e.g., “I can face criticisms with an open mind”), “moral competence” (e.g., “I will fulfil my promise”), “self-determination” (e.g., “I am able to make wise choices”), “self-efficacy” (e.g., “I can finish almost everything that I am determined to do”), “clear and positive identity”(e.g., “I know my strengths and weaknesses”), “beliefs in the future” (e.g., “I have confidence to graduate from university”), “prosocial norms” (e.g., “It is my pleasure to obey rules and regulations”), and “spirituality” (e.g., “My life is colourful and full of excitements”). In addition, there are three higher order factors which are “cognitive-behavioural competencies”, “general PYD qualities”, and “positive identity”. Each item was rated through a six-point scale (from “1 = Strongly Disagree” to “6 = Strongly Agree”). The short-version CPYDS obtained acceptable to good internal consistency for each primary factor and higher order factor in a recent study [ 7 ]. The Cronbach’s alpha for the primary factors and higher order factors ranged from 0.72 to 0.94 in the present study (Table 2 ) Table 2. Results of one-way repeated measures MANOVA (Students’ change between preand post-test) ( n = 630) Variables Pre-Test Post-Test Mean (SD) a Mean (SD) a F Value η 2 p Primary factors of Positive Youth Development 18.17 ** 0.261 Resilience 4.71 (0.87) 0.86 5.00 (0.87) 0.92 70.51 *** 0.101 Social Competence 4.69 (0.81) 0.90 5.00 (0.84) 0.94 75.63 *** 0.107 Emotional Competence 4.57 (0.79) 0.78 4.94 (0.84) 0.90 111.34 *** 0.150 Cognitive Competence 4.72 (0.72) 0.86 5.02 (0.81) 0.93 86.59 *** 0.121 Behavioural Competence 4.69 (0.77) 0.76 4.99 (0.83) 0.89 77.40 *** 0.110 Moral Competence 4.86 (0.74) 0.78 5.06 (0.80) 0.90 35.23 *** 0.053 Self-determination 4.57 (0.83) 0.83 4.93 (0.87) 0.89 92.76 *** 0.129 Self-efficacy 4.64 (0.84) 0.72 4.92 (0.92) 0.87 49.71 *** 0.073 Clear and Positive Identity 4.32 (0.91) 0.88 4.78 (0.92) 0.92 156.13 *** 0.199 Beliefs in the Future 4.79 (0.82) 0.87 5.01 (0.87) 0.91 42.68 *** 0.064 Prosocial Norms 4.92 (0.79) 0.76 5.10 (0.80) 0.83 25.30 *** 0.039 Spirituality 4.63 (0.92) 0.78 4.86 (0.95) 0.85 39.09 *** 0.059

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 8 of 17 Table 2. Cont Variables Pre-Test Post-Test Mean (SD) a Mean (SD) a F Value η 2 p Higher-order Positive Youth Development factors 32.28 *** 0.171 Cognitive-behavioural Competencies 4.67 (0.69) 0.92 4.99 (0.80) 0.96 102.14 *** 0.140 General PYD Qualities 4.68 (0.69) 0.94 4.97 (0.79) 0.97 87.06 *** 0.122 Positive Identity 4.52 (0.83) 0.92 4.88 (0.87) 0.95 114.58 *** 0.154 Total PYD 4.66 (0.68) 0.97 4.96 (0.78) 0.99 102.18 *** 0.140 Wellbeing 89.99 *** 0.223 Thriving 4.30 (0.67) 0.60 4.71 (0.82) 0.78 142.69 *** 0.185 Life Satisfaction 3.96 (0.98) 0.88 4.50 (1.08) 0.93 162.52 *** 0.205 Desirable Graduate Attributes 27.42 *** 0.150 Problem Solving 4.61 (0.75) 0.83 4.96 (0.85) 0.92 105.95 *** 0.145 Lifelong Learning 4.76 (0.80) 0.69 5.01 (0.83) 0.82 50.12 *** 0.074 Ethical Leadership 4.75 (0.63) 0.93 4.97 (0.74) 0.97 57.77 *** 0.084 Critical Thinking 4.76 (0.73) 0.88 5.03 (0.82) 0.94 66.61 *** 0.096 Note: ** p < 0.01; *** p < 0.001 2.2.2. Wellbeing We used two indicators to assess student wellbeing: life satisfaction and thriving. Life satisfaction was evaluated through the “Satisfaction with Life Scale” (SWLS) [ 86 ]. SWLS contains five items assessing the global life satisfaction of people which demonstrated good psychometric properties [ 86 , 87 ]. Each item is evaluated using a six-point scale (from “1 = Strongly Disagree” to “6 = Strongly Agree”). A sample item is “In most ways, my life is close to my ideal”. Thriving was assessed by a scale developed based on Lerner and his colleagues’ [ 40 ] theoretical proposition of thriving as exemplified condition of PYD The scale consists of five items evaluated through a six-point scale (from “1 = Strongly Disagree” to “6 = Strongly Agree”). The scale obtained good internal consistency in a previous study [ 7 ]. In the present study, the Cronbach’s alpha values for the scale are 0.60 and 0.78 in pre-test and post-test questionnaires, respectively 2.2.3. Desired Graduate Attributes Four desired graduate attributes were assessed, including perceived problem-solving ability, lifelong learning, ethical leadership, and critical thinking. Problem solving was assessed by 3 items with a sample item being “I know how to effectively solve problems in my daily life”. Lifelong learning was assessed by 2 items with a sample item being “It is important to understand the development of oneself”. Critical thinking was assessed by 3 items with a sample item being “I know how to use critical thinking skills when solving problems”. Ethical leadership was assessed by 15 items with one sample item being “I treat other people with dignity and respect”. All the items are rated on a six-point scale (from “1 = Strongly Disagree” to “6 = Strongly Agree”). The scales showed acceptable to good internal consistency in a previous study [ 7 ]. The Cronbach’s alpha values for the four scales in the present study ranged from 0.69 to 0.97 2.2.4. Subjective Outcome Evaluation Scale The participants’ satisfaction with their learning experience in TL subject was assessed by a Subjective Outcome Evaluation Scale (SOES) developed by the curriculum team based on validated measures used in previous studies [ 73 , 88 ]. The scale contains 41 items assessing students’ satisfaction with three aspects of their learning experience, namely their satisfaction with the course design (10 items), teacher (10 items), and benefits of the course (21 items). There are three additional items in the scale assessing students’ overall satisfaction with the course, their willingness to recommend the course to others, and their willingness to join similar courses in future. Previous studies showed that the scale possessed good

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 9 of 17 psychometric properties [ 73 , 74 ]. The Cronbach’s alpha values for satisfaction with the course design, satisfaction with the teacher, and satisfaction with the benefits of the course were 0.97, 0.98, and 0.99, respectively, indicating excellent internal consistency 2.3. Data Analyses First, descriptive analyses were conducted to understand the demographic information of the participants. To understand the participants’ changes in PYD attributes, wellbeing, and desired graduate attributes after taking the TL subject, a one-way repeated measures multivariate analysis of variance (MANOVA) was conducted. To understand students’ satisfaction with course design, teacher, and benefits of the course, descriptive analyses were computed to calculate frequencies and percentages of students’ ratings in subjective outcome evaluation. Multiple regression was also computed to examine whether students’ satisfaction with course design, teacher, and benefits of the course predicted their overall satisfaction with the subject. To examine the association between objective outcome evaluation and subjective outcome evaluation, hierarchical regression was conducted to examine the predicting effect of students’ perceived course design, teacher, and course benefits on post-test PYD qualities after controlling pre-test PYD qualities and demographic factors. All the above analyses were conducted using SPSS Version 25 3. Results 3.1. Objective Outcome Evaluation One-way repeated measures multivariate analysis of variance (MANOVA) was conducted to examine students’ changes in PYD attributes, wellbeing, and desired graduate attributes from pre-test to post-test. We found significant multivariate effects of time on 12 primary factors of PYD, the four higher-order factors of PYD, wellbeing, and desired graduate attributes ( F ranged between 18.17 and 89.99, ps < 0.001, η 2 p ranged between 0.039 and 0.261). Results showed positive improvement in all the 12 PYD attributes, higher-order PYD qualities, life satisfaction, thriving, and four desired graduate attributes ( ps < 0.001). Details can be found in Table 2 . The findings are presented in accordance with the format commonly adopted in quantitative studies including those published in the International Journal of Environmental Research and Public Health [ 7 , 89 ]. 3.2. Subjective Outcome Evaluation Students had very positive perceptions of course design, teacher quality, and course benefits. The percentages of positive ratings of the participants on different items in subjective outcome evaluation questionnaire ranged from 77.8% to 88.5% for course design, 91.7% to 94.9% for teacher quality, and 73.1% to 80.8% for course benefits. Details can be found in Table 3 . Table 3. Descriptive statistics and frequencies of positive responses in subjective outcome evaluation Items Participants with Positive Responses (Options 4–5) Mean (SD) n Student Perceptions of Course Design and Quality n (%) 1. The objectives of the curriculum are very clear 553 (88.5%) 4.24 (0.74) 625 2. The design of the curriculum is very good 509 (81.7%) 4.15 (0.83) 623 3. The activities were carefully arranged 546 (86.9%) 4.22 (0.77) 628 4. The classroom atmosphere was very pleasant 537 (85.6%) 4.24 (0.79) 627 5. There was much peer interaction amongst the students during classes 523 (83.3%) 4.19 (0.82) 628 6. I participated actively during lessons (including discussions, sharing, games, etc.) 508 (81.0%) 4.12 (0.84) 627 7. I was encouraged to do my best 535 (85.6%) 4.20 (0.77) 625 8. The learning experience I encountered enhance my interest towards the subject area 504 (80.0%) 4.09 (0.91) 630 9. Overall speaking, I have very positive evaluation of the program 528 (83.9%) 4.18 (0.84) 629 10. On the whole, I like this curriculum very much 487 (77.8%) 4.04 (0.92) 626

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 10 of 17 Table 3. Cont Items Participants with Positive Responses (Options 4–5) Mean (SD) n Student Perceptions of Lecturer Qualities 1. The lecturer has a good mastery of the curriculum 578 (91.7%) 4.38 (0.72) 630 2. The lecturer was well prepared for the lessons 590 (93.9%) 4.43 (0.68) 628 3. The teaching skills of the lecturer was good 576 (92.2%) 4.38 (0.70) 625 4. The lecturer showed good professional attitudes 593 (94.4%) 4.43 (0.67) 628 5. The lecturer was very involved 594 (94.6%) 4.44 (0.67) 628 6. The lecturer encouraged students to participate in the activities 588 (93.8%) 4.43 (0.67) 627 7. The lecturer cared for the students 581 (93.0%) 4.42 (0.69) 625 8. The lecturer was ready to offer help to students when needed 595 (94.9%) 4.44 (0.66) 627 9. The lecturer had much interaction with the students 583 (93.0%) 4.38 (0.68) 627 10. Overall speaking, I have very positive evaluation of the lecturer 587 (93.8%) 4.41 (0.68) 626 Perceptions of the Course Benefits 1. The subject has strengthened my resilience in adverse conditions 483 (77.0%) 4.05 (0.93) 627 2. The subject has enhanced my social competence 498 (79.3%) 4.08 (0.90) 628 3. The subject has improved my ability in expressing and handling my emotions 484 (77.1%) 4.05 (0.94) 628 4. The subject has enhanced my analytical ability 486 (77.3%) 4.03 (0.93) 629 5. The subject has enhanced my critical thinking 481 (76.5%) 4.04 (0.95) 629 6. The subject has strengthened my ability to distinguish between the good and the bad 481 (76.7%) 4.03 (0.95) 627 7. The subject has increased my competence in making sensible and wise choices 487 (77.7%) 4.06 (0.95) 627 8. This subject has helped me to have life satisfactions 499 (79.6%) 4.13 (0.92) 627 9. This subject has strengthened my self-confidence 459 (73.1%) 3.96 (1.01) 628 10. This subject has increased my self-awareness 495 (78.7%) 4.07 (0.92) 629 11. This subject has helped me to face the future with a positive attitude 483 (76.9%) 4.03 (0.98) 628 12. This subject has helped me to cultivate compassion and care about others 488 (77.6%) 4.03 (0.97) 629 13. This subject has strengthened my motivation to learn something new every day 477 (76.2%) 4.02 (0.97) 626 14. This subject has helped me to make ethical decision 498 (79.0%) 4.09 (0.91) 630 15. This subject has enhanced my desire for lifelong learning to improve leadership competence 490 (77.9%) 4.04 (0.97) 629 16. This subject has increased my ability to become an ethical leader 492 (78.2%) 4.07 (0.96) 629 17. The theories, research and concepts covered in the course have enabled me to understand the characteristics of successful leaders 509 (80.8%) 4.15 (0.90) 630 18. The theories, research and concepts covered in the course have helped me synthesize the characteristics of successful leaders 507 (80.5%) 4.13 (0.89) 630 19. It has enabled me to understand the importance of interpersonal relationship in successful leadership 503 (80.6%) 4.14 (0.88) 624 20. It has promoted my sense of responsibility in serving the society 485 (77.4%) 4.06 (0.95) 627 21. It has enriched my overall development 505 (80.4%) 4.11 (0.93) 628 Willingness to recommend friends to take the course Participants with Positive Responses (Options 3–4) Will you suggest your friends to take this course? 520 (83.1%) 3.08 (0.77) 626 Student Overall Satisfaction Participants with Positive Responses (Options 4–6) On the whole, are you satisfied with this course? 585 (92.9%) 4.70 (1.08) 630 For “Student Perceptions of Course Design and Quality” and “Students Perceptions of Lecturer Qualities”, all items are rated on a 5-point Likert-type scale with 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, 5 = strongly agree. Only the positive responses (Options 4–5) are shown. For “Student Perceptions of the Course Benefits”, all items are rated on a 5-point Likert-type scale with 1 = unhelpful, 2 = not very helpful, 3 = slightly helpful, 4 = helpful, 5 = very helpful. Only the positive responses (Options 4–5) are shown. The item “Will you suggest your friends to take this course?” is rated on a 4-point Likert-type scale with 1 = definitely will not suggest, 2 = will not suggest, 3 = will suggest, 4 = definitely will suggest. Only the positive responses (Options 3–4) are shown. The item “On the whole, are you satisfied with this course?” is rated on a 6-point Likert-type scale with 1 = very dissatisfied, 2 = moderately dissatisfied, 3 = dissatisfied, 4 = satisfied, 5 = moderately satisfied, 6 = very satisfied. Only the positive responses (Options 4–6) are shown.

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 11 of 17 Hierarchical multiple regression analyses showed that students’ perceived course design, teacher quality, and course benefits significantly predicted their perceived overall satisfaction with this subject after controlling the effects of gender, age, place of birth, and faculty (course design: β = 0.45, p < 0.001, Cohen’s f 2 = 0.098; teacher quality: β = 0.13, p < 0.01, Cohen’s f 2 = 0.020; course benefits: β = 0.27, p < 0.001; Cohen’s f 2 = 0.049), respectively. The findings provided support for Hypothesis 2. Details can be found in Table 4 . Table 4. Impact of perceived course design, lecturer quality and course benefits on overall satisfaction Predictors Overall Satisfaction Step 1 β t Cohen’s f 2 Gender 0.09 2.22 * 0.008 Age − 0.07 − 1.83 0.003 Place of Birth 0.27 6.87 *** 0.076 Faculty − 0.001 − 0.04 0.000 R 2 0.084 F 14.10 *** Step 2 β t Cohen’s f 2 Perceived Course Design 0.45 7.77 *** 0.098 Perceived Lecturer Quality 0.13 3.43 ** 0.020 Perceived Course Benefits 0.27 5.45 *** 0.049 R 2 0.693 R 2 Change 0.610 F 198.30 *** Note: * p < 0.05; ** p < 0.01; *** p < 0.001 3.3. Predictive Effects of Students’ Perceptions of the Subject on Their Objective Outcomes A two-step model based on hierarchical multiple regression was conducted to examine the relationship between subjective outcome evaluation and objective outcome evaluation. In Step 1, gender, age, place of birth, faculty, and students’ pre-test scores of PYD attributes were entered in the analyses as independent variables and students’ post-test PYD scores were entered as dependent variable. Results showed that place of birth and students’ pre-test PYD scores significantly predicted students’ post-test PYD scores (place of birth: β = 0.13, p < 0.01, Cohen’s f 2 = 0.019; pre-test PYD: β = 0.47, p < 0.001, Cohen’s f 2 = 0.271) In Step 2, students’ perceptions of course design, teacher quality, and course benefits were put in the analyses as independent variables to examine their predicting effects on students’ post-test PYD scores. Results showed that students’ perceptions of course design, teacher quality, and benefits were significantly related to their change in post-test scores in PYD attributes after the effects of gender, age, place of birth, faculty, and pre-test PYD attributes were controlled (course design: β = 0.21, p < 0.01, Cohen’s f 2 = 0.013; teacher quality: β = 0.12, p < 0.01, Cohen’s f 2 = 0.011; course benefits: β = 0.26, p < 0.001; Cohen’s f 2 = 0.028) The findings provided support for Hypothesis 3. Details can be found in Table 5 . Table 5. Predicting effect of perceived course design, lecturer, and course benefits on post-test PYD after controlling pre-test PYD and demographic factors Predictors Post-Test PYD Step 1 β t Cohen’s f 2 Gender 0.04 1.15 0.003 Age − 0.001 − 0.02 0.000 Place of Birth 0.13 3.46 ** 0.019 Faculty 0.02 0.55 0.000 Pre-test PYD 0.47 12.92 *** 0.271 R 2 0.267 F 44.85 ***

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 12 of 17 Table 5. Cont Predictors Post-Test PYD Step 2 β t Cohen’s f 2 Perceived Course Design 0.21 2.93 ** 0.013 Perceived Lecturer Quality 0.12 2.64 ** 0.011 Perceived Course Benefits 0.26 4.24 *** 0.028 R 2 0.528 R 2 Change 0.262 F 85.88 *** Note: ** p < 0.01; *** p < 0.001 4. Discussion With regard to the first research question, results showed that the participants showed significant positive changes in PYD qualities, wellbeing, and desired graduate attributes from pre-test to post-test, which supports Hypotheses 1 a, 1 b, and 1 c. The findings are in line with previous evaluation findings that students gained significant and positive changes in PYD qualities and life satisfaction after taking this subject [ 7 , 70 – 72 ]. Although the previous evaluation findings focused on the effectiveness of the subject in early years of the implementation, the present study indicated that the subject has sustained positive impact over time. It reinforces the thesis that this credit-bearing subject based on the PYD approach is effective in promoting the positive development and wellbeing of university students In addition, the findings advance our understanding of the effectiveness of this subject in promoting the well-being and desired graduate attributes of the students. This observation has implication for understanding the role of PYD programs in higher education. One significant challenge to the curricula in higher education is the difficulty in implementing effective general education program due to the over-division of knowledge and specialized disciplines [ 90 , 91 ]. Many general education programs were criticized as fragmented and piecemeal [ 92 ], which makes them incapable of fostering students’ holistic development The present study suggests that the subject based on the PYD approach is a promising strategy to promote students’ holistic development and desired graduate attributes In addition, the findings contribute to our limited understanding of the effects of hybrid mode of teaching in higher education, particularly during the COVID-19. Although hybrid mode of teaching has become a trend or widely adopted practice in higher education under COVID-19, there is limited empirical studies on this new teaching mode [ 59 ]. There are also inconclusive findings in the existing literature. Although some studies showed benefits of this mode of teaching [ 52 , 93 ], other studies suggested this teaching mode may have negative influence on students’ learning motivation and outcomes [ 94 , 95 ]. The findings of this study contribute to this body of literature by showing the benefits and positive impact of hybrid-mode teaching on students’ learning experience and outcomes in higher education Regarding the second research question, results showed that the participants had a high-level of satisfaction with their learning experience in the subject conducted in the hybrid mode, including their satisfaction with course design, teacher, and benefits of the subject. The findings support Hypothesis 2, and they are consistent with previous subjective outcome evaluation findings that students were generally satisfied with the course design and teaching staff and benefited from the subject after taking the course [ 73 , 74 , 88 ]. However, as the previous evaluation studies mainly focused on the subject based on the face-to-face mode, the present findings suggest that students also showed positive perceptions of the subject conducted in hybrid mode. This is also in line with a recent study of students’ perceptions of HyFlex mode TL subject based on post-lecture evaluation, showing that students had positive views towards the subject and perceived the subject beneficial to their positive development [ 51 ]. Similar support also emerged from hybrid teaching and provision of service under the context of service-learning [ 50 , 76 ].

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 13 of 17 Subjective outcome evaluation is a commonly used approach in program and curriculum evaluation. Different from objective outcome evaluation focusing on students’ changes in important indicators, subjective outcome evaluation focuses on different stakeholders’ perceptions and satisfaction with different aspects of programs and their subjective experience, which offers important information for course evaluation and further improvement [ 32 , 96 ]. One important rationale for subjective outcome evaluation is that program participants should “have their opinions heard” as important information for program improvement [ 97 ] (p. 7). The present study showed students were highly satisfied with all the three aspects of the subject delivered via the hybrid mode Regarding the third research question, results showed that the participants’ satisfaction with their learning experience positively predicted their post-test PYD scores in objective outcome evaluation after the effect of demographic factors and pre-test PYD scores were controlled, hence supporting Hypothesis 3. This observation contributes to the scholarship on evaluation methodology. Argued by Shek [ 64 ], it is significant and meaningful to examine the association between objective outcome evaluation and subjective outcome evaluation because the use of subjective outcome evaluation would be meaningless if it is unrelated to intervention effects. The result sheds light on the disputes and inconclusive findings on relationship between objective and subjective outcome evaluation in existing literature. Although some studies showed a significant positive relationship between objective and subjective outcome evaluation [ 63 , 98 , 99 ], other studies showed no significant association between the two evaluation measures and outcomes [ 61 , 62 ]. One possible reason might be due to the subjectivity of subjective outcome evaluation [ 100 ]. Zhu and Shek [ 100 ] argued that the objectivity of subjective outcome evaluation could be increased through valid and reliable measures. Several studies based on multidimensional and validated measures of subjective outcome evaluation also showed significant association between subjective and objective outcome measures [ 64 , 65 , 100 ]. Although the present study included both objective and subjective outcome evaluation strategies to gain a more comprehensive understanding of the effects of the subject conducted via hybrid mode, several limitations should be noted. First, the study adopted one group pre-test and post-test design in objective outcome evaluation, which may have bias arising from the absence of a control group, since the participants’ improvement may be due to other factors, such as maturation, but not the participation in the subject [ 81 ]. Therefore, future research based on quasi-experimental design should be conducted to strengthen the present findings. Second, while the present study adopted objective and subjective outcome evaluation, other evaluation methods might also need to be involved, such as qualitative evaluation to triangulate the data and provide deeper understanding of underlying mechanisms for subject effectiveness. Despite of the limitations, these findings are important because COVID-19 has substantially changed the teaching and learning mode under COVID-19 [ 101 , 102 ]. In conjunction with recent studies reporting the value and benefits of online teaching and learning [ 10 , 48 , 49 , 103 , 104 ], the present study sheds light on the impacts and effectiveness of PYD programs conducted via a Hybrid mode in higher education as an important approach for promoting university students’ positive development, wellbeing, and holistic development 5. Conclusions There are very few studies studying the effectiveness of hybrid teaching, especially in the Chinese context under COVID-19. This study examined the effectiveness of a leadership subject utilizing positive youth development principles delivered via a hybrid teaching mode. After taking the subject, students showed positive changes and they had positive perceptions of the subject content, instructors, and benefits. We also found that students’ satisfaction with the subject predicted post-test measures of well-being and desired graduate attributes.

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 14 of 17 Author Contributions: Conceptualization, W.C. and D.T.L.S.; methodology, W.C., X.L. and D.T.L.S.; formal analysis, W.C. and D.T.L.S.; writing—original draft preparation, W.C., X.L. and D.T.L.S.; writing—review and editing, W.C., X.L. and D.T.L.S.; supervision, D.T.L.S.; project administration, D.T.L.S.; funding acquisition, D.T.L.S. All authors have read and agreed to the published version of the manuscript Funding: The subject “Tomorrow’s Leaders” and related research are financially supported by the Department of Applied Social Sciences, The Hong Kong Polytechnic University, Li and Fung Endowed Professorship and UGC Research Matching Fund (#5-ZH 4 Q) Institutional Review Board Statement: The study was conducted according to the guidelines of the Declaration of Helsinki, and the related studies were approved by the Ethics Committee of The Hong Kong Polytechnic University (protocol code: HSEARS 20130204003; February 2013) Informed Consent Statement: Informed consent was obtained from all subjects involved in the study Data Availability Statement: The data presented in this study are available on request from the corresponding author Conflicts of Interest: The authors declare no conflict of interest References 1 Chan, C.K.; Fong, E.T.; Luk, L.Y.; Ho, R. A review of literature on challenges in the development and implementation of generic competencies in higher education curriculum Int. J. Educ. Dev 2017 , 57 , 1–10. [ CrossRef ] 2 Regueiro, B.; Rodr í guez-Fern á ndez, J.E.; Crespo, J.; Pino-Juste, M.R. Design and validation of a questionnaire for university students’ generic competencies (COMGAU) Front. Educ. Front 2021 , 6 , 606216. [ CrossRef ] 3 Macaskill, A. The mental health of university students in the United Kingdom Br. J. Guid. Couns 2013 , 41 , 426–441. [ CrossRef ] 4 Clinciu, A.I. Adaptation and stress for the first year university students Procedia Soc. Behav. Sci 2013 , 78 , 718–722. [ CrossRef ] 5 Hamaideh, S.H. Stressors and reactions to stressors among university students Int. J. Soc. Psychiatry 2011 , 57 , 69–80. [ CrossRef ] 6 Mudhovozi, P. Social and academic adjustment of first-year university students J. Soc. Sci 2012 , 33 , 251–259. [ CrossRef ] 7 Li, X.; Shek, D.T.L. Objective outcome evaluation of a leadership course utilizing the positive youth development approach in Hong Kong Assess. Eval. High. Educ 2020 , 45 , 741–757. [ CrossRef ] 8 Ibrahim, A.K.; Kelly, S.J.; Adams, C.E.; Glazebrook, C. A systematic review of studies of depression prevalence in university students J. Psychiatr. Res 2013 , 47 , 391–400. [ CrossRef ] 9 Stallman, H.M. Psychological distress in university students: A comparison with general population data Aust. Psychol 2010 , 45 , 249–257. [ CrossRef ] 10 Shek, D.T.L.; Yu, L.; Lin, L.; Li, X.; Zhu, X.Q.; Dou, D.; Chai, W.Y.; Chak, Y.; Ho, W.; Leung, E.; et al. Nurturing leadership qualities under COVID-19: Student perceptions of the qualities and effectiveness of online teaching and learning on leadership development Int. J. Child Adolesc. Health 2021 , 14 , 89–100 11 Lo, S.M.; Wong, H.C.; Lam, C.Y.; Shek, D.T.L. An innovative multidisciplinary healthcare model in student mental health: Experience in Hong Kong Appl. Res. Qual. Life 2021 , 16 , 301–313. [ CrossRef ] 12 Matar Boumosleh, J.; Jaalouk, D. Depression, anxiety, and smartphone addiction in university students-A cross sectional study PLoS ONE 2017 , 12 , e 0182239. [ CrossRef ] 13 Serin, N.B.; Serin, O.; Özba¸s, L.F. Predicting university students’ life satisfaction by their anxiety and depression level Procedia Soc. Behav. Sci 2010 , 9 , 579–582. [ CrossRef ] 14 Sharp, J.; Theiler, S. A review of psychological distress among university students: Pervasiveness, implications and potential points of intervention Int. J. Adv. Couns 2018 , 40 , 193–212. [ CrossRef ] 15 Elias, H.; Noordin, N.; Mahyuddin, R.H. Achievement motivation and self-efficacy in relation to adjustment among university students J. Soc. Sci 2010 , 6 , 333–339 16 Sagone, E.; De Caroli, M.E. A correlational study on dispositional resilience, psychological well-being, and coping strategies in university students Am. J. Educ. Res 2014 , 2 , 463–471. [ CrossRef ] 17 Kay, K.; Greenhill, V. Twenty-first century students need 21 st century skills. In Bringing Schools into the 21 st Century ; Springer: Dordrecht, The Netherlands, 2011; pp. 41–65 18 Bellanca, J.A 21 st Century Skills: Rethinking How Students Learn ; Solution Tree Press: Bloomington, Indiana, 2010 19 Singh, H.; Gera, M. Generic skills for sustainable development Indian J. Res 2015 , 4 , 290–292 20 Cornalli, F. Training and developing soft skills in higher education. In Proceedings of the 4 th International Conference on Higher Education Advances HEAD 2018, Polytechnic University of Valencia, Valencia, Spain, 20–22 June 2018; pp. 961–967. [ CrossRef ] 21 Kennedy, T.J.; Sundberg, C.W. 21 st century skills. In Science Education in Theory and Practice ; Springer: Cham, Switzerland, 2020; pp. 479–496 22 Benson, P.L.; Scales, P.C.; Syvertsen, A.K. The contribution of the developmental assets framework to positive youth development theory and practice Adv. Child Dev. Behav 2011 , 41 , 197–230.

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 15 of 17 23 Catalano, R.F.; Berglund, M.L.; Ryan, J.A.; Lonczak, H.S.; Hawkins, J.D. Positive youth development in the United States: Research findings on evaluations of positive youth development programs Ann. Am. Acad. Polit. Soc. Sci 2004 , 591 , 98–124. [ CrossRef ] 24 Lerner, R.M. The positive youth development perspective: Theoretical and empirical bases of a strength-based approach to adolescent development. In The Oxford Handbook of Positive Psychology , 2 nd ed.; Snyder, C.R., Lopez, S.J., Eds.; Oxford University Press: Oxford, UK, 2009; pp. 149–163 25 Lerner, R.M.; Lerner, J.V.; Lewin-Bizan, S.; Bowers, E.P.; Boyd, M.J.; Mueller, M.K.; Schmid, K.L.; Napolitano, C.M. Positive youth development: Processes, programs, and problematics J. Youth Dev 2011 , 6 , 38–62. [ CrossRef ] 26 Scales, P.C.; Benson, P.L.; Leffert, N.; Blyth, D.A. Contribution of developmental assets to the prediction of thriving among adolescents Appl. Dev. Sci 2000 , 4 , 27–46. [ CrossRef ] 27 Shek, D.T.L.; Dou, D.; Zhu, X.; Chai, W. Positive youth development: Current perspectives Adolesc. Health Med. Ther 2019 , 10 , 131–141. [ CrossRef ] 28 Leung, C.L.; Bender, M.; Kwok, S.Y. A comparison of positive youth development against depression and suicidal ideation in youth from Hong Kong and the Netherlands Int. J. Adolesc. Med. Health 2020 , 32 , 20170105. [ CrossRef ] 29 Zhou, Z.; Shek, D.T.L.; Zhu, X. The importance of positive youth development attributes to life satisfaction and hopelessness in mainland Chinese adolescents Front. Psychol 2020 , 11 , 553313. [ CrossRef ] 30 Zhou, Z.; Shek, D.T.L.; Zhu, X.; Dou, D. Positive youth development and adolescent depression: A longitudinal study based on mainland Chinese high school students Int. J. Environ. Res. Public Health 2020 , 17 , 4457. [ CrossRef ] 31 Dou, D.; Shek, D.T.L. Concurrent and longitudinal relationships between positive youth development attributes and adolescent Internet addiction symptoms in Chinese Mainland high school students Int. J. Environ. Res. Public Health 2021 , 18 , 1937 [ CrossRef ] 32 Zhu, X.; Shek, D.T.L. Impact of a positive youth development program on junior high school students in mainland China: A pioneer study Child. Youth Serv. Rev 2020 , 114 , 105022. [ CrossRef ] 33 Abdul Kadir, N.B.; Mohd, R.H. The 5 Cs of positive youth development, purpose in life, hope, and well-being among emerging adults in Malaysia Front. Psychol 2021 , 12 , 641876. [ CrossRef ] 34 Shek, D.T.L.; Chai, W. The impact of positive youth development attributes and life satisfaction on academic well-being: A longitudinal mediation study Front. Psychol 2020 , 11 , 2126. [ CrossRef ] 35 Sanders, J.; Munford, R.; Thimasarn-Anwar, T.; Liebenberg, L.; Ungar, M. The role of positive youth development practices in building resilience and enhancing wellbeing for at-risk youth Child Abus. Negl 2015 , 42 , 40–53. [ CrossRef ] 36 Sun, R.C.F.; Shek, D.T.L. Life satisfaction, positive youth development, and problem behaviour among Chinese adolescents in Hong Kong Soc. Indic. Res 2010 , 95 , 455–474. [ CrossRef ] [ PubMed ] 37 Sun, R.C.F.; Shek, D.T.L. Longitudinal influences of positive youth development and life satisfaction on problem behaviour among adolescents in Hong Kong Soc. Indic. Res 2013 , 114 , 1171–1197. [ CrossRef ] 38 Jelicic, H.; Bobek, D.L.; Phelps, E.; Lerner, R.M.; Lerner, J.V. Using positive youth development to predict contribution and risk behaviors in early adolescence: Findings from the first two waves of the 4-H study of positive youth development Int. J. Behav Dev 2007 , 31 , 263–273. [ CrossRef ] 39 Lerner, R.M Liberty: Thriving and Civic Engagement among America’s Youth ; SAGE Publications Inc.: Thousand Oaks, CA, USA, 2004 40 Lerner, R.M.; Dowling, E.M.; Anderson, P.M. Positive youth development: Thriving as the basis of personhood and civil society. In Beyond the Self ; Routledge: Oxfordshire, UK, 2019; pp. 172–180 41 Curran, T.; Wexler, L. School-based positive youth development: A systematic review of the literature Int. J. Sch. Health 2017 , 87 , 71–80. [ CrossRef ] [ PubMed ] 42 Weiss, M.R.; Stuntz, C.P.; Bhalla, J.A.; Bolter, N.D.; Price, M.S. ‘More than a game’: Impact of The First Tee life skills programme on positive youth development: Project introduction and Year 1 findings Qual. Res. Sport Exerc. Health 2013 , 5 , 214–244. [ CrossRef ] 43 Taylor, R.D.; Oberle, E.; Durlak, J.A.; Weissberg, R.P. Promoting positive youth development through school-based social and emotional learning interventions: A meta-analysis of follow-up effects Child Dev 2017 , 88 , 1156–1171. [ CrossRef ] 44 Ciocanel, O.; Power, K.; Eriksen, A.; Gillings, K. Effectiveness of positive youth development interventions: A meta-analysis of randomized controlled trials J. Youth Adolesc 2017 , 46 , 483–504. [ CrossRef ] 45 Ma, C.M.S.; Shek, D.T.L. Objective outcome evaluation of a positive youth development program: The Project PATHS in Hong Kong Res. Soc. Work Pract 2019 , 29 , 49–60. [ CrossRef ] 46 Qi, S.; Hua, F.; Zhou, Z.; Shek, D.T.L. Trends of positive youth development publications (1995–2020): A scientometric review Appl. Res. Qual. Life 2022 , 17 , 421–446. [ CrossRef ] 47 Shek, D.T.L.; Yu, L. A review of validated youth prevention and positive youth development programs in Asia Int. J. Adolesc Med. Health 2011 , 23 , 317–324. [ CrossRef ] 48 Lin, L.; Shek, D.T.L. Serving children and adolescents in need during the COVID-19 pandemic: Evaluation of service-learning subjects with and without face-to-face interaction Int. J. Environ. Res. Public Health 2021 , 18 , 2114. [ CrossRef ] 49 Leung, H.; Shek, D.T.L.; Dou, D. Evaluation of service-learning in Project Wecan under COVID-19 in a Chinese context Int. J Environ. Res. Public. Health 2021 , 18 , 3596. [ CrossRef ] 50 Shek, D.T.L.; Li, X.; Yu, L.; Lin, L.; Chen, Y. Evaluation of electronic service-learning (e-service-learning) projects in mainland China under COVID-19 Appl. Res. Qual. Life 2022 , 1–24. [ CrossRef ]

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 16 of 17 51 Shek, D.T.L.; Zhu, X.; Li, X.; Dou, D. Satisfaction with HyFlex teaching and law-abiding leadership education in Hong Kong university students under COVID-19 Appl. Res. Qual. Life 2022 , 1–26. [ CrossRef ] 52 Kohnke, L.; Moorhouse, B.L. Adopting HyFlex in higher education in response to COVID-19: Students’ perspectives Open Learn J. Open Distance e-Learn 2021 , 36 , 231–244. [ CrossRef ] 53 Rashid, S.; Yadav, S.S. Impact of COVID-19 pandemic on higher education and research Indian J. Hum. Dev 2020 , 14 , 340–343 [ CrossRef ] 54 Bashir, A.; Bashir, S.; Rana, K.; Lambert, P.; Vernallis, A. Post-COVID-19 adaptations; the shifts towards online learning, hybrid course delivery and the implications for biosciences courses in the higher education setting Front. Educ 2021 , 6 , 711619 [ CrossRef ] 55 Beatty, B.J. Teaching a Hybrid-Flexible Course. Hybrid-Flexible Course Design. 2019. Available online: https://edtechbooks.org/ hyflex/teaching_hyflex (accessed on 8 April 2022) 56 Nassr, R.M.; Aborujilah, A.; Aldossary, D.A.; Aldossary, A.A.A. Understanding education difficulty during COVID-19 lockdown: Reports on Malaysian university students’ experience IEEE Access 2020 , 8 , 186939–186950. [ CrossRef ] 57 Gonzalez, T.; de la Rubia, M.A.; Hincz, K.P.; Comas-Lopez, M.; Subirats, L.; Fort, S.; Sacha, G.M. Influence of COVID-19 confinement in students performance in higher education PLoS ONE 2020 , 15 , e 0239490. [ CrossRef ] 58 Ming, T.S.; King, L.Y.; Tuen, Y.N.L.; Yee, C.M Evaluation Study on The Women’s Foundation’s Financial Literacy Programme for Marginalised Women ; Department of Social Work, The Chinese University of Hong Kong: Hong Kong, China, 2014; Available online: https://twfhk.org/sites/default/files/u 21/CUHK_final_report.pdf (accessed on 10 April 2022) 59 Tse, S.K.; To, H.; Pang, E.Y.W.; Tsui, P.F.; Lam, L.S.; Laing, V.; Wu, K.F. Using the home-school-community model to enhance the Chinese language proficiency of non-Chinese speaking children Creat. Educ 2020 , 11 , 2069–2095. [ CrossRef ] 60 Gauthier, B. Client satisfaction in program evaluation Soc. Indic. Res 1987 , 19 , 229–254. [ CrossRef ] 61 El Ansari, W.; Stock, C. Is the health and wellbeing of university students associated with their academic performance? Cross sectional findings from the United Kingdom Int. J. Environ. Res. Public Health 2010 , 7 , 509–527. [ CrossRef ] 62 Walsh, T.; Lord, B. Client satisfaction and empowerment through social work intervention Soc. Work Public Health 2004 , 38 , 37–56 [ CrossRef ] [ PubMed ] 63 Shek, D.T.L. Subjective outcome and objective outcome evaluation findings: Insights from a Chinese context Res. Soc. Work Pract 2010 , 20 , 293–301. [ CrossRef ] 64 Shek, D.T.L. Is subjective outcome evaluation related to objective outcome evaluation? Insights from a longitudinal study in Hong Kong J. Pediatr. Adolesc. Gynecol 2014 , 27 , S 50–S 56. [ CrossRef ] [ PubMed ] 65 Shek, D.T.L.; Lee, T.Y.; Siu, A.M.H.; Ma, H.K. Convergence of subjective outcome and objective outcome evaluation findings: Insights based on the Project P.A.T.H.S Sci. World J 2007 , 7 , 258–267. [ CrossRef ] 66 Lun, K.W.C.; Chan, C.K.; Ip, P.K.Y.; Ma, S.Y.K.; Tsai, W.W.; Wong, C.S.; Wong, C.H.T.; Wong, T.W.; Yan, D. Depression and anxiety among university students in Hong Kong Hong Kong Med. J 2018 , 24 , 466–472. [ CrossRef ] 67 Li, L.; Lok, G.K.I.; Mei, S.-L.; Cui, X.-L.; An, F.-R.; Li, L.; Cheung, T.; Ungvari, G.S.; Xiang, Y.-T. Prevalence of depression and its relationship with quality of life among university students in Macau, Hong Kong and mainland China Sci. Rep 2020 , 10 , 15798 [ CrossRef ] 68 Shek, D.T.L.; Chung, P.P.Y.; Leung, H. Manufacturing economy vs. service economy: Implications for service leadership Int. J Disabil. Hum. Dev 2015 , 14 , 205–215. [ CrossRef ] 69 Zhu, X.; Shek, D.T.L. Promoting service leadership qualities and well-being among university students: Evaluation findings in Hong Kong Appl. Res. Qual. Life 2021 , 16 , 2475–2495. [ CrossRef ] 70 Shek, D.T.L.; Sun, R.C.F.; Tsien-Wong, T.B.K.; Cheng, C.T.; Yan, H.R. Objective outcome evaluation of a leadership and intrapersonal development subject for university students Int. J. Disabil. Hum. Dev 2013 , 12 , 221–227. [ CrossRef ] 71 Shek, D.T.L.; Ma, C.M.S. Do university students change after taking a subject on leadership and intrapersonal development? Int J. Disabil. Hum. Dev 2014 , 13 , 451–456. [ CrossRef ] 72 Shek, D.T.L.; Sun, R.C.F. Promoting psychosocial competencies in university students: Evaluation based on a one-group pretest/post-test design Int. J. Disabil. Hum. Dev 2012 , 11 , 229–234. [ CrossRef ] 73 Shek, D.T.L.; Yu, L. Post-course subjective outcome evaluation of a subject on leadership and intrapersonal development for university students in Hong Kong Int. J. Disabil. Hum. Dev 2014 , 13 , 457–464. [ CrossRef ] 74 Yu, L.; Lin, L.; Shek, D.T.L.; Chai, W. Students’ perceived attributes and benefits of a leadership course: Subjective outcome evaluation Res. Soc. Work Pract 2022 , 32 , 199–214. [ CrossRef ] 75 Noda, T.; Sugiyama, N.; Ito, H.; Soininen, P.; Putkonen, H.; Sailas, E.; Joffe, G. Secluded/restrained patients’ perceptions of their treatment: Validation and reliability of a new questionnaire Psychiatry Clin. Neurosci 2012 , 66 , 397–404. [ CrossRef ] 76 Lin, L.; Shek, D.T.L. Who benefit and appreciate more? An evaluation of online service-learning projects in Mainland China during the COVID-19 pandemic Appl. Res. Qual. Life 2022 , 1–22. [ CrossRef ] 77 Gil, M.; Kim, S. Feasibility and preliminary efficacy of a new online self-help intervention for depression among Korean college students’ families Int. J. Environ. Res. Public Health 2022 , 19 , 2142. [ CrossRef ] 78 Chan, C.; Holosko, M.J. Utilizing youth media practice to influence change: A pretest-posttest study Res. Soc. Work Pract 2020 , 30 , 110–121. [ CrossRef ]

Int. J. Environ. Res. Public Health 2022 , 19 , 9809 17 of 17 79 Son, H. The effects of simulation problem-based learning on the empathy, attitudes toward caring for the elderly, and team efficacy of undergraduate health profession students Int. J. Environ. Res. Public Health 2021 , 18 , 9658. [ CrossRef ] 80 Cranmer, G.A. One-group pretest-posttest design. In The Sage Encyclopedia of Communication Research Methods ; Sage Publication: Thousand Oaks, CA, USA, 2017; pp. 1125–1126 81 Knapp, T.R. Why is the one-group pretest-posttest design still used? Clin. Nurs. Res 2016 , 25 , 467–472. [ CrossRef ] 82 Kurki, M.; Gilbert, S.; Mishina, K.; Lempinen, L.; Luntamo, T.; Hinkka-Yli-Salomaki, S.; Sinokki, A.; Upadhyaya, S.; Wei, Y.; Sourander, A. Digital mental health literacy-program for the first-year medical students’ wellbeing: A one group quasiexperimental study BMC Med. Educ 2021 , 21 , 563 83 Siu, O.L.; Cooper, C.L.; Phillips, D.R. Intervention studies on enhancing work well-being, reducing burnout, and improving recovery experiences among Hong Kong health care workers and teachers Int. J. Stress Manag 2014 , 21 , 69–84. [ CrossRef ] 84 Shek, D.T.L.; Siu, A.M.H.; Lee, T.Y.; Cheng, H.; Tsang, S.; Chui, Y.H.; Lui, J.; Lung, D. Development and validation of a positive youth development scale in Hong Kong Int. J. Adolesc. Med. Health 2006 , 18 , 547–558. [ CrossRef ] 85 Shek, D.T.L.; Ma, C.M.S. Dimensionality of the Chinese positive youth development scale: Confirmatory factor analyses Soc Indic. Res 2010 , 98 , 41–59. [ CrossRef ] 86 Diener, E.; Emmons, R.A.; Larsen, R.J.; Griffin, S. The satisfaction with life scale J. Pers. Assess 1985 , 49 , 71–75. [ CrossRef ] 87 Pavot, W.; Diener, E.; Colvin, C.R.; Sandvik, E. Further validation of the satisfaction with life scale: Evidence for the cross-method convergence of well-being measures J. Pers. Assess 1991 , 57 , 149–161. [ CrossRef ] 88 Shek, D.T.L.; Sun, R.C.F. Post-course subjective outcome evaluation of a course promoting leadership and intrapersonal development in university students in Hong Kong Int. J. Disabil. Hum. Dev 2013 , 12 , 193–201. [ CrossRef ] 89 Liu, X.; Zhao, L.; Su, Y. Impact of parents’ attitudes on learning ineffectiveness: The mediating role of parental self-efficacy Int. J Environ. Res. Public Health 2022 , 19 , 615. [ CrossRef ] 90 Newton, R.R. Tensions and models in general education planning J. Gen. Educ 2000 , 49 , 165–181. [ CrossRef ] 91 Jaffee, D. The general education initiative in Hong Kong: Organized contradictions and emerging tensions High. Educ 2012 , 64 , 193–206. [ CrossRef ] 92 Boning, K. Coherence in general education: A historical look J. Gen. Educ 2007 , 56 , 1–16 93 Austin, A.; Rudolf, F.; Fernandez, J.; Ishimine, P.; Murray, M.; Suresh, P.; McDaniel, M.; Shishlov, K.; Oyama, L. COVID-19 educational innovation: Hybrid in-person and virtual simulation for emergency medicine trainees AEM Educ. Train 2021 , 5 , e 10593. [ CrossRef ] [ PubMed ] 94 Meltzer, L.J.; Saletin, J.M.; Honaker, S.M.; Owens, J.A.; Seixas, A.; Wahlstrom, K.L.; Wolfson, A.R.; Wong, P.; Carskadon, M.A. COVID-19 instructional approaches (in-person, online, hybrid), school start times, and sleep in over 5000 U.S. adolescents Sleep 2021 , 44 , zsab 180. [ CrossRef ] 95 Raes, A.; Vanneste, P.; Pieters, M.; Windey, I.; Van Den Noortgate, W.; Depaepe, F. Learning and instruction in the hybrid virtual classroom: An investigation of students’ engagement and the effect of quizzes Comput. Educ 2020 , 143 , 103682. [ CrossRef ] 96 Maccio, E.M.; Voorhies, R.A. Social work students’ perceptions of service-learning J. Ser.-Learn. High. Educ 2012 , 1 , 50–69 97 Applebaum, R.; Straker, J.; Geron, S Assessing Satisfaction in Health and Long-Term Care: Practical Approaches to Hearing the Voices of Consumers ; Springer Publishing Company: New York, NY, USA, 1999 98 Abrams, P.; Artibani, W.; Gajewski, J.B.; Hussain, I. Assessment of treatment outcomes in patients with overactive bladder: Importance of objective and subjective measures Urology 2006 , 68 (Suppl. S 2), 17–28. [ CrossRef ] 99 Edwards, S.G.M.; Playford, E.D.; Hobart, J.C.; Thompson, A.J. Comparison of physician outcome measures and patients’ perception of benefits of inpatient neurorehabilitation BMJ 2002 , 324 , 1493. [ CrossRef ] 100. Zhu, X.; Shek, D.T.L.; Chan, C.H.M. Promoting service leadership qualities and well-being among university students through an online course during COVID-19 pandemic Int. J. Environ. Res. Public Health 2021 , 18 , 8162. [ CrossRef ] 101. Pokhrel, S.; Chhetri, R. A literature review on impact of COVID-19 pandemic on teaching and learning High. Educ. Future 2021 , 8 , 133–141. [ CrossRef ] 102. Shek, D.T.L. COVID-19 and quality of life: Twelve reflections Appl. Res. Qual. Life 2021 , 16 , 1–11. [ CrossRef ] 103. Shek, D.T.L.; Chak, Y.; Li, X.; Leung, E.; Li, P.; Mok, B.; Shek, V. Service-learning under COVID-19: Experience gained from Project WeCan in Hong Kong Int. J. Child. Adolesc. Health 2021 , 14 , 155–164 104. Li, X.; Shek, D.T.L.; Shek, E.Y. Psychological morbidity among university students in Hong Kong (2014–2018): Psychometric properties of the depression anxiety stress scales (DASS) and related correlates Int. J. Environ 2021 , 18 , 8305. [ CrossRef ]

Other Environmental Sciences Concepts:

[back to top]

Discover the significance of concepts within the article: ‘The Effectiveness of a Leadership Subject Using a Hybrid Teaching Mode during...’. Further sources in the context of Environmental Sciences might help you critically compare this page with similair documents:

Well-being, Mental health, Primary factor, Critical thinking, Higher education, Self-determination, General education, Thriving, Holistic development, Quality of life, Objective outcome, Sense of responsibility, Leadership, Civil society, Lifelong learning, Longitudinal study, Chinese University, Positive change, Life satisfaction, Problem solving, Academic performance, Learning experience, Statistical Significance, Self efficacy, COVID 19, COVID-19 Pandemic, Post-test design, Psychological distress, Psychometric properties, Objective Evaluation, Internal Consistency, Positive identity, Research gap, Wellbeing, One Group Pretest-Posttest Design, Online Learning, Client satisfaction, Internal validity, Hybrid mode, Multivariate analysis of variance, Service learning, Mental health problem, Resilience, Psychosocial competencies, Student perception, Overall satisfaction, Satisfaction with Life Scale, Risk behavior, Ethical Leadership, Positive perception, Learning outcome, Problem behaviour, Empirical findings, Hong Kong, Learning motivation, Online teaching, Positive youth development, Problem behavior, Adolescent depression, Social and emotional learning, School students, Chinese adolescents, Online course, University student, Mainland China, Asian context, Overall development, High school student, Internal challenges, Graduate attributes, Youth development, Higher education sector, Ethical decision, Positive evaluation, Course design, Subjective outcome, Academic wellbeing, Emotional competence, Subjective wellbeing, Positive social behavior, Assessment strategies, Effective leaders, External challenges, Chinese context, Moral competence, Leadership program, Social emotional learning, Positive development, Cognitive competence, Predictive effect, Teacher quality, Interpersonal competence, Internal assets, Social emotional competence, Prosocial Norms, Service Economy, Developmental Assets, Students' Learning Experience, Positive Ratings, Hierarchical Multiple Regression, Internet addiction symptoms, Academic well-being, At-risk youth, Service leadership, Positive Youth, Youth Development Program, Positive Youth Development Program, Positive Youth Development Practices.