Education for Sustainable Development in Higher Education Rankings

[Full title: Education for Sustainable Development in Higher Education Rankings: Challenges and Opportunities for Developing Internationally Comparable Indicators]

[Find the meaning and references behind the names: Ran, Deal, Doi, Ranking, Basel, Local, Key, Lili, Ann, Quest, Show, Maps, Development, Tool, February, Multi, Pace, Future, Data, Journey, Under, Campus, Iii, Lack, Far, Open, Manner, Place, Cai, Progress, Area, Wolff, Take, Green, April, Still, Author, Strong, Good]

Citation: Veidemane, A. Education for Sustainable Development in Higher Education Rankings: Challenges and Opportunities for Developing Internationally Comparable Indicators Sustainability 2022 , 14 , 5102. https://doi.org/ 10.3390/su 14095102 Academic Editors: Yuzhuo Cai and Lili-Ann Wolff Received: 17 February 2022 Accepted: 20 April 2022 Published: 23 April 2022 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations Copyright: © 2022 by the author Licensee MDPI, Basel, Switzerland This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/) sustainability Article Education for Sustainable Development in Higher Education Rankings: Challenges and Opportunities for Developing Internationally Comparable Indicators Anete Veidemane Center for Higher Education Policy Studies, University of Twente, 7500 AE Enschede, The Netherlands; a.veidemane@utwente.nl Abstract: As more higher education institutions strive to embed sustainable development principles in their teaching, it becomes increasingly important to identify indicators that can measure institutional contribution in a meaningful and internationally comparable manner. This paper shows that existing sustainability rankings, such as the UI Green Metric and THE Impact ranking, have paid relatively little attention to indicators on Education for Sustainable Development (ESD). In a quest to develop such indicators for U-Multirank—the multi-dimensional transparency and ranking tool—we reviewed the literature, consulted experts, and ran a survey amongst practitioners. This article summarises opportunities and challenges for developing internationally comparable ESD indicators in the higher education sector, discussing indicator relevance, validity, and feasibility. The results suggest that (i) ESD indicators are considered highly relevant by diverse stakeholders; (ii) the majority of HEIs surveyed are planning to collect ESD data within 3 years, signalling good prospects for data feasibility; (iii) the ESD indicators proposed so far still lack criteria that would allow one to sufficiently identify and compare these indicators across countries, inhibiting indicator validity At least three potential definitions are used by HEIs. The results of this paper can contribute to the discussion on identifying appropriate criteria for the development of ESD indicators and their use in international rankings Keywords: education for sustainable development; ESD; sustainable development; SDGs; sustainability; higher education; indicators; rankings; assessment; university rankings; green universities; green campus; education; learning 1. Introduction Problem Statement and Rationale for ESD Indicators Responding to urgent policy priorities [ 1 – 3 ] and societal concerns [ 4 – 6 ], a steeply growing number of higher education institutions (HEIs) have embarked on a journey to embed sustainable development in their key functions. For example, in 2021, more than 1100 institutions from 94 countries participated in the Times Higher Education (THE) Impact Ranking to show their contributions to the Sustainable Development Goals (SDGs) [ 7 ]. However, the pace at which sustainability transformations take place at HEIs varies considerably depending on the local context and capacity of each institution [ 8 ]. Internationally comparable yet locally meaningful indicators can support HEIs in learning from each other and assessing their progress While transformation towards sustainability in the higher education (HE) sector requires a systematic approach across all functions and the HE ecosystem [ 9 ], this article zooms in on one promising area—education and, particularly, Education for Sustainable Development (ESD). Education prepares (or underprepares) today’s students to be the decision makers of tomorrow, who must deal with the current and future sustainability challenges [ 10 , 11 ]. ESD supports students in developing critical competencies, such as systems thinking and integrated problem solving to address these challenges [ 12 , 13 ]. Sustainability 2022 , 14 , 5102. https://doi.org/10.3390/su 14095102 https://www.mdpi.com/journal/sustainability

[Find the meaning and references behind the names: Element, Meet, Own, Better, Body, Fields, Present, Base, Time, Beyond, Novel, Part, Ability, Year, Member, Council, Given, Knowledge, Focus]

Sustainability 2022 , 14 , 5102 2 of 20 A substantial body of literature on sustainability indicators already exists [ 14 – 20 ], yet widespread use of indicators is hindered by uncoordinated practices and a lack of consensus on appropriate frameworks, methodologies, and impact assessments [ 19 , 21 ]. Numerous researchers have proposed that stronger harmonisation is needed at a global scale [ 21 – 24 ]. Cooperative action is required to monitor and achieve goals and reduce inefficiencies. Moreover, a widely shared and accepted knowledge base may better inform sustainability policies and create synergies across global efforts [ 24 ]. At the same time, the indicators must remain relevant for the local context [ 16 , 25 ]. The objective of this article is to explore promising ESD indicators for the HE sector by learning from existing rankings, such as the UI Green Metric and THE Impact ranking This paper aims to address three research questions. First, it explores the extent to which ESD-related indicators are available in the existing higher education sustainability rankings. Second, it identifies challenges for developing internationally comparable ESD indicators. Lastly, it reviews and proposes the most promising ESD indicators against three commonly found criteria in the literature on indicator assessment—relevance, validity, and feasibility [ 26 – 28 ]. The criteria are discussed in detail in Section 3.7 . The following questions are proposed: • RQ 1: To what extent are ESD-related indicators available in the existing higher education sustainability rankings? • RQ 2: What are the challenges for developing internationally comparable ESD indicators in the higher education sector? • RQ 3: What are the most promising ESD indicators in the higher education sector, assessed against relevance, validity, and feasibility? This research was undertaken as part of the U-Multirank project. U-Multirank is a multi-dimensional ranking and transparency tool featuring over 1900 HEIs [ 29 ], aiming to provide relevant and user-driven comparisons [ 30 ]. Recognising that the needs and priorities of U-Multirank participants are shifting, in 2019, a stakeholder consultation process was started to identify promising ESD indicators to include in U-Multirank. The author of this article is a member of the U-Multirank team for identifying and proposing the indicators 2. Relevance of Sustainability in Higher Education 2.1. ESD in Global and European Policy Priorities To determine the relevance of sustainable development indicators, we first reviewed recent policy documents. Sustainable development in policy discourse is commonly defined as “the ability to [ . . . ] meet[s] the needs of the present without compromising the ability of future generations to meet their own needs” [ 31 ] (p. 16). Over the years, the concept has evolved to reflect three dimensions of sustainable development—environmental, social, and economic [ 1 ]. Although highly relevant today, sustainable development is not a novel policy priority. Already in 1987, an urgent call to action was made “to propose longterm environmental strategies for achieving sustainable development by the year 2000 and beyond” [ 31 ] (p. 5). More recently, the Agenda 2030 for Sustainable Development placed these aspirations in clear focus with 17 Sustainable Development Goals (SDGs) [ 1 ], mobilising stakeholders from all regions and diverse fields. As an essential element of transformations towards sustainable development, ESD is gaining increased recognition globally [ 32 ]. The European higher education policy documents and initiatives signal a clear priority given to sustainable development for the upcoming decade (2020–2030). Within the European Union (EU), education is seen as a critical component for building the capabilities of the EU’s citizens to attain sustainable development goals. The European Commission (EC) envisions that European education institutions at all levels should embrace the SDGs, transforming into organisations where skills for sustainability are both taught and practised. To enact the vision, reforms ranging from building green campuses to adjustments in the curriculum have been proposed [ 33 ]. The Council of the European Union [ 34 ]

[Find the meaning and references behind the names: Just, Gap, Change, New, Level, Aishe, Links, Europe, Long, Dutch, Agreement, Prior, Paris, Aimed, Tools, Large, Stars, Welcome, Few, Big, Non]

Sustainability 2022 , 14 , 5102 3 of 20 has recommended that, as part of the revised ‘key competences for lifelong learning’, all member states should mainstream sustainability education, including ESD, across entire education levels. As part of the European Green Deal, which outlines a long-term strategy for 2050, the Commission intends to support the development of a European competence framework to “assess knowledge, skills and attitudes on climate change and sustainable development” [ 2 ] (p. 19) and support teacher-training programs. One of the most notable EU initiatives in recent years is the European Universities Initiative. Launched in 2018 by the EC, it has financed 41 alliances across Europe to tackle “big issues facing Europe (such as climate protection, democracy, health, big data, migration)” [ 35 ] (p. 1). Many initiatives take place at a national level. In Finland, a forum for sustainable development in higher education has been established [ 36 ]. 2.2. Education for Sustainable Development over Time Since Education for Sustainable Development (ESD) is used as a key reference framework for indicator development in this paper, a brief overview of ESD and its development over time is provided in this section. ESD is a holistic learning approach that “empowers learners to take informed decisions and responsible actions for environmental integrity, economic viability and a just society, for present and future generations, while respecting cultural diversity” [ 37 ] (p. 4). First institutionalised in 1992 by the United Nations Educational, Scientific, and Cultural Organisation (UNESCO) [ 16 ], it is now recognised in key policy documents, including the 2030 Agenda for Sustainable Development (SDG 4.7), the Paris Agreement [ 32 ], and the EU higher education policy on key competencies [ 34 ]. ESD is also directly linked to the 17 SDGs [ 32 ] and aligned with the 2030 Agenda for Sustainable Development [ 38 ]. While ESD as a concept was first promoted by UNESCO in 1992, wider recognition of ESD increased during the UN Decade on Education for Sustainable Development (DESD, 2005–2014), followed by the Global Action Programme on ESD (GAP, 2014–2019). Both programs aimed to integrate sustainable development principles into education and learning [ 16 ], with the GAP program scaling up DESD achievements [ 32 ]. The most recent follow-up program is ESD for 2030, which “aims to build a more just and sustainable world through strengthening ESD and contributing to the achievement of the 17 SDGs” [ 38 ] (p. 14). Prior to ESD for 2030, UNESCO published a report titled ‘Education for Sustainable Development Goals: learning objectives’, linking ESD to all 17 SDGs [ 32 ]. Given ESD’s (i) direct links to sustainable development [ 32 , 38 ], (ii) continued presence and growing international recognition in policy, practice, and academia over time [ 12 , 39 – 42 ], and (iii) prescriptive yet non-restrictive definition [ 32 ] (p. 7), it can serve as a valuable reference framework for building international consensus on education indicators centred around sustainable development 2.3. Higher Education Sustainability Rankings Higher education rankings have been widely researched and thoroughly criticised [ 43 – 47 ], yet have remained relatively popular. While a large number of assessment tools exist for sustainability and sustainable development in the higher education sector (e.g., AISHE 2.0, STARS, GASU, ASSC, PSIR, SAQ, SustainTool, UniSAF) [ 14 – 16 , 18 – 20 ], only a few international rankings so far have undertaken the task—most notably, the UI Green Metric and THE Impact Ranking. A handful of articles have reviewed the approaches used in these rankings [ 48 – 51 ], and several recognised the importance of ESD, but did not specifically focus on ESD. Other sustainability rankings likely exist, particularly at the national level. One such example is a Dutch student-led sustainability ranking—Sustainabul. Thus, other researchers are welcome to identify new sustainability rankings and further contribute to the discussion.

[Find the meaning and references behind the names: Top, Resources, Four, Gain, Vary, Caeiro, Peer, Lozano, Staff, Soon, Six, Major, Middle, Cortese, Role, Cover, Quality, Shown, Common]

Sustainability 2022 , 14 , 5102 4 of 20 3. Methodology 3.1. Conceptualising Sustainability in Key HE Functions Sustainability in higher education can be embedded in all key functions. According to the United Nations guidelines, an integrative, whole-school/institution approach to embedding sustainability in HEIs is preferred. Such an approach often distinguishes six major dimensions that vary slightly in their formulation: (i) education (also ‘teaching and curriculum’, (ii) research, (iii) societal engagement (also ‘external community’, ‘community outreach’), (iv) campus operations (also ‘facilities’, ‘operations’) (v) organisational management (also ‘governance’, ‘leadership’), and (vi) assessment and communication (also ‘assessment and reporting’, ‘monitoring’). In this article, we refer to six such functions, as indicated in Figure 1 , while recognising that multiple naming conventions are possible and have evolved over time [ 15 , 52 , 53 ]. Sustainability 2022 , 14 , x FOR PEER REVIEW 4 of 22 tional level. One such example is a Dutch student-led sustainability ranking—Sustainabul. Thus, other researchers are welcome to identify new sustainability rankings and further contribute to the discussion. 3. Methodology 3.1. Conceptualising Sustainability in Key HE Functions Sustainability in higher education can be embedded in all key functions. According to the United Nations guidelines, an integrative, whole-school/institution approach to embedding sustainability in HEIs is preferred. Such an approach often distinguishes six major dimensions that vary slightly in their formulation: (i) education (also ‘teaching and curriculum’, (ii) research, (iii) societal engagement (also ‘external community’, ‘community outreach’), (iv) campus operations (also ‘facilities’, ‘operations’) (v) organisational management (also ‘governance’, ‘leadership’), and (vi) assessment and communication (also ‘assessment and reporting’, ‘monitoring’). In this article, we refer to six such functions, as indicated in Figure 1, while recognising that multiple naming conventions are possible and have evolved over time [15,52,53]. Figure 1. Key functions for promoting sustainability in higher education (integrative approach). For example, initially, Cortese [52] proposed four dimensions: education, research, campus operations, and community outreach. Soon after, Lozano [53] added ‘assessment and reporting’. More recently, Caeiro [15] used ‘external community’ instead of ‘community outreach’ and ‘assessment and communication’ in place of ‘assessment and reporting’. In this paper, we use ‘societal engagement’ instead of ‘external community’ to indicate mutually beneficial relationships between higher education institutions and wider society [54]. Finally, in recent years, more research has been done on the role of leadership [55–57], governance [18,58], and organisational change and transformation [59–62] in promoting sustainability in higher education. Since change can come from top-down, bottomup, and middle-out approaches, organisational management is more encompassing than the other two and aligns with Caeiro’s suggestion [15]. As Lozano [53] notes, these dimensions are interdependent. This paper focuses on sustainability through the function of education while also touching upon assessment and communication. This limitation should be acknowledged. It is our intention to further elaborate on the remaining dimensions in future. 3.2. Conceptualizing ESD Dimensions in Education When developing indicators on education quality, it is common to distinguish between input, process, and output variables, which are sometimes called presage, process, and product variables, respectively [63,64]. Input variables exist within a university context prior to students beginning their studies, and they include resources, degree of student selectivity, the quality of students and staff. Process variables characterise the teaching and learning process, including measures of student engagement, pedagogies employed, teacher training, and learning environment. Output variables focus on student outcomes, including grades, student retention, and employment. Out of the three categories, process variables are shown to be the best predictors of the learning gain, or how much students learn at HEIs [63]. ESD is increasingly considered “an integral part of quality education” and “encompasses learning content and outcomes, pedagogy and the learning environment” [38] (p. 8). While mapping of the variables to three categories is not always straightforward [63], Figure 2 depicts how the ESD dimensions could be linked to the three commonly used Figure 1. Key functions for promoting sustainability in higher education (integrative approach) For example, initially, Cortese [ 52 ] proposed four dimensions: education, research, campus operations, and community outreach. Soon after, Lozano [ 53 ] added ‘assessment and reporting’. More recently, Caeiro [ 15 ] used ‘external community’ instead of ‘community outreach’ and ‘assessment and communication’ in place of ‘assessment and reporting’. In this paper, we use ‘societal engagement’ instead of ‘external community’ to indicate mutually beneficial relationships between higher education institutions and wider society [ 54 ]. Finally, in recent years, more research has been done on the role of leadership [ 55 – 57 ], governance [ 18 , 58 ], and organisational change and transformation [ 59 – 62 ] in promoting sustainability in higher education. Since change can come from top-down, bottom-up, and middle-out approaches, organisational management is more encompassing than the other two and aligns with Caeiro’s suggestion [ 15 ]. As Lozano [ 53 ] notes, these dimensions are interdependent This paper focuses on sustainability through the function of education while also touching upon assessment and communication. This limitation should be acknowledged. It is our intention to further elaborate on the remaining dimensions in future 3.2. Conceptualizing ESD Dimensions in Education When developing indicators on education quality, it is common to distinguish between input, process, and output variables, which are sometimes called presage, process, and product variables, respectively [ 63 , 64 ]. Input variables exist within a university context prior to students beginning their studies, and they include resources, degree of student selectivity, the quality of students and staff. Process variables characterise the teaching and learning process, including measures of student engagement, pedagogies employed, teacher training, and learning environment. Output variables focus on student outcomes, including grades, student retention, and employment. Out of the three categories, process variables are shown to be the best predictors of the learning gain, or how much students learn at HEIs [ 63 ]. ESD is increasingly considered “an integral part of quality education” and “encompasses learning content and outcomes, pedagogy and the learning environment” [ 38 ] (p. 8). While mapping of the variables to three categories is not always straightforward [ 63 ], Figure 2 depicts how the ESD dimensions could be linked to the three commonly used dimensions for education quality—input, process, and output. The ESD dimensions cover some, but not all variables for education quality.

[Find the meaning and references behind the names: Aim, Board, Falls, Live, Held, Comment, Table, Semi, Short]

Sustainability 2022 , 14 , 5102 5 of 20 Sustainability 2022 , 14 , x FOR PEER REVIEW 5 of 22 dimensions for education quality—input, process, and output. The ESD dimensions cover some, but not all variables for education quality. Figure 2. Comparison of education quality dimensions compared to ESD dimensions. When aligning the education quality and ESD dimensions, ESD’s learning content falls under input variables, since curricula and the corresponding learning content are often prepared prior to the teaching process. In ESD, learning content is addressed by integrating critical issues such as climate change and poverty into the learning curriculum. More recently, topics linked to 17 SDGs have been emphasised. ESD’s dimensions on pedagogy and the learning environment are mapped to process variables, as these dimensions directly affect the learning process and educational gain. In ESD, these dimensions are addressed by utilising interactive, learner-centred pedagogy and providing a whole-institution approach that would enable learners to “live what they learn and learn what they live” [38] (p. 8). Finally, learning outcomes are mapped to output variables. In the ESD approach, learning outcomes are designed to empower learners to take responsibility for the present and future generations and are linked to eight key competencies, including systems thinking, anticipatory thinking, and integrated problem solving [32]. As part of U-Multirank’s new indicator development process, it was decided to focus on three out of the four dimensions, eliminating the ‘learning environment’. Although highly relevant, it was considered too novel and vague to be operationalised in specific indicators in the short-term future. 3.3. Overview of the Research Process The research used an exploratory sequential mixed-method design [65], and it was carried out in three consecutive phases. First, an exploratory literature review was conducted to identify the relevance of sustainability in the higher education sector. Resources were gathered from the academic literature, recent policy documents, and higher education sustainability ranking and assessment tools (e.g., THE Impact Ranking, UI Green Metric, UniSAF). The insights obtained helped to prepare for the second phase and are summarized in Section 2. In the second phase, two semi-structured focus groups were organised in an online setting to collect qualitative feedback ( n = 13). The aim of the focus groups was to discuss indicator relevance, validity, and feasibility with international experts, practitioners, and student representatives. In addition, an open consultation was held with the U-Multirank advisory board, where they were invited to comment on the proposed indicators. The insights from the semi-structured focus groups and the advisory board meeting informed the design of the quantitative indicator survey utilised in the third phase. The survey was distributed to the U-Multirank participants to obtain quantitative data on indicator validity ( n = 227) and feasibility ( n = 256). An overview of the stakeholders consulted is displayed in Table 1 below. Figure 2. Comparison of education quality dimensions compared to ESD dimensions When aligning the education quality and ESD dimensions, ESD’s learning content falls under input variables, since curricula and the corresponding learning content are often prepared prior to the teaching process. In ESD, learning content is addressed by integrating critical issues such as climate change and poverty into the learning curriculum. More recently, topics linked to 17 SDGs have been emphasised. ESD’s dimensions on pedagogy and the learning environment are mapped to process variables, as these dimensions directly affect the learning process and educational gain. In ESD, these dimensions are addressed by utilising interactive, learner-centred pedagogy and providing a whole-institution approach that would enable learners to “live what they learn and learn what they live” [ 38 ] (p. 8) Finally, learning outcomes are mapped to output variables. In the ESD approach, learning outcomes are designed to empower learners to take responsibility for the present and future generations and are linked to eight key competencies, including systems thinking, anticipatory thinking, and integrated problem solving [ 32 ]. As part of U-Multirank’s new indicator development process, it was decided to focus on three out of the four dimensions, eliminating the ‘learning environment’. Although highly relevant, it was considered too novel and vague to be operationalised in specific indicators in the short-term future 3.3. Overview of the Research Process The research used an exploratory sequential mixed-method design [ 65 ], and it was carried out in three consecutive phases. First, an exploratory literature review was conducted to identify the relevance of sustainability in the higher education sector. Resources were gathered from the academic literature, recent policy documents, and higher education sustainability ranking and assessment tools (e.g., THE Impact Ranking, UI Green Metric, UniSAF). The insights obtained helped to prepare for the second phase and are summarized in Section 2 . In the second phase, two semi-structured focus groups were organised in an online setting to collect qualitative feedback ( n = 13). The aim of the focus groups was to discuss indicator relevance, validity, and feasibility with international experts, practitioners, and student representatives. In addition, an open consultation was held with the U-Multirank advisory board, where they were invited to comment on the proposed indicators. The insights from the semi-structured focus groups and the advisory board meeting informed the design of the quantitative indicator survey utilised in the third phase The survey was distributed to the U-Multirank participants to obtain quantitative data on indicator validity ( n = 227) and feasibility ( n = 256). An overview of the stakeholders consulted is displayed in Table 1 below.

[Find the meaning and references behind the names: Mode, America, South, List, North, Male, Final, Set, Panel, September, Erasmus, Asia, Aurora, High, Point, Esu, Sample, Female, Due, See, Ceaser, General, Africa, Hub, Need]

Sustainability 2022 , 14 , 5102 6 of 20 Table 1. Overview of the stakeholders consulted Stakeholders Mode of Consultation Nr. of Participants Gender Geographic Representation Type of Organization Experts and practitioners Semi-structured focus group 6 50% female, 50% male Europe, North America Higher Education Institutions, Intergovernmental Organisations (UNESCO, Green Office Movement, Aurora Network, Green Hub) U-Multirank Advisory Board Open format consultation 16 50% female, 50% male Worldwide Europecentred Higher Educations Institutions, Intergovernmental Organisations (EUA, EURASHE, OECD, ESU, CEASER, ESN, IAU) Student representatives Semi-structured focus group 7 57% female, 47% male Europe Student organisations (ESU, ESN) U-Multirank participants Survey 227 256 Not available Worldwide Higher Education Institutions 3.4. Rankings and the Assessment Tool Reviewed To identify existing indicators that focus on either education for sustainable development (ESD) or education linked to SDGs or sustainability, we reviewed rankings that are specifically focused on sustainability in the higher education sector. As a reference point, one holistic assessment tool was also included. The rankings were selected using a convenience sample based on their (i) focus on sustainability, (ii) visibility in the higher education sector, and (iii) potential relevance for U-Multirank. The rankings reviewed were: the Impact Ranking, UI Green Metric, Sustainabul, and UniSAF framework. The UniSAF framework is a holistic assessment tool that is used as a reference due to the relatively high number of education indicators. While the list is not exhaustive and cannot claim to be representative of all sustainability rankings, and particularly not assessment tools, it highlights some of the more recent indicator trends in sustainability or SDG-related rankings As a result of the indicator review, a preliminary list of potential new ESD indicators was compiled. This list was used as a starting point for stakeholder consultations 3.5. Focus Groups Between September 2020 and April 2021, the U-Multirank project team consulted a diverse group of stakeholders to evaluate the need for internationally comparable indicators on ESD. Amongst the participants were experts, practitioners, policymakers, and student representatives. Three separate consultations were carried out. First, an expert panel on Education for Sustainable Development (6 participants) provided comprehensive feedback on the relevance, validity, and feasibility of indicators. Next, the U-Multirank Advisory Board (16 participants) was invited to provide general feedback on the relevance of the indicators in an open consultation format. A wide spectrum of representatives was included in these consultations (e.g., representatives from UNESCO, AURORA network, OECD, IAU, and EUA). Lastly, a separate consultation was set up with student representatives from the European Student Union (ESU) and Erasmus Student Network (ESN) (7 participants) to obtain student perspectives on indicator relevance, yet students also shared some feedback on indicator validity and feasibility. The final report, including meeting notes, is publicly available While the stakeholders consulted represented several international institutions (e.g., UNESCO, OECD, IAU), the majority of the participants were engaged in European institutions, initiatives, or networks. Therefore, the insights obtained during the consultation process would be of higher importance to European countries and, to a certain extent, North America. No stakeholders from Africa, Asia, Australia, or South America were consulted. For a brief overview of the geographic coverage of stakeholders consulted, see Table 1 .

[Find the meaning and references behind the names: June, Answer, Areas, Forward, Sent]

Sustainability 2022 , 14 , 5102 7 of 20 3.6. Practitioners Surveyed In order to identify ESD indicators with the highest feasibility (availability of data) and the most commonly used definitions for ESD or sustainability-related education, the U-Multirank team conducted a survey amongst HE practitioners. The survey was sent out to U-Multirank participants between February and June 2021. It was permitted to forward the survey to other colleagues at one’s institution, resulting in a larger number of participants than institutions. In the survey, neither of the two questions were required, which led to a different number of answers per question. The question on indicator data availability was completed by 256 respondents, while the question on the most commonly used ESD definitions was answered by 227 respondents. Respondents held various positions, including ranking officers, sustainability coordinators, and international affairs coordinators 3.7. Criteria for Indicator Assessment Building on the insights from the academic literature, policy papers, and existing rankings, a preliminary list of indicators was prepared for stakeholder consultations. During the focus groups, stakeholders were invited to assess indicators on three criteria: relevance, validity, and feasibility. Such criteria are frequently used in the research literature on education indicators [ 26 – 28 , 66 ]. The same criteria were used to assess other new indicator groups for U-Multirank, such as social inclusion [ 66 ] and effective teaching and learning. The relevance criterion was seen as critical, since indicators would only be used if the phenomenon measured is considered important by the users [ 27 ]. If the relevance criterion is not met, institutional leaders and practitioners will not invest time in data collection. The following criterion—validity—evaluates if an indicator reflects the phenomenon it is meant to represent [ 67 ]. Validity is particularly important for ESD indicators, since no operational definition is available at an international level, and multiple different definitions are used, a result indicated by the feasibility survey. The third criterion, feasibility, addresses practical aspects of the data collection process and data availability, including institutions’ capacity and readiness to obtain the required data considering their available resources and expertise [ 28 ]. 4. Results 4.1. Rankings Reviewed (RQ 1) Section 4.1 aims to answer the first research question: (RQ 1) To what extent are ESDrelated indicators available in the existing higher education sustainability rankings? The rankings reviewed were the Impact Ranking, UI Green Metric, and Sustainabul. In addition, UniSAF, a holistic assessment tool, was included as a reference due to the large number of education indicators it features. Below, Table 2 provides a comparison of all four tools 4.1.1. Indicators in the THE Impact Ranking The Times Higher Education (THE) Impact ranking, first released in 2019, claims to be the only ranking to assess the performance of higher education institutions against the Sustainable Development Goals as defined in the Agenda 2030 for Sustainable Development The participation in the ranking is voluntary, its coverage is global, and assessment is done at the institutional level across the SDGs, as well as separately on each SDG. To participate, institutions need to submit information on at least three SDGs in addition to SDG 17 (strong institutions), which is mandatory [ 68 ]. The second edition (2020) included 768 HEIs from 85 countries [ 69 ]. SDGs were assessed across four areas—(1) research, (2) stewardship, (3) outreach, and (4) teaching. The THE Impact ranking collects information about sustainability-related education under two goals: SDG 13—Climate Action and SDG 17—Partnerships for the Goals. Under SDG 13, HEIs need to submit information about education programs on climate action, and for SDG 17, education must be centred around the SDGs [ 70 , 71 ]. Several other metrics are related to sustainable development, but extend beyond the scope of education. For example, SDG 4

[Find the meaning and references behind the names: Range, Transport, Pillar, Energy, Self, Location, Seven]

Sustainability 2022 , 14 , 5102 8 of 20 (Quality education) [ 72 ] reports on inclusive access, while several other SDGs (SDGs 6, 7, 14) consider outreach programs to local communities ranging from topics about good water management (SDG 6) [ 73 ], [ . . . ] energy efficiency and clean energy (SDG 7) [ 74 ], and sustainable management of fisheries [ . . . ] [ 75 ] (SDG 14) Table 2. An overview of the sustainability ranking and assessment tools Category THE Impact UI Green Metric Sustainabul UniSAf Focus SDGs Environmental Sustainability Student-driven assessment Holistic sustainability self-assessment Level Institutional Institutional Institutional Institutional Coverage International International Dutch International, EU-oriented Release year 2019 2010 2012 Not applicable Participants (2020) 768 912 30 Not applicable Areas covered 17 SDGs in the areas of (1) research, (2) stewardship, (3) outreach, (4) teaching, (1) energy and climate change (2) education and research (3) transportation (4) waste (5) setting and infrastructure (6) water (1) education, (2) research, (3) operations, (1) education, (2) research, (3) community, (4) operations, (5) governance Indicators: Learning Content (1) local education programs on climate change/SDGs; (2) community outreach programs (1) ratio of sustainability courses to total courses/subjects (1) sustainabilityoriented education programs (2) sustainabilityoriented education minors (3) SDGs in education curriculum (E-1) courses focused on sustainability (E-2) percentage of courses (E-3) availability of courses to students (E-4) educational offerings for general public (E-5) sustainability specialisation (E-6) sustainability focused service learning (E-7) student enrolment Indicators: Pedagogy (4) training and support for teaching staff (E-11) professional development of sustainability educators (E-12) links between sustainability research and education (E-13) partnerships (E-8) student involvement Indicators: Learning Outcomes (3) graduates in relevant fields, such as % of health graduates (SDG 3) (E-9) alumni destinations (E-10) educational outcomes 4.1.2. Indicators in the UI Green Metric The UI Green Metric World University Ranking, launched by Universities Indonesia in 2010, aims to provide information about the sustainability of universities around the world. Participation in the ranking is voluntary, its coverage is global, and assessment is done at the institutional level. Data collection happens through online questionnaires sent to university administrators [ 76 , 77 ]. The ranking criteria are built around six pillars—energy and climate change (21%), education and research (18%), transportation (18%), waste (18%), setting and infrastructure (15%), and water (10%). The assignment of weights suggests that the ranking is skewed towards operational measures (82%), utilising a wide range of criteria, such as the campus location, amount of green space, energy use, transport, water use, recycling, and waste treatment. Under the ”Education and Research” pillar, one out of seven indicators is dedicated to education—”ratio of sustainability courses to total courses/subjects” [ 78 ].

[Find the meaning and references behind the names: Eng, Broad, Morgen, Feel, Hand, Run, Pre]

Sustainability 2022 , 14 , 5102 9 of 20 4.1.3. Indicators in Sustainabul Sustainabul is a Dutch national sustainability ranking of higher education institutions (HEIs). First released in 2012, Sustainabul is an example of a bottom-up ranking, commenced and run annually by a national student network, ”Studenten voor Morgen” (Eng: Students for tomorrow). ”Studenten voor morgen” created the ranking to encourage institutions to become more sustainable through competition and knowledge sharing. Sustainabul assesses HEIs on three dimensions—education, research, and operational management—and gathers information about the best sustainability practices [ 79 ]. In 2020, Sustainabul featured the thirty largest Dutch HEIs in terms of student enrolments, covering both universities and universities of applied sciences. Sustainabul’s methodology allocates equal weights to three key dimensions—sustainability in education, research, and operational management (maximum of 110 points per category)—while best practices can receive an additional 30 points. The education dimension consists of four measures: education programs centred on sustainability (50%), minors centred on sustainability (30%), training and support for teaching staff (15%), and integration of SDGs in the curriculum (5%) [ 80 ]. 4.1.4. Indicators in the UniSAF Framework (Green Office Movement) The Green Office Movement was launched in 2010, aiming to create a sustainability platform that empowers students and staff to embed sustainability in the curriculum, research, operations, community, and governance of their higher education institution [ 81 ]. One of the resources provided by the Green Office Movement is University Sustainability Assessment Framework (UniSAF). Although not a ranking, the UniSAf framework offers a broad spectrum of indicators for a holistic assessment of an institution. The indicator categories include education, research, community, operations, and governance. Thirteen indicators have been proposed for education, and these are split into three categories— educational offering, students, and course quality [ 82 ]. 4.1.5. ESD Indicators in the Existing Sustainability Rankings and Assessment Tool An overview of the indicators in the HE sustainability rankings allows us to answer the first question: To what extent are ESD-related indicators available in the existing higher education sustainability rankings? All three rankings have indicators on learning content—namely, programs, minors, courses, or community outreach focused on climate change, sustainability, or SDG topics. Only one of the rankings, Sustainabul, has indicators on pedagogy, referring to training and support provided for teaching staff. In addition, for learning outcomes, only one of the rankings provides an indicator—the number of health graduates On the other hand, the reference tool UniSAF has multiple indicators in each category Learning content also considers not only courses, but also the availability of courses to the student population (e.g., access in terms of regulations, timing, pre-requisites) and service-learning courses. For pedagogy (course quality in UniSAf), UniSAF considers professional development and student involvement (the extent to which students feel that they can shape their learning experiences) and links to research and partnerships with external parties. Finally, for learning outcomes, UniSAF considers educational outcomes based on the sustainability competencies acquired and the alumni destinations Thus, it appears that the existing rankings provide little input for ESD indicators, mostly emphasising learning content, but with somewhat limited attention to pedagogies and learning outcomes. However, frameworks such as UniSAF are helpful in developing new ESD indicators, since they consider learning content, student involvement, professional development of teachers, sustainability competencies, and alumni destinations, effectively addressing most ESD dimensions (see Figure 2 ).

[Find the meaning and references behind the names: Stage, Add, Teach, Minor, Main, Last, Jobs]

Sustainability 2022 , 14 , 5102 10 of 20 4.2. Challenges when Developing ESD Indicators (RQ 2) Section 4.2 aims to answer the second research question: (RQ 2) What are the challenges for developing internationally comparable ESD indicators in the higher education sector? Building on the indicators found in the aforementioned rankings and, particularly, the UniSAf framework, a preliminary list of ESD indicators was developed for stakeholder consultations. During this stage, potential challenges for internationally comparable indicators were identified An overview of the potential indicators, their operationalisation, and challenges is available in Table 3 . Indicators were grouped around three ESD dimensions—learning content, pedagogy, and learning outcomes. Under learning outcomes, graduate outcomes were also included, though these are technically not learning outcomes, but a broader outcome measure. In addition, a preliminary list of challenges for operationalising these indicators was identified, from which most were rooted in three main challenges. The first challenge stems from the need to agree on thresholds or guidelines to identify and, consequently, create a classification mechanism for ESD-related education/teacher training/jobs/competencies (see Table 3 , challenges 1, 3, 6, 7). Since ESD is a holistic framework that encompasses multiple aspects of education, it is challenging to propose a specific yet contextually relevant definition that would allow the classification of educational offerings as ESD. Secondly, comparing workloads across different systems on a global scale is difficult (see Table 3 , challenges 2, 4). At the EU level, the European Credit Transfer System (ECTS) can be used, but a conversion mechanism needs to be established at a global level. Lastly, national qualifications for teacher training differ substantially across countries. It is possible that in some systems, teachers are better prepared to teach ESD topics and teaching methods and, therefore, should not be penalised for not having additional training afterwards (see Table 3 , challenge 5). This challenge was identified during the consultations, but has been included in this table for a better overview. This list is the first attempt to identify challenges and add to the existing literature. Any additions and modifications are welcome in future research Table 3. Preliminary ESD indicators and underlying challenges Indicator Category Indicators Challenge Learning content ESD course offering (% of total courses) (1) lack of criteria to identify education that meets ESD standards; (2) limited ability to compare education workload internationally ESD minor offering (%of total minors) ESD program offering (% of total programs) Pedagogy ESD training for educators (% trained within last 5 years) (3) lack of criteria to identify training that meets ESD standards; (4) limited ability to compare training workload internationally; (5) limited ability to recognise initial teacher training’s contribution to the ESD approach Learning Outcomes ESD course graduates (% of total course graduates) (1) lack of criteria to identify education that meets ESD standards; (2) limited ability to compare education workload internationally ESD minor graduates (% of total minor graduates) ESD program graduates (% of total program graduates) ESD competencies (alumni self-reported scores) (6) lack of criteria to assess ESD competencies (eight official ESD competencies identified by UNESCO) ESD alumni in relevant SD-related jobs (% of total alumni) (7) lack of criteria to identify SD-related jobs 4.3. Results from the Stakeholder Consultations and Feasibility Survey (RQ 3) Section 4.3 aims to answer the third research question: (RQ 3) What are the most promising ESD indicators in the higher education sector, assessed against relevance, validity,

[Find the meaning and references behind the names: Rank, Keep, Drop, Pool, Fit, Balance, Case, Rather]

Sustainability 2022 , 14 , 5102 11 of 20 and feasibility? To answer the question, the outcomes of stakeholder consultations are discussed, summarising reflections on the proposed ESD indicators’ relevance, validity, and feasibility. The procedure for stakeholder consultations consisted of focus groups and a survey, and it is described in Sections 3.5 and 3.6 . 4.3.1. Relevance Indicator relevance is one of the critical factors when developing new indicators While the policy documents signalled clear priority for sustainability indicators, additional consultations experts, practitioners, and students provided contextualised insights on the potential use and importance of indicators. During the stakeholder consultations on indicator relevance, three questions emerged: (i) Are ESD indicators relevant? (ii) Relevant for what purpose? (iii) Relevant for whom? All stakeholders agreed that ESD and, more broadly, education linked to sustainable development are highly relevant and critical to attaining global policy goals. Therefore, (i) ESD indicators are relevant, since they can help to establish a baseline, reflect progress over time, and allow one to learn from others by providing a contextualised comparison. At the same time, indicators are imperfect measures and ‘what gets measured, gets noticed’ Therefore, all stakeholders should remain critical and keep examining the purpose of the indicators and parties involved Stakeholders noted that ESD indicators should (ii) fit the purpose, stimulating institutional learning and transparency rather than competition amongst HEIs. Higher education rankings typically create tension and competition and, therefore, are not well suited for new ESD indicators. For example, in a ranking, a higher education institution may drop in a rank even if it has improved its sustainability performance if the participant pool has changed from one year to another. Instead, ratings with transparent criteria and predetermined levels may be used. Moreover, the indicators should be (ii) holistic rather than focusing on education only, covering other functions, such as operations and governance. In addition, the indicators should also (ii) reflect student and educator experiences Stakeholders emphasised that indicators should be relevant (iii) for HEIs with diverse profiles, including universities of applied sciences and vocational education institutions Still today, many rankings predominantly focus on traditional research universities. The new ESD indicators should reflect the needs and indicators relevant for all institutions, not favouring research-intensive institutions. In addition, (iii) national context should be considered as much as possible when developing indicators to make them relevant for a large number of countries and geographic regions. For example, while all ESD indicators were generally seen as relevant, it was noted that some national systems do not use a minor system, and such indicators would not be relevant in their system 4.3.2. Validity In addition to being relevant, indicators must be valid, representing the phenomenon that they aim to capture. Since the definition of ESD is sufficiently abstract, there is no one manner in which to operationalise ESD education. Therefore, a careful operationalisation of indicators is important. Such operationalisation requires agreeing on the criteria used to classify education as related to ESD. Since many HEIs focus on SDGs, yet are not familiar with ESD, we expanded the operationalisation to capture ESD education as well as SDGrelated education. This led to the following question: (iv) What criteria can be used to classify education as ESD or SDG related? In the case of U-Multirank, the selected criteria need to be sufficiently flexible to acknowledge efforts of diverse higher education institutions and geographic locations, yet they must also be internationally comparable. This creates a tension between flexibility and comparability, where an appropriate balance needs to be found During the stakeholder consultations, three classification criteria were proposed, yet a consensus on the preferred approach was not reached. The first focused on the (i) content covering complex problems from three perspectives (social, environmental,

[Find the meaning and references behind the names: Field, Cost]

Sustainability 2022 , 14 , 5102 12 of 20 and economic), while the second emphasised the need for (ii) action-oriented teaching methods typical of the ESD approach. The last addressed (iii) SDG perspectives in a specific field. To investigate what criteria are used in practice, the U-Multirank team ran a survey, proposing the three potential definitions, as well as allowing respondents to add their own answers. Respondents could select multiple answers. As can be seen from Figure 3 , in total, 227 respondents provided their answers, with many selecting multiple definitions. The most common ESD definition focused on complex problems considering three perspectives (36%), but alternative options, such as definitions linked to teaching methods (31%) and field-specific focus on SDGs (29%), were also commonly used. Amongst the 4% who mentioned other answers, the most common alternative was using learning goals that are often linked to key competencies (e.g., systems thinking, critical thinking) as a criterion for ESD or SDG-related education, while others mentioned extracurricular activities, such as learning with communities Sustainability 2022 , 14 , x FOR PEER REVIEW 13 of 22 criterion for ESD or SDG-related education, while others mentioned extracurricular activities, such as learning with communities. Figure 3. What criteria does your institution use to classify a program or course as ESD or SDGrelated? [Multiple answers are possible]. Validity is important, and for ESD or SDG-related education, establishing consensus on how to operationalise these indicators is essential. It should provide sufficient flexibility while also ensuring transparency and contextual relevance. The survey results indicate that all three definitions are seen as helpful, while the most common one is built on the commonly used definition of sustainable development, addressing complex problems across three dimensions—social, economic, and environmental. During the stakeholder consultation, we also received feedback on other indicators. In particular, the validity of an indicator on teacher training was criticised. Teacher qualifications differ considerably across countries. Since some might entail elements of ESD and transformational teaching methods while others may not, the starting points of the educators differ. Hence, an indicator on ESD training may not represent the quality of ESD teaching. However, it was suggested that an indicator on ESD competencies, particularly those assessed by students, could serve as a good representation of learning outcomes. Overall, consultations showed that operationalising ESDor SDG-related indicators is challenging. However, in the long run, building international consensus on the relevant criteria may facilitate the process. 4.3.3. Feasibility Indicator feasibility is critical in determining whether indicators are eventually used. If data are not available or the cost of collecting the required data elements is too high for an institution, even highly relevant and potentially valid indicators may not be used. In order to understand the feasibility of the proposed indicators, we ran a survey, aiming to understand (v) what indicators have the highest feasibility scores? The survey was shared with representatives from HEIs participating in U-Multirank. The representatives were asked whether they were collecting data on proposed ESD or SDG-related education measures. In this context, ‘SDG-related measures’ refer to SDGs Figure 3. What criteria does your institution use to classify a program or course as ESD or SDGrelated? [Multiple answers are possible] Validity is important, and for ESD or SDG-related education, establishing consensus on how to operationalise these indicators is essential. It should provide sufficient flexibility while also ensuring transparency and contextual relevance. The survey results indicate that all three definitions are seen as helpful, while the most common one is built on the commonly used definition of sustainable development, addressing complex problems across three dimensions—social, economic, and environmental During the stakeholder consultation, we also received feedback on other indicators In particular, the validity of an indicator on teacher training was criticised. Teacher qualifications differ considerably across countries. Since some might entail elements of ESD and transformational teaching methods while others may not, the starting points of the educators differ. Hence, an indicator on ESD training may not represent the quality of ESD teaching. However, it was suggested that an indicator on ESD competencies, particularly those assessed by students, could serve as a good representation of learning outcomes. Overall, consultations showed that operationalising ESDor SDG-related indicators is challenging. However, in the long run, building international consensus on the relevant criteria may facilitate the process.

[Find the meaning and references behind the names: Less, Right, Able]

Sustainability 2022 , 14 , 5102 13 of 20 4.3.3. Feasibility Indicator feasibility is critical in determining whether indicators are eventually used If data are not available or the cost of collecting the required data elements is too high for an institution, even highly relevant and potentially valid indicators may not be used. In order to understand the feasibility of the proposed indicators, we ran a survey, aiming to understand (v) what indicators have the highest feasibility scores? The survey was shared with representatives from HEIs participating in U-Multirank The representatives were asked whether they were collecting data on proposed ESD or SDG-related education measures. In this context, ‘SDG-related measures’ refer to SDGs and content linked to SDGs with a more holistic approach (teaching methods, ESD competencies). As shown in Figure 4 , respondents could indicate if data were already collected or would be collected in the near future (within 1–3 years). The results were ordered based on the likelihood that the data would be readily available within three years. When we refer to the likelihood, we combine values for already available data with the data that shall become available in the near future. The total number of respondents answering the question was 256. However, the answers were not forced and varied slightly for each indicator, ranging from 229 to 247 Sustainability 2022 , 14 , x FOR PEER REVIEW 14 of 22 and content linked to SDGs with a more holistic approach (teaching methods, ESD competencies). As shown in Figure 4, respondents could indicate if data were already collected or would be collected in the near future (within 1–3 years). The results were ordered based on the likelihood that the data would be readily available within three years. When we refer to the likelihood, we combine values for already available data with the data that shall become available in the near future. The total number of respondents answering the question was 256. However, the answers were not forced and varied slightly for each indicator, ranging from 229 to 247. Figure 4. Data collection practices on the proposed ESD and SDG-related indicators. As can be seen from Figure 4, the four most common indicators, judged on the current and near-future data availability, are ”ESD courses offered” (89%), ”ESD-related programs” (80%), ESD course graduates (78%), and ESD program graduates (76%), featuring two indicators from the learning content category and two from the learning outcome category. For these indicators, more than 30% of the respondents indicated that the data were already available, while at least 40% reported that they would be able to provide it in the near future. For the remaining indicators, between 20 and 27% of the respondents were able to provide data right away, yet between 40 and 51% indicated that such data would be available in the future. The indicator with the lowest feasibility was ESD alumni in relevant jobs (64%), but even for this indicator, the majority of respondents expected to have data in the near future. During the consultations, stakeholders provided qualitative feedback on indicators they considered less feasible in terms of obtaining the required data. For example, it was noted that, in some national systems, minor programs are not Figure 4. Data collection practices on the proposed ESD and SDG-related indicators As can be seen from Figure 4 , the four most common indicators, judged on the current and near-future data availability, are ”ESD courses offered” (89%), ”ESD-related programs” (80%), ESD course graduates (78%), and ESD program graduates (76%), featuring two indicators from the learning content category and two from the learning outcome category.

[Find the meaning and references behind the names: Move, Low, Job]

Sustainability 2022 , 14 , 5102 14 of 20 For these indicators, more than 30% of the respondents indicated that the data were already available, while at least 40% reported that they would be able to provide it in the near future. For the remaining indicators, between 20 and 27% of the respondents were able to provide data right away, yet between 40 and 51% indicated that such data would be available in the future. The indicator with the lowest feasibility was ESD alumni in relevant jobs (64%), but even for this indicator, the majority of respondents expected to have data in the near future. During the consultations, stakeholders provided qualitative feedback on indicators they considered less feasible in terms of obtaining the required data. For example, it was noted that, in some national systems, minor programs are not used, and therefore, it would neither be possible to obtain data on such indicators, nor it would be relevant. On the other hand, ‘alumni in relevant jobs’ was seen as highly relevant, yet less feasible. The most likely data source would be existing institutional alumni surveys, with little control over response rates and often predefined job sectors. With predefined sectors, it might not be possible to identify if alumni are working on sustainability-related topics. For example, some engineers might be heavily involved in addressing sustainability challenges, while others are not. Thus, an additional question might be needed to understand whether alumni are engaged in sustainability-related jobs. Nonetheless, the feasibility survey suggests that most HEIs are proactively working on obtaining ESD-related data for even more challenging indicators 5. Discussion 5.1. Prioritising ESD Indicators Insights from stakeholder consultations and a feasibility survey allowed us to assess indicators on their relevance, validity, and feasibility and to prioritise them in three tiers based on the overall scores (see Table 4 ). More detailed feedback for each indicator is available in Table 5 . The feasibility of the indicators was assessed against the possibility of obtaining the data within three years rather than immediately. In the first tier, four indicators were included: ESD courses, ESD programs, ESD course graduates, and ESD program graduates. All four indicators were assessed as highly relevant by stakeholders, had medium validity due to the need to clarify definitions, and medium to high feasibility, as indicated by the survey. By establishing a consensus around definitions for ”ESD educational offering”, it would be possible to increase the perceived validity of the indicators, as well as to enhance the feasibility of data collection. The insights from the feasibility survey indicate that, in the following years, the feasibility assessment is likely to move from a medium-high to a high score Table 4. Evaluating the new ESD indicators on their relevance, validity, and feasibility Category Indicator Relevance Validity Feasibility Priority Learning content ESD course offering High Medium High 1 st tier ESD minor offering Medium Medium Medium 2 nd tier ESD program offering High Medium High 1 st tier Pedagogy ESD training for educators High Low Medium 2 nd tier Learning Outcomes ESD course graduates High Medium High 1 st tier ESD minor graduates Medium Medium Medium 2 nd tier ESD program graduates High Medium High 1 st tier ESD competencies High Medium Medium 2 nd tier ESD alumni in relevant jobs High Medium Low 2 nd tier

[Find the meaning and references behind the names: Ready, Pay, Market, Full, Row]

Sustainability 2022 , 14 , 5102 15 of 20 Table 5. Detailed feedback on indicator relevance, validity, and feasibility Category Indicator Relevance Validity Feasibility Learning content ESD course offering High relevance, showcases availability of courses, often to a broad audience Criteria for identifying and comparing (workload) ESD-related education are required; criteria need to be sufficiently flexible to fit various contexts and geographies; at least three definitions are commonly used: (i) multidisciplinary sustainability content, (ii) field-specific sustainability content and applications, (iii) multidisciplinary sustainability content combined with ESD teaching methods High feasibility, over 1/3 (41%) already collect data, 89% will be ready within 3 years ESD minor offering Medium relevance, showcases availability of minor/specialisation yet ‘minors’ are not used in all countries See above (row 2, column 4) Medium feasibility, less than 1/3 (27%) already collect data, 74% will be ready within 3 years ESD programs offering High relevance, showcases availability of full programs See above (row 2, column 4) High feasibility, over 1/3 (40%) already collect data, 80% will be ready within 3 years Pedagogy ESD training for educators High relevance, reflects educators’ knowledge and competencies, impacts educational quality See above (row 2, column 4). In addition, the validity of the indicator has been criticised for not considering national teacher training qualifications, since some systems pay more attention to ESD-related teaching methods (e.g., student-centred, transformational learning) Medium feasibility, less than 1/3 (27%) already collect data, 72% will be ready within 3 years Learning Outcomes ESD course graduates Medium relevance, showcases the number of beneficiaries from the course See above (row 2, column 4) High feasibility, over 1/3 (31%) already collect data, 78% will be ready within 3 years ESD minor graduates Medium relevance, showcases the number of beneficiaries from the minor yet ‘minors’ are not used in all countries See above (row 2, column 4) Medium feasibility, less than 1/3 (23%) already collect data, 70% will be ready within 3 years ESD program graduates High relevance, showcases the number of beneficiaries from the program See above (row 2, column 4) High feasibility, over 1/3 (31%) already collect data, 76% will be ready within 3 years ESD competencies High relevance, showcases to what extent students have acquired ESD competencies Currently, 8 ESD competencies have been defined by UNESCO (e.g., systems thinking, anticipatory competency) However, criteria need to be established to assess these competencies Medium feasibility, less than 1/3 (24%) already collect data, 75% will be ready within 3 years ESD alumni in relevant jobs High relevance, showcases to what extent students utilise their education in the labour market and create impact Criteria for identifying jobs related to sustainable development are needed; most likely requires self-assessment due to the multi-faceted nature of jobs Low feasibility, less than 1/3 (20%) already collect data, less than 2/3 (64%) will be ready within 3 years.

[Find the meaning and references behind the names: Bias, Share, Strand, Study, Small, Deans, Look]

Sustainability 2022 , 14 , 5102 16 of 20 Furthermore, in the second tier, five indicators were included—ESD minors and minor graduates, ESD competencies, ESD training for educators, and ESD alumni in relevant jobs. ESD minors and ESD minor graduates received a medium score on all three dimensions. This may be partially attributed to some educational systems not offering minors, as suggested in the stakeholder consultations. However, for institutions where such programs are offered, such indicators could still be relevant. Furthermore, ESD training for educators was considered a relevant indicator, yet its validity was criticised due to differences in national systems for teacher qualifications and the potential to signal mistrust towards educators. It received a medium feasibility score. ESD competencies were seen as a promising way to incorporate student views, but were criticised for their subjectivity. Moreover, while eight ESD competencies have been communicated [ 32 ], these competencies still need to be operationalised into measurable indicators. ”ESD alumni in relevant jobs” would require thresholds to establish what constitutes ESD-relevant jobs While highly relevant, the biggest drawback of the indicator was its feasibility score, as shown by the survey results and indicated by stakeholders. Alumni surveys might suffer from job classification methods that do not reveal whether sustainability is embedded in the job role and would likely differ across countries and even institutions 5.2. Limitations To identify new indicators, we used ESD as a reference framework. A different framework would most likely yield an alternative set of indicators. Moreover, as indicated by the UniSAF indicator, several additional indicators could be included, such as “educational offerings for general public (E-4)” and “sustainability focused service learning (E-6)” [ 82 ] if the education of local community members would have been included in the scope Moreover, throughout the stakeholder consultations, we aimed to find indicators that could be internationally comparable. For institutions that look to measure their own progress, it is not always necessary to develop internationally comparable indicators. Even though international indicators can facilitate peer-to-peer learning and provide additional reference points, at times, local contexts and needs might be stronger drives for certain indicators [ 16 ]. Furthermore, we used a snowballing method to select the stakeholders consulted, including international experts. While it is a relatively common approach given the limited availability of international experts, some selection bias can be expected. Similarly, for the feasibility survey, self-selection bias exists. While the survey was sent to all U-Multirank participants ( n > 1700), only a relatively small share completed the survey. It is likely that these participants were already more interested and enthusiastic about sustainability topics 5.3. Future Research In order to establish ESD indicators, further research could be carried out on the specific guidelines that could be utilised for establishing ESD indicators. Moreover, a repeated feasibility study could be conducted in the near future to evaluate whether institutions have started collecting data on ESD indicators, as proposed by the feasibility survey in this study. In addition, this study explored indicators on three out of four ESD dimensions, eliminating the ‘learning environment’, since it was considered too novel and broad. However, further research could attempt to measure this dimension [ 38 ]. More broadly, another strand of future research could investigate the role of academic leaders, such as rectors, deans, and heads of departments, in implementing and supporting initiatives for monitoring ESD or SDG-related education at their institutions and academic units 6. Conclusions Global societal trends (e.g., Extinction Rebellion), international policy discourse [ 1 ], and large-scale initiatives [ 35 ] signal a clear priority given to sustainable development initiatives. Higher education institutions have a critical role in contributing to these priorities, particularly by preparing future leaders in a diverse set of fields. Hence, these institutions

[Find the meaning and references behind the names: Plan, Kaiser, Enough, Frans, White, Half]

Sustainability 2022 , 14 , 5102 17 of 20 should be transparent in communicating their efforts to existing and prospective students, as well as broader society. To identify internationally comparable ESD indicators, this paper reviewed the relevant literature and existing sustainability rankings, conducted stakeholder consultations, and ran a survey. The results of this paper are threefold. First, it compares existing higher education sustainability rankings—the Impact Ranking, UI Green Metric, Sustainabul—and highlights that these rankings pay limited attention to ESD indicators. In particular, only one out of three rankings provides measures on pedagogy and learning outcomes. Indicators on the available sustainability education (courses, minors, programs) are common across all three rankings. A holistic assessment framework, UniSAF, can provide valuable insights into the future development of ESD indicators, since it contains a much more comprehensive list of indicators, including measures on pedagogy and learning outcomes Second, building on the existing rankings and using ESD as a reference, the paper provides a preliminary list of indicators together with the underlying challenges. The challenges include a lack of criteria for identifying ESD-related education while recognising that such criteria need to be sufficiently flexible to fit diverse contexts and geographic locations. This challenge is not limited to educational offerings, and also includes ESDrelated competencies, teacher training, and alumni job destinations. In addition, criteria for comparing workloads across different systems need to be established, which can be similar to the ECTS system used in Europe Third, stakeholder consultations and surveys provided insights on indicator relevance, validity, and feasibility. ESD indicators were considered highly relevant by all stakeholder groups (students, experts, and practitioners). However, assessing general relevance is not enough. The potential users of the indicators need to consider ‘for what purpose are indicators relevant?’ and ‘for whom are these indicators relevant’? The stakeholders believed that indicators should promote institutional learning and be inclusive of all types of institutions, not only research universities. The discussions on indicator validity revealed that a consensus needs to be established on appropriate criteria for identifying ESD-related courses/programs/minors/relevant alumni jobs and educator training. At least three types of definitions are available to operationalise ESD courses and programs—(i) content-driven knowledge linked to complex interdisciplinary challenges from social, economic, and environmental perspectives, (ii) teaching methods focused on transformational learning, and (iii) field-specific content knowledge of sustainability. Lastly, the survey on indicator feasibility revealed that, while less than half of the respondents currently collect information on any of the proposed ESD indicators, more than half (64%) plan to collect such information within three years, signalling strong interest in and commitment to monitoring this area Funding: This research was not directly funded by any external party, but was built on the insights obtained as part of the U-Multirank project, which is partially funded by the European Commission Institutional Review Board Statement: Not applicable Informed Consent Statement: Informed consent was obtained from all subjects involved in the study Data Availability Statement: Data are not yet publicly available Acknowledgments: The paper was built on the insights gathered in several internal reports, including a white paper on ESD indicators, meeting minutes with stakeholders, and a final procedure report. Frans Kaiser, a colleague, provided valuable feedback and revisions for internal reports and this paper Conflicts of Interest: The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

[Find the meaning and references behind the names: De Moor, Davies, Asc, Martins, Urbanski, Ahmad, Strike, Hammond, Bauer, January, Pires, Kingsley, Belgium, Evidence, Chiba, Soc, Martinuzzi, Dores, Int, Rica, Sci, York, Coelho, Jabbour, Berlin, Mascarenhas, October, Subtil, Uba, Bernaldo, Saito, Cave, Germany, Leal, Jaeger, London, Hamon, Brick, Goo, Bormann, Filho, Century, Aldaz, Sommer, Fern, Keele, Feijoo, Edwards, Mccormick, Franco, Prod, France, Sanchez, Ramos, Dev, Hardi, Chen, Hoogendoorn, Vaz, Dhakal, Goal, Jessica, Redman, Washington, March, Wouters, Canada, Early, Moor, Schools, Springer]

Sustainability 2022 , 14 , 5102 18 of 20 References 1 General Assembly Resolution Adopted by the General Assembly on 19 September 2016 ; A/RES/71/1, 3 October 2016 (The New York Declaration); General Assembly: New York, NY, USA, 2015 2 European Commission The European Green Deal [Communication from the Commission to the European Parliament, the European Council, the Council, the European Economic and Social Committee and the Committee of the Regions ] ; European Commission: Brussels, Belgium, 2019 3 Franco, I.; Saito, O.; Vaughter, P.; Whereat, J.; Kanie, N.; Takemoto, K. Higher education for sustainable development: Actioning the global goals in policy, curriculum and practice Sustain. Sci 2019 , 14 , 1621–1642. [ CrossRef ] 4 Wahlström, M.; Sommer, M.; Kocyba, P.; De Vydt, M.; De Moor, J.; Davies, S.; Wouters, R.; Wennerhag, M.; van Stekelenburg, J.; Uba, K. Protest for a future: Composition, mobilization and motives of the participants in Fridays For Future climate protests on 15 March, 2019 in 13 European cities. 2019, Project Report. Protest for a Future. Available online: https://eprints.keele.ac.uk/6571/ (accessed on 20 January 2022) 5 Noth, F.; Tonzer, L. Understanding climate activism: Who participates in climate marches such as “Fridays for Future” and what can we learn from it? Energy Res. Soc. Sci 2022 , 84 , 102360. [ CrossRef ] 6 Cologna, V.; Hoogendoorn, G.; Brick, C. To strike or not to strike? An investigation of the determinants of strike participation at the Fridays for Future climate strikes in Switzerland PLoS ONE 2021 , 16 , e 0257296. [ CrossRef ] 7 Times Higher Education. Impact Rankings 2021. Available online: https://www-timeshighereducation-com.ezproxy 2.utwente. nl/impactrankings#!/page/0/length/25/sort_by/rank/sort_order/asc/cols/stats (accessed on 20 January 2022) 8 Cai, Y.; Ahmad, I. From an Entrepreneurial University to a Sustainable Entrepreneurial University: Conceptualization and Evidence in the Contexts of European University Reforms High. Educ. Policy 2021 , 1–33. [ CrossRef ] 9 Cai, Y.; Ma, J.; Chen, Q. Higher education in innovation ecosystems Sustainability 2020 , 12 , 4376. [ CrossRef ] 10 Findler, F.; Schönherr, N.; Lozano, R.; Reider, D.; Martinuzzi, A. The impacts of higher education institutions on sustainable development: A review and conceptualization Int. J. Sustain. High. Educ 2019 , 20 , 23–38. [ CrossRef ] 11 Garc í a-Feijoo, M.; Eizaguirre, A.; Rica-Aspiunza, A. Systematic review of sustainable-development-goal deployment in business schools Sustainability 2020 , 12 , 440. [ CrossRef ] 12 Edwards, D.B.; Sustarsic, M.; Chiba, M.; McCormick, M.; Goo, M.; Perriton, S. Achieving and Monitoring Education for Sustainable Development and Global Citizenship: A Systematic Review of the Literature Sustainability 2020 , 12 , 1383. [ CrossRef ] 13 Wiek, A.; Withycombe, L.; Redman, C.L. Key competencies in sustainability: A reference framework for academic program development Sustain. Sci 2011 , 6 , 203–218. [ CrossRef ] 14 Berzosa, A.; Bernaldo, M.; Fern á ndez-Sanchez, G. Sustainability assessment tools for higher education: An empirical comparative analysis J. Clean. Prod 2017 , 161 , 812–820. [ CrossRef ] 15 Caeiro, S.; Hamon, L.A.S.; Martins, R.; Aldaz, C.E.B. Sustainability Assessment and Benchmarking in Higher Education Institutions-A Critical Reflection Sustainability 2020 , 12 , 30. [ CrossRef ] 16 Caeiro, S.; Leal Filho, W.; Jabbour, C.; Azeiteiro, U Sustainability Assessment Tools in Higher Education Institutions: Mapping Trends and Good Practices around the World ; Springer: Berlin/Heidelberg, Germany, 2013 17 Findler, F.; Schönherr, N.; Lozano, R.; Stacherl, B. Assessing the Impacts of Higher Education Institutions on Sustainable Development—An Analysis of Tools and Indicators Sustainability 2018 , 11 , 59. [ CrossRef ] 18 Niedlich, S.; Bauer, M.; Doneliene, M.; Jaeger, L.; Rieckmann, M.; Bormann, I. Assessment of Sustainability Governance in Higher Education Institutions—A Systemic Tool Using a Governance Equalizer Sustainability 2020 , 12 , 1816. [ CrossRef ] 19 Ramos, T.; Pires, S.M. Sustainability assessment: The role of indicators. In Sustainability Assessment Tools in Higher Education Institutions ; Springer: Berlin/Heidelberg, Germany, 2013; pp. 81–99 20 Urbanski, M.; Leal Filho, W. Measuring sustainability at universities by means of the Sustainability Tracking, Assessment and Rating System (STARS): Early findings from STARS data Environ. Dev. Sustain 2015 , 17 , 209–220. [ CrossRef ] 21 Hammond, A.L.; Institute, W.R Environmental Indicators: A Systematic Approach to Measuring and Reporting on Environmental Policy Performance in the Context of Sustainable Development ; World Resources Institute: Washington, DC, USA, 1995; Volume 36 22 Coelho, P.; Mascarenhas, A.; Vaz, P.; Dores, A.; Ramos, T.B. A framework for regional sustainability assessment: Developing indicators for a Portuguese region Sustain. Dev 2010 , 18 , 211–219. [ CrossRef ] 23 Mascarenhas, A.; Coelho, P.; Subtil, E.; Ramos, T.B. The role of common local indicators in regional sustainability assessment Ecol. Indic 2010 , 10 , 646–656. [ CrossRef ] 24 Pint é r, L.; Hardi, P.; Bartelmus, P Sustainable Development Indicators: Proposals for the Way Forward Prepared for the United Nations Division for Sustainable Development ; International Institute for Sustainable Development: Winnipeg, MB, Canada, 2005 25 Dhakal, S.; Imura, H. Policy-based indicator systems: Emerging debates and lessons Local Environ 2003 , 8 , 113–119. [ CrossRef ] 26 Cave, M The Use of Performance Indicators in Higher Education: The Challenge of the Quality Movement ; Jessica Kingsley Publishers: London, UK, 1997 27 Kaiser, F. System-level indicators for higher/tertiary education: Some notes on requirements and use Syst. -Level Strateg. Indic Monit. High. Educ. Twenty-First Century 2003 , 31–35. Available online: https://research.utwente.nl/en/publications/systemlevel-indicators-for-highertertiary-education-some-notes-o (accessed on 20 January 2022) 28 Nuttal, D. Choosing Indicators. In Making Education Count, Developing and Using International Indicators ; OECD: Paris, France, 1994; pp. 37–53.

[Find the meaning and references behind the names: Dos Santos, Wells, Sari, Eustachio, Santos, Press, Khalid, Mission, Caldana, Europa, Heiss, Katainen, Marope, Anand, Christ, Promise, Lex, Kieu, Elgar, Gannon, Webster, Lukman, Uri, Teles, Barber, Edward, Lauder, Ghidini, Cham, Agnelli, Poza, Salvia, Byun, Comm, Back, Celani, Culture, Junior, Merello, Fall, Kovaleva, Leicht, Timmermans, Oxford, Ragazzi, Freitas, Singer, Amor, Legal, Lambrechts, Shen]

Sustainability 2022 , 14 , 5102 19 of 20 29 U-Multirank. U-Multirank. Available online: https://www.umultirank.org/ (accessed on 28 October 2021) 30 Van Vught, F.A.; Ziegele, F Multidimensional Ranking: The Design and Development of U-Multirank ; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2012; Volume 37 31 Brundtland, G.H.; Khalid, M.; Agnelli, S.; Al-Athel, S.; Chidzero, B Our Common Future ; Oxford University Press: New York, NY, USA, 1987 32 Rieckmann, M Education for Sustainable Development Goals: Learning Objectives ; Unesco Publishing: Paris, France, 2017 33 Timmermans, F.; Katainen, J. Reflection Paper Towards a Sustainable Europe by 2030 Eur. Comm 2017 . [ CrossRef ] 34 Council of the European Union. Council recommendation of 22 May 2018 on key competences for lifelong learning (text with EEA relevance) Off. J. Eur. Union 2018/C 189/01 2018 , 1–13. Available online: https://eur-lex.europa.eu/legal-content/EN/TXT/ PDF/?uri=CELEX:32018 H 0604(01) (accessed on 28 October 2021) 35 European Commission European Universities: A Key Pillar of the European Education Area ; European Commission: Brussels, Belgium, 2019; p. 2 36 The Baltic University Programme. Forum for SDG’s in Finland. Available online: https://bup.fi/forum-for-sdgs-in-finland/ (accessed on 17 February 2022) 37 Leicht, A.; Heiss, J.; Byun, W.J Issues and Trends in Education for Sustainable Development ; UNESCO Publishing: Paris, France, 2018; Volume 5 38 UNESCO Education for Sustainable Development—A Roadmap, ESD for 2030 ; Unesco Publishing: Paris, France, 2020 39 Anand, C.K.; Bisaillon, V.; Webster, A.; Amor, B. Integration of sustainable development in higher education—A regional initiative in Quebec (Canada) J. Clean. Prod 2015 , 108 , 916–923. [ CrossRef ] 40 Kieu, T.K.; Singer, J.; Gannon, T.J. Education for sustainable development in Vietnam: Lessons learned from teacher education Int. J. Sustain. High. Educ 2016 , 17 , 853–874. [ CrossRef ] 41 Sonetti, G.; Barioglio, C.; Campobenedetto, D. Education for Sustainability in Practice: A Review of Current Strategies within Italian Universities Sustainability 2020 , 12 , 5246. [ CrossRef ] 42 Wu, Y.-C.J.; Shen, J.-P. Higher education for sustainable development: A systematic review Int. J. Sustain. High. Educ 2016 , 17 , 633–651. [ CrossRef ] 43 Hazelkorn, E. Are Rankings (Still) Fit for Purpose? In The Promise of Higher Education ; Springer: Cham, Switzerland, 2021; pp 293–299 44 Hazelkorn, E.; Mihut, G. Introduction: Putting rankings in context looking back, looking forward. In Research Handbook on University Rankings ; Edward Elgar Publishing: Cheltenham, UK, 2021 45 Kehm, B.M. Global university rankings—Impacts and unintended side effects Eur. J. Educ 2014 , 49 , 102–112. [ CrossRef ] 46 Marope, P.T.M.; Wells, P.J.; Hazelkorn, E Rankings and Accountability in Higher Education: Uses and Misuses ; Unesco: Paris, France, 2013 47 Sarrico, C.S.; Godonoga, A. Higher education system rankings and benchmarking. In Research Handbook on University Rankings ; Edward Elgar Publishing: Cheltenham, UK, 2021 48 De la Poza, E.; Merello, P.; Barber á , A.; Celani, A. Universities’ Reporting on SDGs: Using THE Impact Rankings to Model and Measure their Contribution to Sustainability Sustainability 2021 , 13 , 2038. [ CrossRef ] 49 Galleli, B.; Teles, N.E.B.; dos Santos, J.A.R.; Freitas-Martins, M.S.; Junior, F.H. Sustainability university rankings: A comparative analysis of UI green metric and the times higher education world university rankings Int. J. Sustain. High. Educ 2021 , 23 , 404–425 [ CrossRef ] 50 Lauder, A.; Sari, R.F.; Suwartha, N.; Tjahjono, G. Critical review of a global campus sustainability ranking: GreenMetric J. Clean Prod 2015 , 108 , 852–863. [ CrossRef ] 51 Ragazzi, M.; Ghidini, F. Environmental sustainability of universities: Critical analysis of a green ranking Energy Procedia 2017 , 119 , 111–120. [ CrossRef ] 52 Cortese, A.D. The critical role of higher education in creating a sustainable future Plan. High. Educ 2003 , 31 , 15–22 53 Lozano, R. Incorporation and institutionalization of SD into universities: Breaking through barriers to change J. Clean. Prod 2006 , 14 , 787–796. [ CrossRef ] 54 Farnell, T.; Veidemane, A.; Westerheijden, D. Piloting the TEFCE Community Engagement Toolbox at the University of Twente. In Re-Envisioning Higher Education’s Public Mission ; Springer: Cham, Switzerland, 2020; pp. 187–203 55 Lozano, R.; Lukman, R.; Lozano, F.J.; Huisingh, D.; Lambrechts, W. Declarations for sustainability in higher education: Becoming better leaders, through addressing the university system J. Clean. Prod 2013 , 48 , 10–19. [ CrossRef ] 56 Kolenick, P. Waiting for Godot: Leadership for sustainability in higher education and the emergence of Regional Centres of Expertise (RCEs) J. Sustain. Educ 2017 , 16 . Available online: http://www.susted.com/wordpress/wp-content/uploads/2018/0 1/Kolenick-JSE-Fall-2017-General-PDF 1.pdf (accessed on 17 February 2022) 57 Leal Filho, W.; Eustachio, J.H.P.P.; Caldana, A.C.F.; Will, M.; Salvia, A.L.; Rampasso, I.S.; Anholon, R.; Platje, J.J.; Kovaleva, M. Sustainability Leadership in Higher Education Institutions: An Overview of Challenges Sustainability 2020 , 12 , 3761. [ CrossRef ] 58 Bauer, M.; Niedlich, S.; Rieckmann, M.; Bormann, I.; Jaeger, L. Interdependencies of culture and functions of sustainability governance at higher education institutions Sustainability 2020 , 12 , 2780. [ CrossRef ] 59 Giesenbauer, B.; Müller-Christ, G. University 4.0: Promoting the transformation of higher education institutions toward sustainable development Sustainability 2020 , 12 , 3371. [ CrossRef ]

[Find the meaning and references behind the names: De Souza, Razak, Files, Clark, Raath, Haddad, Life, Bottani, Craciun, Vargas, Tam, Gov, Scott, Mak, Mader, Harder, Eric, Lazzarini, Get, Gibbs, Souza, Oaks, July, Lies, Sage, Hoover]

Sustainability 2022 , 14 , 5102 20 of 20 60 Hoover, E.; Harder, M.K. What lies beneath the surface? The hidden complexities of organizational change for sustainability in higher education J. Clean. Prod 2015 , 106 , 175–188. [ CrossRef ] 61 Leal Filho, W.; Raath, S.; Lazzarini, B.; Vargas, V.R.; de Souza, L.; Anholon, R.; Quelhas, O.L.G.; Haddad, R.; Klavins, M.; Orlovic, V.L. The role of transformation in learning and education for sustainability J. Clean. Prod 2018 , 199 , 286–295. [ CrossRef ] 62 Mader, C.; Scott, G.; Razak, D.A. Effective change management, governance and policy for sustainability transformation in higher education Sustain. Account. Manag. Policy J 2013 , 4 , 264–284. [ CrossRef ] 63 Gibbs, G Dimensions of Quality ; Higher Education Academy York: York, UK, 2010 64 Tam, M. Measuring quality and performance in higher education Qual. High. Educ 2001 , 7 , 47–54. [ CrossRef ] 65 Creswell, J.W.; Clark, V.L.P Designing and Conducting Mixed Methods Research ; Sage Publications: Thousand Oaks, CA, USA, 2017 66 Veidemane, A.; Kaiser, F.; Craciun, D. Inclusive Higher Education Access for Underrepresented Groups: It Matters, But How Can Universities Measure It? Soc. Incl 2021 , 9 , 44–57. [ CrossRef ] 67 Bottani, N.; Tuijnman, A. International education indicators: Framework, development and interpretation Mak. Educ. Count: Dev. Using Int. Indic 1994 , 21–35. Available online: https://files.eric.ed.gov/fulltext/ED 411322.pdf#page=21 (accessed on 17 February 2022) 68 Times Higher Education. THE Impact Rankings 2020: Methodology. Available online: https://www-timeshighereducation-com. ezproxy 2.utwente.nl/impact-rankings-2020-methodology (accessed on 15 March 2021) 69 Times Higher Education. Impact Ranking 2020. Available online: https://www.timeshighereducation.com/rankings/impact/20 20/overall#!/page/0/length/25/sort_by/rank/sort_order/asc/cols/undefined (accessed on 23 July 2020) 70 Times Higher Education. THE Impact Rankings 2020 by SDG: Climate Action (SDG 13) Methodology. Available online: https: //www.timeshighereducation.com/impact-rankings-2020-sdg-climate-action-sdg-13-methodology (accessed on 23 July 2020) 71 Times Higher Education. THE Impact Rankings 2020 by SDG: Partnerships for the Goals (SDG 17) Methodology. Available online: https://www.timeshighereducation.com/impact-rankings-2020-sdg-partnerships-goals-sdg-17-methodology (accessed on 23 July 2020) 72 Times Higher Education. THE Impact Rankings 2020 by SDG: Quality Education (SDG 4) Methodology. Available online: https: //www.timeshighereducation.com/impact-rankings-2020-sdg-quality-education-sdg-4-methodology (accessed on 23 July 2020) 73 Times Higher Education. THE Impact Rankings 2020 by SDG: Clean Water and Sanitation (SDG 6) Methodology. Available online: https://www.timeshighereducation.com/impact-rankings-2020-sdg-clean-water-and-sanitation-sdg-6-methodology (accessed on 23 April 2020) 74 Times Higher Education. THE Impact Rankings 2020 by SDG: Affordable and Clean Energy (SDG 7) Methodology. Available online: https://www.timeshighereducation.com/impact-rankings-2020-sdg-affordable-and-clean-energy-sdg-7-methodology (accessed on 23 July 2020) 75 Times Higher Education. THE Impact Rankings 2020 by SDG: Life below Water (SDG 14) Methodology. Available online: https: //www.timeshighereducation.com/impact-rankings-2020-sdg-life-below-water-sdg-14-methodology (accessed on 23 July 2020) 76 UI Green Metric World University rankings. Welcome to UI Green Metric. Available online: https://greenmetric.ui.ac.id/about/ welcome (accessed on 12 January 2020) 77 UI Green Metric World University Rankings. FAQ. Available online: https://greenmetric.ui.ac.id/about/faq (accessed on 12 January 2020) 78 UI Green Metric World University Rankings. Criteria & Indicators. Available online: https://greenmetric.ui.ac.id/about/ methodology (accessed on 12 January 2020) 79 Studenten Voor Morgen. SustainaBul. Available online: https://www.studentenvoormorgen.nl/en/sustainabul/ (accessed on 23 July 2020) 80 Studenten Voor Morgen Benchmarkrapport SustainaBul 2020, De Ranglijst Voor Nederlands Hogher Onderwijs op Het Gebeid van Duurzaamheid Door Studenten voor Morgen ; SustainaBul: The Netherlands, 2020 81 Green Office Movement. Green Office Model. Available online: https://www.greenofficemovement.org/ (accessed on 15 March 2021) 82 Green Office Movement. Get Started with Sustainability Assessment at Your University. Available online: https://www. greenofficemovement.org/sustainability-assessment/ (accessed on 23 July 2020).

Other Environmental Sciences Concepts:

[back to top]

Discover the significance of concepts within the article: ‘Education for Sustainable Development in Higher Education Rankings’. Further sources in the context of Environmental Sciences might help you critically compare this page with similair documents:

July, Education, Learning, Survey, Validity, Indicator, Higher education, Data collection, Starting point, Assessment, Climate change, National system, Quality education, Institutional leaders, National context, Teacher training, Sustainable development, European Union, Social inclusion, Focus Group, Response rate, Sustainable management, Assessment tool, Teaching Staff, Environmental sustainability, Extracurricular activities, Professional development, Clean energy, Energy Efficiency, Local communities, Sustainability, Survey results, Data Collection Process, Geographic region, Operational Definition, Online questionnaire, Learning environment, Holistic assessment, Voluntary participation, Education program, Educational outcome, Sustainable development goal, Qualitative feedback, Data source, Global scale, Pedagogy, Learning outcome, Feasibility study, Local context, Learning content, Research universities, Academic Unit, Organisational management, Data availability, Higher education institution, Institutional learning, Feasibility, Higher education sector, Academic literature, Critical factor, Societal concern, Agenda 2030, Policy document, Climate Action, Policy papers, Stakeholder consultation, Institutional level, Operational measures, Operational management, Input variable, National qualification, Relevance, Education and Research, Student involvement, Key functions, Stakeholder, SDG, Potential use, Ranking, Societal engagement, International indicator, Stakeholder Group, Alternative option, Process variable, Output variable, European Green Deal, Green Campus, Education for Sustainable Development, Sustainability education, ESD, Policy priorities, University Administrators, Education quality, Global coverage, Initial teacher training, Sustainability assessment framework, Data collection practices, Indicator development, Relevant criteria, International ranking, Education indicators, Abstract definition, Learning goal, University ranking, Diverse profiles, Graduate outcomes, Available resource, Student network, Educational offering.