Weighting factor elicitation for sustainability assessment of energy technologies

Abstract

In this paper, an approach for sustainability assessment of innovative energy technologies is expanded by multi-criteria decision analysis (MCDA) methods to aggregate indicator results and support decision-making. One of the most important steps for MCDA is to determine weighting factors for individual indicators. Thus, a workshop was performed to elicit weighting factors for sustainability assessments of energy technologies from developers of such technologies and energy system modellers from academia. These stakeholders expressed their preferences with respect to sustainability criteria using the Simple Multi Attribute Rating Technique (SMART). A triple bottom line approach of sustainable development was used as the basis for the aggregation of indicator results. This approach is based on Life Cycle Costing, Life Cycle Assessment and social indicators. Obtained weighting factors were applied to an integrative sustainability assessment with the aggregation method Preference Ranking Organization METHod for Enrichment of Evaluations (PROMETHEE). Hydrogen-based mobility as an important technology to foster decarbonization in the transport sector is used as a case study for the application of the derived weighting factors. A conventional vehicle, powered by fossil fuel, is compared with a fuel cell electric vehicle (FCEV) for the year 2050. Different options (pipeline, compressed gaseous hydrogen, liquid hydrogen, liquid organic hydrogen carrier) are discussed for the supply of hydrogen. The results for this weighting factor set are compared with an equal weighting scenario of the three sustainability dimensions and indicators within one sustainability dimension. The FCEV, using pipelines for hydrogen supply, came out first in the assessment as well as in all sensitivity analyses.

---

Introduction

The transformation of the energy system is a prerequisite to meet the goals of the Paris agreement.¹ To enable this transformation process, innovative energy technologies are necessary. Hence, the European Commission pushed forward the European Green Deal, a concept for Europe to become climate neutral by 2050 and to transform the EU's economy in a sustainable manner.² In order to analyse greenhouse gas reduction potentials without losing track of other associated effects, a comprehensive sustainability assessment is necessary. Besides other environmental impacts, this should also include economic as well as social impacts.³ The interpretation of such a sustainability assessment, however, is challenging because many different single results, i.e. indicator results with different units of measure, are obtained, and it is based thereon not possible to propose one unambiguous solution unless one alternative performs best with respect to all indicators. Finding the best alternative including a set of sustainability indicators within a decision-making processes is a very demanding task: not only indicators within one sustainability dimension, e.g. different environmental indicators, must be put in relation to each other, but also indicators referring to different sustainability dimensions, i.e. environmental, economic, and social indicators if using the triple-bottom line concept of sustainability.⁴ In order to aggregate single indicator results within the framework of a comprehensive sustainability assessment, mathematical procedures can be used, for which the individual indicators generally need to be weighted in a first step. In that sense, Multi-Criteria Decision Analysis (MCDA) methods, developed to guide different decision-making processes, can support and facilitate the interpretation of sustainability assessments. MCDA comprises mathematical approaches to cluster a wide number of individual results to less, but better manageable results.^5,6 Already Hannouf and Assefa⁷ proposed a framework to couple Life Cycle Sustainability Assessment (LCSA) with a decision analysis. Irrespective of the choice of an MCDA approach, the question of different weighting of sustainability indicators remains. In a first approximation, equal weighting of indicators can be combined with a sensitivity analysis to identify tipping points of preferences. Other generic approaches deriving weighting factors include context-specific weighting sets⁸ or standardized profiles.⁹ Apart from these generic approaches, MCDA encourages the integration of stakeholders, e.g. technology developers, citizens, political decision-makers, in different stages of the MCDA process,¹⁰ in particular for the determination of weighting factors for criteria and indicators. This is a crucial step in the MCDA process because it allows the integration of stakeholder values of the stakeholders into the decision-making process. Even if no specific weighting factors are used and every indicator is equally important this is still an important decision that should be made consciously.

Our earlier developed approach for sustainability assessment,¹¹ drawing on the triple-bottom-line approach, offers a comprehensive way to assess energy technologies based on a life cycle perspective. Even though this approach mentions MCDA in order to simultaneously consider the assessed indicators, no recommendations for its implementation are given. In this work, this sustainability assessment is extended using a participatory approach for the determination of weighting factors for sustainability dimensions and respective indicators, involving technology developers and energy system modellers from academia for preference elicitation. The collected weighting factor set is to be used to aggregate the indicator results into an overall performance value within a general framework of sustainability assessment of energy technologies. To account for the triple-bottom-line model of sustainability, a hierarchical weighting approach is obvious, i.e. at first, indicators of one and the same sustainability dimension are related to each other and second, the three sustainability dimensions are related to each other. In this paper, the resulting set of weighting factors is tested on a first case study on hydrogen mobility.

The hydrogen mobility comprises several technologies. The basis are hydrogen vehicles, e.g. buses, trucks, trains, passenger cars, mostly powered by a fuel cell, seldomly by an internal combustion engine, which has a lower efficiency compared to a fuel cell. Hydrogen mobility has the chance to be climate friendly when the hydrogen is produced from renewable energy sources, e.g. wind or solar power.¹² It is one technology to enable the electrification of sectors, where direct electrification is not possible. Hydrogen is easier to store in large quantities over long periods than electricity. Furthermore, it is discussed frequently as a corner stone in energy transformation scenarios^13,14 and current German scenarios show that larger shares of green hydrogen in the system can significantly reduce the import quota of energy.¹⁵

In this case study the focus is on passenger fuel cell electric vehicles (FCEVs), because they are one of the hydrogen technologies that are further developed compared to e.g. hydrogen trains. They emit only water at the point of use and therefore – in addition to lowering impacts on climate change – help reducing emission levels in cities, e.g. particulate matter or nitrogen oxides.¹⁶ However, substantial energy amounts are lost in the hydrogen supply chain during production as well as transport and distribution.¹² Furthermore, trade-offs between mineral resource depletion as well as economic and social impacts occur.¹⁷ Hydrogen refuelling stations are still a technology of concern for many people¹⁸ and currently FCEV are much more expensive in its purchase as well as its operation than comparable vehicles with an internal combustion engine (ICE).¹⁹ Thus, a thorough sustainability assessment of the use of FCEVs is necessary in comparison to a convention gasoline ICE vehicle. For the supply of the hydrogen from renewable sources for the FCEV many different options are available.¹² We decided to focus on supply within Germany, because energy imports have a large political component that are difficult to address. Furthermore, we are only looking at green hydrogen, i.e. hydrogen from water electrolysis, because this is the preferred option for hydrogen production from renewable sources in Europe and Germany.^20,21 Even with these constraints hydrogen supply still has several technology options regarding its transport, which will be part of this case study.

Our previously developed approach for sustainability assessment¹¹ is used to carry out an indicator-based sustainability assessment of different hydrogen mobility options in Germany. Within the framework of the extended approach, weighting factors for sustainability indicators and dimensions are determined through a stakeholder survey and are used for the aggregation of indicators with the MCDA method Preference Ranking Organization METHod for Enrichment of Evaluations (PROMETHEE). Before the (extended) approach for sustainability assessment is introduced in more detail and is subsequently applied to a case study on hydrogen mobility, the current discussion on weighting factor determination is presented.

State of the art of weighting factor determination

In recent years more and more publications regarding environmental and sustainability assessment discussed the use of MCDA and weighting factors in particular for their final evaluation. This includes theoretical approaches as well as the integration of stakeholders. Most of the publications follow a hierarchical structure. For the first hierarchical level a sustainability concept, e.g. the triple-bottom line concept, is used and provided with weighting factors. Then, as a second hierarchical level, each indicator within each sustainability dimension is provided with a weighting factor.

In this section the most important approaches for weighting factor determination are reviewed as a basis for the approach in this paper.

Thies et al.²² showed in their review how sustainability assessment can be complemented and improved by the use of MCDA and other operational research methods. According to their findings an important part of MCDA is the way of preference articulation (weighting) which profoundly impacts the entire decision-making process.²³

As an approximation, theoretical approaches can be applied. Haase et al.²⁴ used such an approach to perform a hierarchical equal weighting for the sustainability assessment comparing different types of passenger vehicles. On the first hierarchy level all three sustainability dimensions were considered equally important and on the second hierarchy level each indicator within a sustainability dimension were considered equally important. The sensitivity analysis proved that the ranking of the vehicles keeps constant even if the weighting factor for the sustainability dimensions (w_i = 0.33) were changed by ±0.10.

Instead of equal weighting, Ekener et al.⁹ proposed to use the stakeholder profiles individualist, hierarchist and egalitarian from cultural theory as a guideline for deriving weighting factors for the three sustainability dimensions within sustainability assessment. Since each of the three stakeholder profiles have different priorities regarding the sustainability dimensions, with the cardinal ranking approach different weighting factors were derived. For each profile the weighting factors of 0.60 (1st priority), 0.28 (2nd priority) and 0.12 (3rd priority) were assigned to the sustainability dimensions. Ekener et al.⁹ applied these weighting factors in a case study regarding four biofuels. In all three profiles the most sustainable fuel stayed the same, but rankings of the other three assessed fuels changed when applying a different stakeholder profile.

For specific decision-making situations it can be better to integrate relevant and – if possible – representative stakeholders to support a participatory and collaborative decision-making process. In this case, an interface that allows stakeholders to express their preferences regarding the selected criteria is required.²⁵ In this way, MCDA methods allow to grasp strategic intelligence of a group of stakeholders which bring together different experiences, knowledge, as well as expectations. This offers a way to identify preferences of the group related to a specific problem. Furthermore, MCDA methods also allow to identify if opinions differ and to enable a process to resolve potential discrepancies.²⁶ There are several forms to reach the participants ranging from interviews, decision conferencing, to online surveys.²³ The selection of the method is dependent on several factors as the number of stakeholders that are involved in the process, available time and money as well as complexity of the problem. Several methods for weighting are available, e.g. methods based on trade-offs (Simple Multi Attribute Rating Technique (SMART)),²⁷ direct rating, lotteries and pairwise comparisons, e.g. the Analytic Hierarch Process (AHP).²⁸ While SMART and the AHP are methods to define weights that are globally applicable, other methods like the conjoint analysis and the discrete choice experiment are based on the respective use cases.^29,30

With respect to environmental indicators, Sala et al.⁸ developed a generic weighting factor set for environmental footprint categories. They conducted surveys among different stakeholder groups, i.e. lay people, LCA experts, as well as a workshop with life cycle impact assessment experts. Those results are combined with a robustness factor for each environmental indicator. This weighting factor set includes 16 different environmental indicators, from which climate change is regarded by far as most important with 21.06 (on a scale to 100). Human toxicity, non-cancer has the lowest weighting factor of 1.84, which is heavily influenced by the included robustness factor.

Tarne et al.²⁹ supported the sustainability assessment of car manufacturing by asking decision makers in a German automotive company with the help of a limit conjoint analysis. Even though the individual results from the 54 participants had a wide range, the average weighting factors for the three sustainability dimensions were close together (environment w_env = 0.352, economy w_eco = 0.335 and social w_soc = 0.312). The derived weighting factors are representative regarding: (i) geographic location, (ii) time, (iii) subject and (iv) perspective. In this case the representativeness is given for (i) Germany, (ii) the year 2018, (iii) cars and (iv) manufactures. This example illustrates the limited applicability and transferability of results gathered through stakeholder integration into sustainability assessment in general.

The papers mentioned so far were guided by the triple-bottom-line approach of sustainability, i.e. environment, economy and society. If a different understanding of sustainability is chosen, results might look much different: for the sustainability assessment of German energy system transformation pathways, Naegler et al.³⁰ conducted a discrete choice experiment with citizens. Unlike many other studies, sustainability was not assumed as three dimensions, but described by seven indicators, i.e. climate change, human health, resources (land), resources (mineral, metals, fossils), system costs, security of supply and employment. The elicited results were adjusted with a robustness factor leading to the dominating weight of climate change (CC) with w_CC = 0.532, which subsequently defined the results of their sustainability assessment.³⁰

None of the sources mentioned elaborate weighting factor sets for both the sustainability dimensions and corresponding specific indicators within the respective dimensions of our approach. Furthermore, none of the surveys took place in the context of energy system transition in Germany.

Approach for sustainability assessment including weighting factor determination and indicator aggregation

In this paper, the earlier mentioned approach for prospective sustainability assessment of energy technologies according to Haase et al.¹¹ was extended. It comprises Life Cycle Assessment (LCA) and Life Cycle Costing (LCC) and corresponding indicators as well as social indicators, derived from a normative concept of sustainable development.¹¹ Furthermore, MCDA was applied for the assessment via two steps, see Fig. 1: (I) preferences, i.e. weighting factors for sustainability indicators and dimensions, were determined through a participatory stakeholder survey and (II) sustainability indicators were aggregated through outranking including the weighting factors from the stakeholder survey. Therefore, the MCDA methods SMART for the stakeholder survey and PROMETHEE for aggregation of sustainability indicators were chosen. All methods used are further described in the following subsections. Lists of the considered criteria and indicators can be found in Table 1. In this paper, a sustainability criterion might contain several indicators describing the criterion.

Fig. 1 Extended approach for sustainability assessment including weighting factor determination and aggregation of sustainability indicators (adapted from Haase et al.¹¹).

Table 1 Overview of sustainability indicators; CTU: comparative toxic unit; PPP: purchasing power parity

Criteria	Indicator	Unit
a Not considered in this study.
Climate change	Climate change	kg CO2 eq.
Acidification	Acidification	Molc H^+ eq.
Human toxicity	Humantox, cancer	C TUh
	Humantox, non-cancer	C TUh
Ionizing radiation	Ionizing radiation	kg U235 eq.
Ecotoxicity	Ecotoxicity	C TUe
Eutrophication	Freshwater	kg P eq.
	Marine	kg N eq.
	Terrestrial	mol N eq.
Ozone depletion	Ozone depletion	kg CFC-11 eq.
Particulate matter	Particulate matter	kg PM2.5 eq.
Photochemical ozone formation (POF)	POF	kg C2H4 eq.
Resource depletion, mineral, fossil and renewable	Resource depletion	kg Sb eq.
Levelized total cost	Levelized total cost	€ per km
Domestic value added	Domestic value added	%/value added
Innovation potential	Patent intensity	Patents/PPP
	Patent growth rate	%/time span
	National technology share	%/national patents
Acceptance	—^a	—

---

Environmental assessment

For the environmental assessment an LCA was carried out. With an LCA two or more alternatives of a product, service or organisation can be assessed concerning the potential impact on ecosystems, human health and resources. The premise of this methodology is that environmental impacts are not limited to the production process itself (foreground). They may also occur in the background, i.e. pre-chains. This includes for example electricity generation or steel production. Here an LCA following ISO 14040 and 14044 was carried out: after goal and scope definition, the so-called Life Cycle Inventory (LCI) is build-up, which includes all resource consumption and emissions along the value chains under consideration. Based thereon, Life Cycle Impact Assessment (LCIA) was carried out and results were interpreted. In the case study (see section Case study: hydrogen mobility), the upstream and downstream processes, e.g. raw materials supply, provision of operating materials and infrastructure, waste and wastewater disposal, and product use, were modelled via the open source software openLCA v1.7 together with datasets from the commercial ecoinvent database v3.3 (cut-off-system model).³¹ As far as possible, specific datasets for Germany (DE) were used. If no datasets were available for Germany, datasets for Switzerland (CH), Europe (RER), or worldwide datasets (GLO) were used. For LCIA, 13 environmental impact categories and corresponding indicators were applied at midpoint level as recommended in the ILCD Handbook of the European Commission³² and implemented in the software openLCA (LCIA methods v2, ILCD 2011, midpoint).³³

Economic assessment

For the economic assessment, LCC was used. LCC aims at assessing all costs related to a product over its entire life cycle, i.e. purchase, use, disposal and recycling, respectively.³⁴ Goal and scope definition is similar to that of an LCA. If LCC is used in parallel with LCA, system boundaries of LCC need to be equivalent to system boundaries of LCA and identical functional units should be used.³⁴ Different parts of the product system may fall below relevant cut-off criteria for the separate LCC and LCA components. For example, early research and development may impose significant costs but little environmental impact.³⁴ As cost data may be gathered in different currencies and reflect different time periods, economic inventory data needs to be adjusted to a common currency and reference year using appropriate exchange and discount rates.³⁴ There is no comparable impact assessment phase in an LCC, because all inventory data comprise a single unit of measure, namely currency.³⁴ Procedures for interpretation, communication, and review are analogous to those for LCA. If LCC analysis are carried out from the user's perspective, the term “total costs of ownership” (TCO) is commonly used.³⁵ According to VDI,³⁶ life cycle costs can be divided into the three stages “before utilization”, “during utilization,” and “after utilization” or three different costs types “Capital Expenditures” (CAPEX), “Operational Expenditures” (OPEX), “End Of Life Expenditures” (EOLEX). Within this study, levelized total costs (LTC) was used as an economic indicator, cf. Haase et al.,¹¹ comprising CAPEX and OPEX. Furthermore, in this LCC no external costs, taxes, or subsidies were considered.³⁷

Social assessment

For the social assessment, three criteria, i.e. acceptance, domestic value added and innovation potential and corresponding indicators were considered based on Haase et al.¹¹ Since these indicators are not standardised yet, their background and assessment are described in more detail below.

Acceptance. The acceptance of energy technologies is a prerequisite for the development and application of the same.³⁸ According to Assefa and Frostell,³⁹ there is an association between what the public feels and thinks about a technology and its knowledge and they emphasize on three indicators, i.e. knowledge, perception, and fear for the assessment of social acceptance. Depending on different factors, e.g. depth and type of information needed, representativeness, or constraints regarding time and costs, different methods can be chosen for data collection. These methods encompass for example in-person interviews or web-based questionnaires.³⁹ In this approach, acceptance is investigated using an online survey based on the methodological background of Huijts et al.⁴⁰ and Miguel et al.⁴¹ After a short description of the respective technology, nine different types of concerns are put to selection, including an open field for further concerns. Furthermore, socio-demographic data is collected regarding gender, residence, income, activity, age, and education. The freely available online platform SoSci-Survey was used for conducting the survey. A more detailed description of the approach can be found in Emmerich et al.⁴²

Due to practical issues this indicator cannot be considered in this study.

Patents as indicator for innovation potential. Patent-based indicators can serve as a proxy for the innovation potential.⁴³ This indicator guarantees protection of company knowledge but can also provide information about environmental benefits and produced social well-being⁴⁴ as well as promotes social change.^45,46 Here, a combination of patent-based indicators is used based on Baumann et al.;⁴³ (i) patent intensity patenting activity in a certain technology field per country and respective GDP – for the GDP the purchase power parity (PPP) adjusted form was used; (ii) patent growth rate (in %) and (iii) national technology share (relative R&D emphasis of a country related to a single technology). The patent intensity mirrors the national patenting activity in relation to a country's economic growth. A high patent growth rate is interpreted as a high innovation potential due to increased research effort in the area, whilst a high national technology share indicates a strong focus of R&D of a country.⁴⁷ The European Patent Office (EPO) database including the Open Patent Service (OPS) was used for patent analysis. For patent search and data analysis, an adopted and freely available python-based patent database crawler together with an MS Excel template was used.⁴³ The patent search was carried out for the time period from 1995 to 2018 using Cooperative Patent Classification (CPC)-codes and keywords (see ESI Table 1 for details†). Based on the results, the template selected the five most active patenting countries in the considered technology field and compares them to Germany. The different patent indicators are presented in a portfolio analysis that includes the relevant countries for the selected technology.

The goal of these indicators is to assess the innovation potential that a technology has for a specific country. Thus, in a comparison between different technologies a higher innovation potential is considered better. In this study, the focus was on Germany. To quantify the innovation potential in a country comparison, the different indicator results are ranked from 1–5 (R_i, the lower the better). For each country under consideration the mean rank of the indicators (n) is calculated R̄ (eqn (1)). Here, three indicators were analysed, thus n = 3.

Domestic value added. Assuming that local investment, i.e. locally produced goods, creates or secures local jobs, the fraction of domestic value added was used here as an indicator for local job creation potential.¹¹ This indicator aims to give an indication whether the energy technology might have a positive effect on job development compared to another (conventional) technology. In this study, the fraction of domestic value added was based on methods, data and results from the economic assessment.⁴⁸ System boundaries, therefore, correspond to those of LCC. For each cost component, considerations were made on the fraction of domestic value added, including information on the country's raw material deposits, located industries and their capabilities. If necessary, cost components were further specified, i.e. the technology under consideration is split up into its modules and all life cycle phases.⁴⁸ Then, percentages of domestic value added were estimated for each cost component and summed up to the fraction of domestic value added of total costs.^11,48 A detailed description of the method can be found in Harzendorf et al.⁴⁸

Multi-criteria decision analysis. MCDA methods allow to organize available information, to explore perceptions and needs, and to identify consequences of a decision to support involved decision makers.⁴⁹ Decision problems are expressed in form of equations, the simplest form and widely known approach would be the weighted sum, inputs, e.g. LCA results, and coefficients, e.g. weighting factors, which can be observed and reproduced. Depending on the problem at hand, stakeholder integration can, should or must be done. In general, MCDA can be divided into two major sequences, which can be overlapping, and which are characterized by an iterative process as follows:¹⁰

(1) Definition of the decision problem, alternatives, and corresponding criteria and indicators as well as selection of suitable (MCDA) methods, preferably in cooperation with stakeholders

(2) Aggregation of indicator or criteria results using weighting factors, preferably from stakeholders, and suitable MCDA methods, corresponding interpretation and analysis of aggregation results and decision process.

Within the second step, it is highly important to select suitable methods that fit to the given type of problem. One well-known method is the weighted sum approach, which is very easily manageable, but which lacks to cover more complex decision-making contexts. In contrast, elaborated MCDA approaches like Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) or PROMETHEE or ELimination Et Choix Traduisant la REalité (ELECTRE), can address more complex problems, for example including thresholds for indicator values. Each of these methods has its strengths and weaknesses and should be selected with care for each assessment.⁵⁰

Within this approach the MCDA procedure according to Fig. 1 is applied:

(I) Determination of weighting factors with an participatory approach: SMART.²⁷ The advantage of SMART is its general low complexity. At the same time the chance of bias is less pronounced compared to other approaches with low complexity.⁵¹

(II) Aggregation of indicator results: PROMETHEE.⁵² Indicator compensation should be avoided. Easiest application for a complex outranking method.⁵³

SMART. The execution of SMART may differ.²⁷ For the approach here, the emphasis lies on simplicity for the included stakeholders. Thus, the following procedure was chosen. The most important indicator receives 100 points. Subsequently, the other indicators are assigned scores, which express the relative importance to the most important criterion. This results in a score range of 0 to 99 points for the remaining indicators. Subsequently, the scores of the individual indicators are then added up to give a total score. The weighting values are then derived from the relative proportion of the total score. The calculation is set out in eqn (2).with w_i = weighting value in percent for the respective criterion, P_i = score of the respective criterion.^27,54

PROMETHEE. PROMETHEE is a family of outranking (A is better than B, not concerning how much better A is compared to B) methods developed by Brans and colleagues in the early 1980s.^52,55 Since then, several variations of this method have been developed. This includes PROMETHEE I–VI.⁵⁶ The most relevant variations are PROMETHEE I and II, which will be used in this paper.

The principle of PROMETHEE is based on a pairwise comparison of alternatives along each criterion. PROMETHEE I gives a partial ranking of the alternatives with the outranking flows Φ⁺ and Φ⁻. The higher Φ⁺ and the lower Φ⁻ are, the better is the overall rank of the analysed option. However, this can also lead to incomparabilities when Φ⁺ and Φ⁻ indicate different preferences. PROMETHEE II adds a step to derive a complete ranking of the alternatives (outranking flow Φ) by calculating the difference between the two flows. This leads to a complete ranking under some degree of detail loss.⁵⁷ To account for very small differences in the pairwise comparisons, preference functions are introduced. They translate the difference between the indicator results obtained by two alternatives into a preference degree ranging from zero to one. In this way, the users can implement their opinion on what preference actually means (A is considered only better as B, when A is at least, e.g., 5% different from B). Based on the analysed indicator, different functions have their purpose regarding the level of uncertainty and the nature of the values, i.e. qualitative, discrete or continuous.⁵⁸

Implementation. The implementation of SMART and PROMETHEE is presented in the following two sections.
Participatory approach for deriving weighting factors using SMART. Researchers and developers of technologies, with a focus on the energy transition to a low carbon future, were asked on their opinion about the importance of sustainability criteria for the assessment of emerging technologies, e.g. biomass, electricity transmission, energy storage and hydrogen-based technologies in the context of the German energy transition. The survey took place during a hybrid (onsite/online) conference with 89 attendees in September 2020. During a 50 minutes-long session the basics of the SMART method were presented as well as a short recap of the criteria, which were included in the survey. For each criterion the influencing factors, e.g. acidifying emissions, the impact pathways, e.g. lowering of the pH-value in the soil, and the final impacts, e.g. decrease in biodiversity, were illustrated. For this introduction of criteria it was considered that the audience came from academia with a technical/natural science background. Due to some technical issues, not all onsite attendees were able to participate. After each presentation the attendees were asked to participate in an online survey. In total three surveys were conducted.

At the first hierarchy level, a weighting procedure based on the triple bottom line approach of sustainability⁴ was chosen. Participants were invited to give a weight to each of the three dimensions. They were asked ‘How important do you consider the following sustainability dimensions (economy, society, environment)?’. The second hierarchy level contains the indicators within each dimension of sustainability. As the economic dimension only contains one indicator, see subsection economic assessment, no further survey was necessary. Questions for environmental and social criteria were asked accordingly to the sustainability dimensions. It should be noted that this weighting factor elicitation was carried out independently from a specific energy technology or case study and is therefore applicable to other energy technologies.

To distinguish between the weighting factors of criteria within one of the three sustainability dimensions, i.e. environment, economy, and society, and the final weighting factors of all criteria, they are from here on forward referred to as local weighting factors, e.g. w_local,env,CC, and global weighting factors, e.g. w_global,CC. The graphical representation is depicted in Fig. 2. The global weighting factors (w_global,j) (with j representing the different sustainability criteria, e.g. climate change (CC)) are the product of the local weighting factor (w_local,ij) (with i representing the three sustainability dimensions) and the weight of the respective sustainability dimension w_Sus,i (eqn (3)).

w_global,j = w_Sus,i × w_local,ij (3)

with

Fig. 2 Combining weighting factors.

Following the approach for sustainability assessment described above, the environmental assessment with LCA included 13 indicators. Even though the SMART method allows the comparison of a higher number of aspects, asking stakeholders for the relative evaluation of 13 different indicators was assumed to be overwhelming the cognitive demand of the participants.⁵⁹ Thus, the 13 indicators were clustered to eight environmental criteria. It was assumed that the indicators within one criterion are equally important. The indicators ionizing radiation, human toxicity, cancer and human toxicity, non-cancer were clustered to one criterion “Human toxicity”. Furthermore, the three indicators eutrophication, freshwater, eutrophication, marine and eutrophication, terrestrial were clustered together with ecotoxicity into the criterion “ecotoxicity”. To derive the weighting factors for the 13 indicators based on the survey of the eight criteria, in a first step, all thirteen indicators within one criterion received the weight of the criterion (environmental indicator criterion weight (w_env,cj)). This violates the condition ∑w_j = 1 and results in a sum larger than one. Thus, in a second step, each environmental indicator criterion weight (w_env,cj) was divided by the summation of the indicator criterion weights (eqn (4)).

This resulted in the local environmental weighting factors.

Definition of the preference functions for PROMETHEE. In this paper, the values of the discussed criteria are continuous as well as discrete. For continuous results the linear preference function is most appropriate which requires the definition of thresholds q (indifference value) and p (preference value).⁶⁰ As innovation potential gives discrete values as result, i.e. technology A is better than technology B, the usual preference function is sufficient, and no thresholds need to be defined. As thresholds q and p for the environmental indicators (index i) 5 and 10% of the minimum value across all analysed options (A, …, N) were chosen, respectively, to account for any uncertainty,⁵⁸ see eqn (5).

q_i = 0.05 Min(A_i, B_i,…, N_i) and p_i = 0.10 Min(A_i, B_i,…, N_i) (5)

For the indicators levelized total costs and domestic value added, thresholds were lowered to 1 and 2% of the minimum value due to less uncertainty of the assessment methods. For acceptance only results on hydrogen refuelling are available and results on the ICE with gasoline are missing. It is worth mentioning, that there are studies on the topic as e.g. Brunner et al.⁶¹ However, the results are not comparable to Emmerich et al.⁴² as every acceptance study is carried out in a specific context (spatial, technology and aim) using different methods. Thus, acceptance could not be included in this case study and no thresholds needed to be defined.

For the execution of PROMETHEE the open access software visual PROMETHEE was used.

Results survey on weighting factors

First, the results for the survey on sustainability dimensions is presented followed by the results from the surveys on environmental and social criteria.

Weighting of sustainability dimensions

In total 60 answers were received for the survey on sustainability dimensions. The participants voted the environmental dimension of sustainability most important with a resulting weighting factor w_Sus,env of 0.385, Fig. 3. This is a significant higher value compared to equal weighting, i.e. w_Sus,i = 0.333, and by far greater than for the next dimension social with w_Sus,soc = 0.320. Economy ranked lowest among the three sustainability dimensions with w_Sus,eco = 0.295. The results of all three dimensions show a certain degree of variance over the 60 answers from the participants. With a standard deviation (μ_Sus,env) of 0.060 the weighting factor for the environmental dimension has the narrowest distribution. The opinions of the participant were more widely spread regarding the importance of the social dimension (μ_Sus,soc = 0.069) like the spread for the economic dimension with μ_Sus,eco = 0.068. As mentioned above, only one economic indicator is considered, thus, w_local,eco = 1 and w_global,eco = w_Sus,eco = 0.295.

Fig. 3 Survey results regarding weighting of sustainability dimensions, n = 60.

Environment

For environmental criteria, 48 answers were received within the survey. The most important environmental criterion for the researchers is climate change (w_CC = 0.169, ∑w_i = 1). At the same time, however, it also has the largest interquartile range (0.050), meaning that the votes from the participants are widely spread, see Fig. 4. The participants consider photochemical ozone formation (POF) least important (w_POF = 0.099) closely followed by acidification (w_Acid = 0.102). To derive the local environmental indicator weights (w_local,env,j) based on the results of the environmental criteria, eqn (4), was applied. All results are listed in Table 2.

Fig. 4 Results for local environmental weighting factors, n = 48, POF: photochemical ozone formation.

Table 2 Resulting weighting factors for environmental criteria and indicators

Criterion	Criteria weight	Indicator	Indicator criteria weight	Local weight	Global weight
Climate change	0.169	Climate change	0.169	0.101	0.039
Acidification	0.102	Acidification	0.102	0.061	0.023
Human toxicity	0.143	Human toxicity, cancer	0.143	0.085	0.033
		Human toxicity, non-cancer	0.143	0.085	0.033
		Ionizing radiation	0.143	0.085	0.033
Ecotoxicity	0.131	Ecotoxicity, freshwater	0.131	0.078	0.030
		Eutrophication, freshwater	0.131	0.078	0.030
		Eutrophication, marine	0.131	0.078	0.030
		Eutrophication, terrestrial	0.131	0.078	0.030
Ozone depletion	0.115	Ozone depletion	0.115	0.069	0.026
Particulate matter	0.105	Particulate matter	0.105	0.063	0.024
Photochemical ozone formation (POF)	0.099	Photochemical ozone formation	0.099	0.059	0.023
Resources	0.136	Resources	0.136	0.081	0.031
Sum	1.000		1.679	1.000	0.385

---

Social

The survey regarding the social criteria was answered by 55 participants. They consider the acceptance of emerging technologies by the general public as the most important social criterion. Domestic value added and innovation potential, received rather similar weighting factors, which are slightly lower than for acceptance, see Fig. 5. As for the subsequent assessment of the case study on hydrogen mobility the criterion acceptance was not considered due to the lack of data for the fossil reference. Thus, the number of criteria is reduced from three to two for the calculation of local weight and global weight, see Table 3. For the calculation of local and global weighting factors, the same method is applied as for the environmental criteria in section Implementation, see eqn (3) and (4). The resulting weighting factors for social indicators are listed in Table 3.

Fig. 5 Results for local social weighting factors, n = 55.

Table 3 Resulting weighting factors for social criteria

Criterion	Criteria weight	Indicator	Indicator criteria weight	Local weight	Global weight
Domestic value added	0.322	Fraction of domestic value added	0.322	0.516	0.165
Innovation potential	0.301	Three patent indicators	0.301	0.484	0.155
Acceptance	0.377		Not considered for case study
Sum	1.000		0.623	1.000	0.320

---

Case study: hydrogen mobility

The weighting factors derived from the participatory approach are tested for the first time on a case study on hydrogen mobility. After describing how hydrogen mobility is modelled, the detailed indicator results are presented. Finally, the MCDA is performed with the derived weighting factors to guide the decision process regarding the different hydrogen and fossil mobility options.

Modelling of hydrogen mobility

In this case study, a passenger FCEV was analysed in detail with different hydrogen supply chains and compared to a conventional ICE vehicle fuelled with gasoline. The boundary conditions were set for Germany for the year 2050. Further background information, e.g. electricity mix and costs, are taken from the Helmholtz-Alliance ENERGY-TRANS.⁶² The two different vehicle types have the same capacity of 100 kW. While the ICE vehicle was based on a VW Golf⁶³ the FCEV is based on a Toyota Mirai,⁶⁴ downscaled to 100 kW. More detailed information regarding the modelling of the vehicles can be found in Haase et al.²⁴ The hydrogen was produced in an alkaline water electrolyser⁶⁵ using wind power as an energy source. The LCI modelling of the wind was based on Schreiber et al.⁶⁶ For transport and distribution of hydrogen, different technologies are available. Currently, the most common transport methods are gaseous hydrogen in high-pressure tanks (compressed gaseous hydrogen CGH₂) and liquid hydrogen (LH₂) in cryogenic tanks by truck. Alternatively, hydrogen storage and transport in liquid organic hydrogen carriers (LOHCs) by truck was considered. The fourth alternative analysed is the construction of a new hydrogen pipeline network in Germany. To be not susceptible to wind power fluctuations, the hydrogen needs to be stored, if necessary, for months. Therefore, for gaseous hydrogen seasonal storage in salt caverns was considered. Liquid hydrogen, as well as hydrogen in LOHCs, can be stored in appropriate tanks. The most important technical parameters are summarized in Wulf et al.⁶⁷ An overview of the system boundaries and the different process steps of the four FCEV options (CGH₂, pipeline, LH₂, LOHC) and the ICE vehicle option (gasoline) are depicted in Fig. 6.

Fig. 6 System boundaries of assessed hydrogen mobility and fossil reference options.

Indicator results

The sustainability assessment of hydrogen mobility was based on earlier publications on LCA, LCC and selected social indicators.^{43,48,67–69}

Life Cycle Assessment of hydrogen mobility. The functional unit of the LCA was one driven vehicle kilometre. The results are depicted in Fig. 7 relatively to the conventional ICE vehicle fuelled by gasoline. In four impact categories, hydrogen mobility has clear advantages compared to the fossil ICE vehicle. Next to the categories climate change and resources, which – of course – benefit from the switch from mineral oil to wind power as primary energy source, ionizing radiation and ozone depletion show much lower results for hydrogen-based options. For ionizing radiation, the much larger impacts for the ICE vehicle are related to the gasoline production (over 60%), where low level radioactive waste is accumulated. Regarding ozone depletion, over 90% of this impact is caused by direct bromotrifluoromethane emissions during petroleum production. Bromotrifluoromethane also known as Halon 1301 is a fire suppressing agent. However, hydrogen-based options show in three categories much higher impacts than the gasoline ICE vehicle. Construction of FCEV demands more mineral resources than conventional ICE vehicles distributed over various components, leading to more waste flows from mining processes. In particular, phosphate emissions to the ground occur during the treatment of tailings leading to high impacts for eutrophication, freshwater. These emissions are, for example, responsible for 55% of this impact category for the pipeline option and are 90% higher than the emissions for the ICE vehicle. Regarding human toxicity, cancer, the higher steel demand in the FCEV leads to chromium IV emissions to the ground during slag treatment from steel production. For human toxicity, non-cancer, – like for eutrophication, freshwater – a high impact comes from the treatment of tailings, which account for 54% of the overall impact human toxicity, non-cancer, for the pipeline option. For this impact category mainly arsenic ion and zinc ion emissions to water are the cause for the high impact. The four hydrogen-based options have, compared to the gasoline vehicle, rather similar results with a small preference for the hydrogen supply chain using pipelines.

Fig. 7 Relative results for the LCA of hydrogen mobility against the reference gasoline mobility in 2050; CGH₂: compressed gaseous hydrogen, LH₂: liquid hydrogen, LOHC: liquid organic hydrogen carrier (based on ref. 10 and ref. 60).

Life Cycle Costing of hydrogen mobility. Analogous to LCA, the functional unit of LCC was one driven kilometre. In this study, capital costs refer to levelized costs of car acquisition without VAT, consumables (fuel supply costs) and other operating costs (maintenance and repairs, insurance). Fuel supply costs comprise production costs of fuel as well as costs for transport, storage, and service stations. The results for the LCC of hydrogen mobility are depicted as absolute numbers in Fig. 8. Under the assumption that FCEV will achieve the same purchase cost level as a similar ICE vehicle in 2050,^19,70 the gasoline ICE vehicle will have slightly higher costs than the hydrogen-based vehicles. Even though gasoline is less expensive than hydrogen supply, the higher costs for maintenance of an ICE¹⁹ offset this advantage. Taking a closer look at the different options for hydrogen supply results for CGH₂ and pipeline supply show equally low costs. Even though a hydrogen refuelling station for liquid hydrogen and transport of liquid hydrogen requires less cost than gaseous hydrogen, costs for liquefaction nullify these cost advantages. The LOHC transport technology is the least advanced hydrogen transport option. From today's view, also with considered technological and economic improvements the LOHC option will be more expensive than the other hydrogen transport options – for this case study. However, it can keep up with the gasoline ICE vehicle. A more detailed discussion of the results for hydrogen mobility, e.g. hydrogen supply cost, can be found in Haase et al.¹¹

Fig. 8 Absolute results for the LCC of hydrogen mobility in 2050; CGH₂: compressed gaseous hydrogen, LH₂: liquid hydrogen, LOHC: liquid organic hydrogen carrier (based on ref. 10 and ref. 60).

Social assessment of hydrogen mobility. Due to the lack of comparable quantitative results for the indicator acceptance, here only the results for the indicator domestic value added and the criterion innovation potential with its indicators are presented. However, for more results and methodology of the indicator and criteria acceptance the articles by Emmerich et al.¹⁸ and Baur et al.⁷¹ are suggested.
Domestic value added. It is assumed that 64% of vehicles are produced domestically and that value added of costs for labour, car maintenance and repairs as well as car insurance are fully domestic. Relative results using the gasoline ICE vehicle as a reference are presented in Fig. 9. As this analysis is based on the LCC it is not surprising that also for this indicator, results lay very close together. It is safe to say that mobility with the gasoline ICE vehicle is less likely to produce domestic value added than the FCEV options due to the higher domestic value added of the hydrogen production compared to gasoline production. The results of the hydrogen-based options are very close together and with the underlying assumptions⁴⁸ it is not possible to make a distinction between the analysed options. Due to the hydrogen production in Germany, i.e. no import, and only national transport and distribution, all four options have a high share of domestic value added (>75%).

Fig. 9 Relative results for domestic value added for hydrogen mobility, gasoline vehicle as reference; CGH₂: compressed gaseous hydrogen, LH₂: liquid hydrogen, LOHC: liquid organic hydrogen carrier.

Innovation potential. To assess the innovation potential for all technologies related to hydrogen mobility options, is very time consuming. Thus, here only a spotlight is put on fuel production. For the FCEV options that means that patents for alkaline water electrolysis are analysed, whereas for the gasoline ICEV patents regarding gasoline refineries are taken under consideration.

The results for the patent-based indicators are displayed in Fig. 10, via a portfolio analysis, which combines the three selected patent indicators. The patent intensity is displayed on the x-axis, while growth rates are shown on the y-axis. The National technology share of a country is indicated through the size of the bubbles.

Fig. 10 Patent based indicators between 2013 and 2018 of alkaline water electrolysis (left) and gasoline refinery (right); PPP: purchasing power parity.

For the alkaline water electrolysis, Japan has the highest patent intensity combined with a comparable high patent growth rate and a high emphasis on alkaline water electrolyzes compared to all patents (high National technology share) in Japan (Fig. 10, Table 2 in the ESI†). This reflects the important position of hydrogen in Japan's way to decarbonization. In contrast, regarding gasoline refineries, Japan is not one of the five most important countries. Only China and the US show patent activities for both technologies. China has, for both technologies, a higher patent intensity than the US and a higher National technology share. This is in line with the findings for other technologies that in China, in general, a lot of patents are filed.⁴³ Germany also shows patent activities for both technologies within the top five countries, but not on a comparable level with the leading countries for each technology. For more details on the patent analysis of alkaline water electrolysis, refer to Baumann et al.⁴³

As can be seen, Germany is neither for alkaline water electrolysis nor for gasoline refinery the leading country. However, for alkaline water electrolysis Germany reaches a mean rank of 3.3, while for gasoline refineries only a mean rank of 4.7 is achieved (Table 3 in the ESI†). Thus, the innovation potential for alkaline water electrolysis can be considered higher as for gasoline refineries.

MCDA results

After discussing results from the individual indicators, in a next step, these results are the basis for the MCDA. They are combined with the preferences of technology developers from academia, see section Results survey on weighting factors, to provide clear guidance regarding the sustainability of gasoline and hydrogen mobility.

According to the PROMETHEE II approach described above, the outranking of the different mobility options is displayed in Fig. 11. The figure shows the results comparing the weighting factors derived from the technology developers with weighting factors based on hierarchical equal weighting (ESI Table 4†). The higher the outranking flow Φ for a technology option is, the more sustainable this option is. Detailed results for PROMETHEE I and II are listed in Table 5 in the ESI.†

Fig. 11 PROMETHEE II results using specific weighting factors from technology developers and equal weighting for the comparison of different mobility options.

Both weighting factor sets support the same conclusion: FCEVs supplied with hydrogen from a pipeline system is more sustainable than the other hydrogen options and the gasoline ICEV is less sustainable than an FCEV.

The LOHC supply chain of hydrogen results in the highest costs and for several environmental indicators in the worst results. Thus, in the overall sustainability ranking this option comes last among the FECV options. The pipeline supply chain of hydrogen convinces mainly through good environmental results. The gasoline vehicle shows favourable results in some environmental indicators, i.e. ecotoxicity, Human toxicity (cancer and non-cancer) and eutrophication, freshwater. However, this cannot offset the higher costs.

As can already be deducted from the PROMETHEE results for the two weighting factor sets, the sustainability ranking of the different technology options is rather robust. When taking the technology developer weighting factor set and varying only one weighting factor from the environmental indicators at the time, the weighting factor for eutrophication, freshwater changes the first. However, its global weight must change from 0.03 to 0.15 to induce a change of the sustainability ranking, which is a rather large change. At that point the LH₂ FCEV would become more sustainable than the CGH₂ FCEV and step up to the second most sustainable option after the pipeline option. Regarding the weighting factor for the levelized costs, a change in the sustainability ranking would not occur when the weighting factor gets higher. In contrast, if the weighting factor of the costs decreases to 0.16 the LH₂ FCEV becomes more sustainable than the CGH₂ FCEV.

As described in section Results survey on weighting factors the results regarding the weighting factors for the three sustainability dimensions showed some variations across the 60 participants. To show the impact of the deviation between the participants, here three weighting factor sets with the maximum values for the social, environmental and economic dimension, respectively, are discussed (Table 4). As no connection between the answers of the survey regarding sustainability dimensions and the surveys regarding single criteria can be made, the results from the other surveys are kept constant. Resulting weighting factors for indicators are listed in Table 6 in the ESI.†

Table 4 Extreme weighting factor sets for sustainability dimensions resulting from the participatory survey

	w Sus,env	w Sus,eco	w Sus,soc
Maximum social	0.250	0.250	0.500
Maximum environment	0.549	0.181	0.269
Maximum economy	0.278	0.556	0.137

---

The sustainability rankings of these new weighting factor sets are displayed in Table 5. The weighting factor set with the highest weighting factor for the social dimension (max social) shows no difference in the sustainability ranking, even though the environmental dimension weights much less than in the average weighting factor set. For the extreme weighting factor set preferring the environment (max environment) a small difference can be detected in the rankings, the CGH₂ option ranks now worse than the LH₂ option due to its more severe environmental impacts. When putting the economic dimension of sustainability first again (max economy), the ranking does not change. However, only 10% of the participants consider the economy as the most important sustainability dimension. Whereas 62% of the participants put the environment first and 28% the social dimension.

Table 5 Sustainability rankings using extreme weighting factor sets

Option	Rank average	Rank max social	Rank max environment	Rank max economy
CGH2	2	2	3	2
Pipeline	1	1	1	1
LH2	3	3	2	3
LOHC	4	4	4	4
Gasoline	5	5	5	5

---

The PROMETHEE outranking flows for these extreme weighting factor sets are listed in Table 7 in the ESI.†

Discussion and conclusions

It was the goal of this paper to extend an existing approach for sustainability assessment¹¹ of energy technologies with MCDA to come to an integrated sustainability assessment. To achieve this goal, the MCDA method PROMETHEE was chosen for the aggregation of indicator results due to its relatively easy application for an outranking method. Furthermore, weighting factors for sustainability dimensions and criteria/indicators were elicited from energy technology developers and energy system modellers from academia for the assessment of innovative energy technologies in the context of Germany. This extended approach was afterwards tested on a case study assessing hydrogen mobility in comparison to gasoline mobility for individual transport. Furthermore, the elicited weighting factors were compared with an equal weighting of sustainability dimensions and criteria/indicators. Additionally, the influence of using extreme weighting factors were discussed with the help of the case study results.

During this work several issues were identified that need to be addressed when using MCDA for sustainability assessment. These include the procedure for weighting factor determination, the necessity to expand the indicator set, the integration of other stakeholders and specifics regarding the case study on hydrogen mobility, which will be discussed in the following.

Weighting factor determination

As mentioned above, defining weighting factors is one of the most important steps in MCDA, but also a challenging procedure that has to be carried out carefully and in a transparent way. The method SMART was used for the elicitation of weighting factors because of its relatively easy procedure. Independent of the chosen method, one of the challenges are limitations of the cognitive spans (memory span, perceptual span etc.) of the participants. This is elementary as only a limited number of distinctions can be grasped by participants at once as a base of making judgements.^51,59 Even though this was taken under consideration for the design of this study with the stakeholders in the workshop, a decrease in participants from the first survey (n = 60) to the last survey (n = 48) was evident. The last survey was about the environmental criteria and was the most complex one with eight different environmental criteria. The highest effort was put into this last survey to provide adequate background information about the environmental criteria and the used scale before eliciting corresponding weighting factors. Therefore, the risk that stakeholders might have not properly understood the criteria or interpreted them differently should be minimized.⁷² Informing the participants about the different criteria so that they can make an informed decision was the most time consuming part of the stakeholder workshop. At the same time, this part of the procedure is most prone to bias due to the way the information is presented – even if it is done subliminally. In contrast, the opinions of the participants might be preconceived by discussions in the media.⁷³ Furthermore, as the creators of the survey have a strong background in LCA there might be a bias towards the presentation of environmental indicators and criteria. To minimize the risk of influencing the participants in their opinion towards the environmental criteria the survey about the sustainability dimensions was held before each environmental criterion was presented. For future surveys it might be helpful to either present the different criteria by different people, each of them experts in the presented criteria or by one person, who is objective towards all the presented criteria. Despite these challenges we were able to generate a meaningful weighting factor set for sustainability assessment of energy technologies.

Indicator expansion

Even though defining and implementing social/socio-economic indicators is challenging, it is of utmost importance to extend these. In particular, the criterion acceptance, which is often discussed in the media, needs to be developed into a functioning indicator that can be used for comparing different technologies. This is also stressed by the fact that acceptance was regarded as most important by the stakeholders for the implementation of innovative energy technologies. In line with acceptance, other criteria and indicators should be developed, e.g. security of energy supply, to make this sustainability assessment approach more holistic. Introducing such new criteria would imply a new stakeholder survey though. Also, we recommend a monitoring of the weighting factors every two to five years due to the temporarily limited validity of the same. This limitation stems from a changing world with hardly predictable conflicts and disasters, making it necessary to include other/more criteria. In line with this, also the perception of the already existing criteria might change.

Stakeholders integrated

An extension of the considered stakeholders beyond academia should be considered in future works to consider the heterogeneity of societal perspectives on the relevance of different aspects. Such an extension could also lead to a shift in preferences e.g. towards social or economic aspects. In particular stakeholders from industry must be integrated when technologies on the verge of commercialization should be assessed. In addition, the involvement of multiple interest groups omits the danger of biased weights that might favour specific technologies. In any case, the selection and involvement of participants should be transparent and equilibrated.²⁵

Case study specifics

The sustainability assessment of hydrogen mobility showed that FCEVs are best supplied with hydrogen by pipelines. However, it was also shown that applying the derived weighting factor sets does not have an impact on overall results of this case study. For hydrogen mobility this means robust results for choosing the most sustainable option under given weight elicitation conditions. For the method for sustainability assessment this means, that more case studies should be performed to test the MCDA approach, e.g. for batteries or second generation biofuels.

After all, the question arises, if the time-consuming process of stakeholder integration for weighting factor elicitation is necessary. Even though the chosen SMART method belongs to the less time-consuming ones. In our opinion, this is necessary because on the one hand results might be more sensitive to different weighting factors for other case studies and on the other hand the awareness of sustainability indicators within energy technology developers was risen. An MCDA with equal weighting can merely give a rough estimation for the comparison of energy technologies. Only the integration of stakeholders can give legitimation for decision-making processes.

Author contributions

Christina Wulf: conceptualization, mthodology, investigation, formal analysis, writing – original draft, writing – reviewing and editing Martina Haase: conceptualization, writing – original draft, writing – review and editing. Manuel Baumann: conceptualization, writing – original draft, writing – reviewing and editing. Petra Zapp: conceptualization, investigation, supervision, writing – reviewing and editing.

Conflicts of interest

There are no conflicts to declare.

Acknowledgements

This paper is based on the presentation “Determination and application of weighting factors for sustainability assessment in the Helmholtz Initiative Energy System 2050” held at the Prospective Multidimensional Assessment of Energy Technologies and Scenarios (ProMETS) Workshop on the 25th of February 2021. This work contributes to the research performed at CELEST (Center for Electrochemical Energy Storage Ulm-Karlsruhe) and was funded by the German Research Foundation (DFG) under Project ID 390874152 (POLiS Cluster of Excellence, EXC 2154).

References

1. UN, Adoption of the Paris Agreement, Framework Convention on Climate Change, United Nations, Paris, 2015.
2. European Commission, The European Green Deal, European Commission, Brussels, 2019.
3. A. Grunwald and C. Rösch, Energy Sustain. Soc., 2011, 1(3).
4. J. Elkington, Environ. Qual. Manag., 1998, 8, 37–51.
5. V. Prado, K. Rogers and T. P. Seager, in Life Cycle Assessment Handbook a Guide for Environmentally Sustainable Products, ed. M. A. Curran, Wiley, Hoboken, 2012, ch. 19, pp. 413–432.
6. M. Jones, Multi-Criteria Decision Analysis Tool, The University of Manchester, Manchester, 2016.
7. M. Hannouf and G. Assefa, Sustainability, 2018, 10, 3863.
8. S. Sala, A. K. Cerutti and R. Pant, Development of a Weighting Approach for the Environmental Footprint, Publications Office of the European Union, Luxembourg, 2018.
9. E. Ekener, J. Hansson, A. Larsson and P. Peck, J. Cleaner Prod., 2018, 181, 337–351.
10. M. Haase, M. Baumann, C. Wulf, C. Rösch and P. Zapp, in Technikfolgenabschätzung: Handbuch für Wissenschaft und Praxis, ed S. Böschen, A. Grunwald, B.-J. M. A. Krings and C. Rösch, Nomos Verlagsgesellschaft mbH & Co. KG, Baden-Baden, 1 edn, 2021, pp. 306–320,  DOI:10.5771/9783748901990-306.
11. M. Haase, C. Rösch, C. Wulf, M. Baumann, M. Weil, P. Zapp and T. Naegler, Energy Sustain. Soc., 2022, 12, 20.
12. C. Wulf and M. Kaltschmitt, Sustainability, 2018, 10, 1699.
13. M. Ram, D. Bogdanov, A. Aghahosseini, A. Gulagi, S. A. Oyewo, M. Cild, U. Caldera, K. Sadovskaia, J. Farfan, L. S. N. S. Barbosa, M. Fashihi, S. Khalili, B. Dalheimer, G. Gruber, T. Traber, F. De Caluwe, H.-J. Fell and C. Breyer, Global Energy System Based on 100% Renewable Energy – Power, Heat, Transport and Desalination Sectors, Lappeenranta University of Technology, Energy Watch Group, Lappeenranta, Berlin, 2019.
14. IEA,The Future of Hydrogen: Seizing Today's Opportunities – Report Prepared by the IEA for the G20, Japan, International Energy Agency, Paris, 2019.
15. D. Stolten, P. Markewitz, T. Schöb, F. Kullmann, S. Risch, T. Groß, M. Hoffmann, D. Franzmann, T. Triesch, S. Kraus, R. Maier, B. Gillessen, H. Heinrichs, N. Pflugradt, T. Grube, J. Linssen and L. Kotzur, Neue Ziele auf alten Wegen? Strategien für eine treibhausgasneutrale Energieversorgung bis zum Jahr 2045, Institut für Energie- und Klimaforschung: Techno-ökonomische Systemanalyse (IEK-3), Forschungszentrum Jülich, Jülich, 2021.
16. C. Wulf, PhD dissertation, Hamburg University of Technology, 2017.
17. Roadmap des Kopernikus-Projektes P2X - Phase II - Optionen für ein nachhaltiges Energiesystem mit Power-to-X Technologien: Transformation – Anwendungen – Potenziale, ed F. Ausfelder and D. D. Tran, DECHEMA Frankfurt am Main, 2022.
18. P. Emmerich, A.-G. Hülemeier, D. Jendryczko, M. J. Baumann, M. Weil and D. Baur, Energy Policy, 2020, 142, 111516.
19. T. Grube, S. Kraus, J. Reul and D. Stolten, Transp. Res. D: Transp. Environ., 2021, 101, 103110.
20. Fuel Cell and Hydrogen Joint Undertaking, Hydrogen Roadmap Europe, Publications Office of the European Union, Luxembourg, 2020.
21. G. F. Government, The National Hydrogen Strategy, German Federal Ministry for Economic Affairs and Energy, Berlin, 2020.
22. C. Thies, K. Kieckhäfer, T. S. Spengler and M. S. Sodhi, Eur. J. Oper. Res., 2019, 274, 1–21.
23. M. Marttunen, J. Mustajoki, M. Dufva and T. P. Karjalainen, EURO J. Decis. Process., 2015, 3, 187–214.
24. M. Haase, C. Wulf, M. Baumann, H. Ersoy, J. C. Koj, F. Harzendorf and L. S. Mesa Estrada, Clean Technol. Environ. Policy, 2022, 24, 3171–3197.
25. M. Baumann, M. Weil, J. F. Peters, N. Chibeles-Martins and A. B. Moniz, Renewable Sustainable Energy Rev., 2019, 107, 516–534.
26. P. Goodwin and G. Wright, Decision Analysis for Management Judgment, Wiley, Hoboken, 5th edn, 2014.
27. W. Edwards, IEEE Trans. Syst. Man Cybern., 1977, 7, 326–340.
28. T. L. Saaty, J. Math. Psychol., 1977, 15, 234–281.
29. P. Tarne, A. Lehmann and M. Finkbeiner, Int. J. Life Cycle Assess, 2019, 24, 530–542.
30. T. Naegler, L. Becker, J. Buchgeister, W. Hauser, H. Hottenroth, T. Junne, U. Lehr, O. Scheel, R. Schmidt-Scheele, S. Simon, C. Sutardhio, I. Tietze, P. Ulrich, T. Viere and A. Weidlich, Sustainability, 2021, 13, 5217.
31. Swiss Centre for live cycle inventories, Ecoinvent Database Version 3.3, Zurich, 2016.
32. EU-JRC, Recommendations for Life Cycle Impact Assessment in the European Context - Based on Existing Environmental Impact Assessment Models and Factors, European Commission-Joint Research Centre - Institute for Environment and Sustainability, Luxembourg, 2011.
33. GreenDelta, Journal, 2018.
34. T. E. Swarr, D. Hunkeler, W. Klöpffer, H.-L. Pesonen, A. Ciroth, A. C. Brent and R. Pagan, Environmental Life Cycle Costing: A Code of Practice, SETAC, Pensacola, Brussels, 2011.
35. IEC, Dependability Management - Part 3-3: Application Guide - Life Cycle Costing, International Electrochemical Commission, Geneva, 2017.
36. VDI, Purchase, Operating and Maintenance of Production Equipment Using Life Cycle Costing (LCC), VDI-Gesellschaft Produktion und Logistik, Düsseldorf, 2005.
37. D. Hunkeler, K. Lichtenvort and G. Rebitzer, Environmental Life Cycle Costing, CRC Press, Boca Raton, 2008.
38. F. Ribeiro, P. Ferreira, M. Araújo and A. C. Braga, Renewable Energy, 2018, 122, 688–697.
39. G. Assefa and B. Frostell, Technol. Soc., 2007, 29, 63–78.
40. N. M. Huijts, E. J. Molin and B. van Wee, J. Environ. Psychol., 2014, 38, 153–166.
41. E. Miguel and C. Oltra, Hyacinth: Report on Methodologies and Factors, Cidaut Ciemat, Valladolid, 2016.
42. P. Emmerich, M. Weil and M. Baumann, Citizen concerns and acceptance for novel energy technologies, IET/CICS.NOVA, Innovation and Technology Studies pole at FCT-UNL, Centro Interdisciplinar de Ciências Sociais, Faculdade de Ciências e Tecnologia, Universidade Nova de Lisboa, Monte de Caparica, 2019.
43. M. Baumann, T. Domnik, M. Haase, C. Wulf, P. Emmerich, C. Rösch, P. Zapp, T. Naegler and M. Weil, Technol. Forecast. Soc. Change, 2021, 165, 120505.
44. V. Cillo, A. M. Petruzzelli, L. Ardito and M. Del Giudice, Corp. Soc. Responsib. Environ. Manag., 2019, 26, 1012–1025.
45. C. M. Christensen, H. Baumann, R. Ruggles and T. M. Sadtler, Harv. Bus. Rev., 2006, 84, 94–101.
46. J. A. Sabadie, Sci. Total Environ., 2014, 481, 668–673.
47. K. Lee and S. Lee, Energy Policy, 2013, 59, 415–432.
48. F. Harzendorf, C. Wulf, M. Haase, M. Baumann, H. Ersoy and P. Zapp, Clean Technol. Environ. Policy, 2022, 24, 3145–3169.
49. I. Wilkens, PhD dissertation, Technische Universität Berlin, 2012.
50. A. Kumar, B. Sah, A. R. Singh, Y. Deng, X. He, P. Kumar and R. C. Bansal, Renewable Sustainable Energy Rev., 2017, 69, 596–609.
51. B. Németh, A. Molnár, S. Bozóki, K. Wijaya, A. Inotai, J. D. Campbell and Z. Kaló, J. Comp. Eff. Res., 2019, 8, 195–204.
52. J. P. Brans, presented in part at the L'aide à la décision: Nature, Instruments et Perspectives d'Avenir, Québec, 1982.
53. I. Siksnelyte, E. K. Zavadskas, D. Streimikiene and D. Sharma, Energies, 2018, 11, 2754.
54. J. Geldermann and N. Lerche, Leitfaden zur Anwendung von Methoden der multikriteriellen Entscheidungsunterstützung - Methode: PROMETHEE, Georg-August-Universität; Wirtschaftswissenschaftliche Fakultät Professur für Produktion und Logistik, Göttingen, 2014.
55. J. P. Brans, P. Vincke and B. Mareschal, Eur. J. Oper. Res., 1986, 24, 228–238.
56. J.-P. Brans and B. Mareschal, in Multiple Criteria Decision Analysis: State of the Art Surveys, Springer New York, New York, NY, 2005, pp. 163–186,  DOI:10.1007/0-387-23081-5_5.
57. J. P. Brans and P. Vincke, Manag. Sci., 1985, 31, 647–656.
58. C. Wulf, P. Zapp, A. Schreiber and W. Kuckshinrichs, Sustainability, 2021, 13, 7009.
59. M. Brunelli, Introduction to the Analytic Hierarchy Process, Springer Nature, Cham, 2015.
60. B. Mareschal, How to Choose the Correct Preference Function, Rabat, 2020.
61. T. Brunner and J. Axsen, Energy Res. Soc. Sci., 2020, 67, 101511.
62. T. Pregger, T. Naegler, W. Weimer-Jehle, S. Prehofer and W. Hauser, Clim. Change, 2020, 162, 1743–1762.
63. ADAC, VW Golf 1.5 TSI OPF ACT IQ.Drive (01/19 - 08/19), https://www.adac.de/rund-ums-fahrzeug/autokatalog/marken-modelle/vw/golf/vii-facelift/296758/#technische-daten, accessed 18.11.2020.
64. ADAC, Toyota Mirai (ab 12/15), https://www.adac.de/rund-ums-fahrzeug/autokatalog/marken-modelle/toyota/mirai/ad1/251604/#technische-daten, accessed 29.07.2020.
65. J. C. Koj, C. Wulf, A. Schreiber and P. Zapp, Energies, 2017, 10, 860.
66. A. Schreiber, J. Marx and P. Zapp, J. Cleaner Prod., 2019, 233, 561–572.
67. C. Wulf and P. Zapp, J. Sustain. Dev. Energy Water Environ. Syst., 2021, 9, 1080371.
68. A. Benitez, C. Wulf, A. de Palmenaer, M. Lengersdorf, T. Röding, T. Grube, M. Robinius, D. Stolten and W. Kuckshinrichs, J. Cleaner Prod., 2021, 278, 123277.
69. C. Wulf and P. Zapp, Int. J. Hydrogen Energy, 2018, 43, 11884–11895.
70. FEV, Impact of Fuel Cell Technology on the Machinery and Component Supplier Industry, FEV Consulting, Aachen, 2020.
71. D. Baur, P. Emmerich, M. J. Baumann and M. Weil, Energy Sustain. Soc., 2022, 12, 4.
72. K. Steele, Y. Carmel, J. Cross and C. Wilcox, Risk Anal., 2009, 29, 26–33.
73. W. Zeug, A. Bezama, U. Moesenfechtel, A. Jähkel and D. Thrän, Sustainability, 2019, 11, 1511.

---

Footnote

† Electronic supplementary information (ESI) available. See DOI: https://doi.org/10.1039/d2se01170k

---