The impact of convexity on expansion planning in low-carbon electricity markets

Abstract

Expansion planning models are tools frequently employed to analyze the transition to a carbon-neutral power system. Such models provide estimates for an optimal technology mix and optimal operating decisions, but they are also often used to obtain prices and subsequently calculate profits. This paper analyzes the impact of modeling assumptions on convexity for power system outcomes and, in particular, on investment cost recovery. Through a case study, we find that although there is a long-term equilibrium for producers under convex models, introducing realistic constraints, such as non-convexities/lumpiness of investments, inelastic demand or unit commitment constraints, leads to profitability challenges. We furthermore demonstrate that considering only short-term marginal costs in market-clearing may potentially create a significant missing-money problem caused by a missing-market problem and dual degeneracy in a 100 percent renewable system.

1 Introduction

On its path towards carbon neutrality, the European Union has established clear long-term goals of reducing emissions (European Commission 2021), and more recently of achieving a carbon-neutral economy by 2050 (European Commission 2019). Carbon neutrality by 2050 is now also the US administration’s goal (Biden 2021, https://joebiden.com/clean-energy/). The goal is clear; however, the path to get there is not. A carbon-neutral power sector requires profound changes concerning electricity system and market design, consumer integration and capacity mix. To guide us through this transition and to obtain an optimal capacity mix, stakeholders, policymakers, market participants, and consumers often employ expansion planning models to support optimal decision making.

There exist numerous expansion planning models Hemmati et al. (2013); Koltsaklis and Dagoumas (2018); Dagoumas and Koltsaklis (2019); Gonzalez-Romero et al. (2020); Wogrin et al. (2020) that differ with respect to many aspects: whether they are static or dynamic, deterministic or stochastic, single-node or network-based, optimization or equilibrium models, linear or non-linear etc. A particular aspect that we want to assess in this work is the assumption of convexity of these models, mainly what is considered discrete and what is continuous. These assumptions impact corresponding optimization results on both primal variables and potentially shadow prices, that is dual variables and what type of costs are accounted for in these dual variables. This impact, however, is rarely ever discussed in the literature. We, therefore, want to dedicate this work to discuss this exact issue and demonstrate that, as a modeler, one has to be careful about the underlying assumptions of convexity of an expansion model and how they impact investment and potential market outcomes.

The majority of expansion planning models are in fact Mixed Integer Programs (MIPs), which consider discrete investment decisions and possibly discrete operating decisions such as start-up, shut-down and commitment decisions. When such MIPs are used to obtain market prices, as the dual variables of the corresponding constraints, long-term investment costs and short-term costs associated with discrete decisions are neglected. Only short-term variable costs¹ are reflected in these short-term prices. We want to assess different degrees of convexity of such expansion planning models and contrast results such as investments, market prices, and resulting cost/benefit of the technologies in the optimal mix , in order to indicate that the effect on model results often depends on the source of non-convexity. Economic theory states that, under simplifying assumptions Korpås and Botterud (2020), short-term prices are sufficient to recover investment costs Rodilla (2010). Some of these assumptions, however, are relatively strong and require: a perfectly competitive market (real electricity markets more resemble oligopolies at times Bushnell et al. (2008) or monopolistic competition); the convexity of generators’ cost functions (this does not allow to capture discrete unit commitment costs or economies of scale); no lumpy investments (in traditional power systems with significant thermal generators investments are the epitome of lumpy although in large systems this becomes less of an issue).

In practice, there exist many power system planning models, such as EMPIRE, GenX or LEGO, that do not satisfy all of those assumptions but that are being used for decision making. In particular, they have inelastic demand. As a matter of fact, we show in the appendix that in such models short-run (SR) marginal costs are not uniquely defined due to a missing-market problem and dual degeneracy. In practice, numerical solvers adopt solutions where short-run marginal costs only reflect operating costs and therefore cannot achieve cost recovery. This is to say that the commonly adopted approach of estimating market prices by fixing discrete (investment or UC) variables and using Lagrange multipliers of the relaxed model is bound to lead to a missing-money problem that model users need to be mindful of. Moreover, the missing money problem worsens in a 100% renewable power system, a topic of rising interest Hansen et al. (2019).

Therefore, in this paper we want to quantify the misalignment between the ideal cost recovery and actual results of frequently employed models (that may be affected by dual degeneracy in market prices). We also analyze that it largely depends on the type of model employed and its degrees of convexity. This convexity issue in pricing is explored in Sioshansi et al. (2008); Ruiz et al. (2012); Liberopoulos and Andrianesis (2016); Kuang et al. (2019) but mainly revolves around operational problems. In this paper, we want to stress that the missing-money problem extends to investment problems, and that non-convexities can impact optimal capacity investments as stated in Mays et al. (2021).

The contributions of this paper are as follows: we present a detailed analysis of the impact of different degrees of convexity in planning models on optimal expansion decisions and cost recovery, both in current and future power systems. Our results indicate that while in theory, everything works out in terms of recovering costs in a convex market, in practice it does not when non-convexities are accounted for and demand is inelastic. Moreover, we compare the profitability obtained by using either short-term or long-term marginal prices. The latter prices reflect both operational and investment costs, and we discuss whether they are necessary to recover investment costs. We show that short-term prices are likely to create a missing-money problem (due to a missing-market problem and dual degeneracy as shown in the appendix), which is exacerbated in future electricity markets with high shares of variable renewable energy sources (VRES).

The rest of the paper is organized as follows. Section 2 provides a brief overview of the generation expansion planning models used for the analyses and describes the model data. In Sect. 3 we propose five model paradigms with varying degrees of convexity. Section 4 discusses model results under all five paradigms for a current power system. Section 5 carries out similar analyses for future (100% renewable) power systems and discusses how this could lead to a missing-money problem. Finally, Sect. 6 concludes the paper.

2 Expansion planning model and case study data

In this paper, we carry out the case study using the open-source Low-carbon Expansion Generation Optimization (LEGO) model Wogrin et al. (2022) available on GitHub², stemming from previous research by the authors Wogrin et al. (2020). As a brief overview: LEGO is an optimization model that decides generation expansion, production and unit clustered commitment (UC)³ decisions while minimizing overall total system cost and accounting for optimal power flow constraints (both AC and DC) and inertia requirements. LEGO is very flexible and modular because each model block, e.g., inertia considerations, can be included (or not) depending on the study of interest. Note that the contribution of this paper is not the mathematical formulation of a generation expansion planning (GEP) model, but the analysis of the impact of convexity in such models on power system results. Therefore, the complete formulation of LEGO is omitted here, and the interested reader is referred to Wogrin et al. (2020). However, we provide a stylized formulation of the model below to facilitate the exposition in the paper.

To that purpose, we introduce the nomenclature of the stylized⁴ model in Table 1. The standard (primal) model outputs are: investment decisions $x_g$, production decisions per generator $p_g$, and UC decisions ($u_t,y_t,z_t$) per generator. The model allows the option to shed load through variable power non-supplied pns at a high cost $C^{pns}$ that is set at 10000 €. In the case studies presented later, however, no non-supplied energy appears and therefore it is not discussed further.

We now briefly discuss the stylized model formulation given in (1). Note that we present this formulation for one representative hour h, ⁵ in order to avoid additional indices; however, in the full LEGO model multiple time periods and chronological constraints (e.g., ramping) are represented, and shadow prices are obtained correspondingly (e.g., $\lambda$ for each time period). In the objective function (1a) we minimize total system cost as given by: operation and maintenance cost; fixed costs including commitment costs, startup and shutdown costs of thermal generators; reserve costs; and, investment costs. We furthermore consider upper and lower bounds (1b) on production, storage units (1c), and specific upper and lower bounds for thermal constraints including the technical minimum (1d). In order to establish the logic between commitment $u_t$, startup $y_t$ and shutdown $z_t$ decisions, we need temporal chronology and to that purpose we have included an $h-1$ where necessary, such as constraints (1e), ramping constraint (1f) and the definition of the storage⁶ state of charge (1g). Constraint (1h) represents the demand balance constraint and (1i) enforces the minimum reserve requirement. Finally, constraint (1j) defines UC and investment variables as discrete.

Table 1 Nomenclature of relevant indices, variables and parameters of LEGO model

$$\begin{aligned} min \sum _g C^{OM}_g \cdot p_{g} + \sum _t C^{F}_t \cdot (y_t,z_t,u_t) + \sum _{t/s} C^{R} \cdot r_{t/s} \nonumber \\ + \sum _g C^{INV}_g \cdot x_g + C^{pns} \cdot pns \end{aligned}$$

(1a)

$$\begin{aligned} 0 \le p_{g} \le {\overline{P}}_g \cdot (EU_g+x_g) \quad \forall g \end{aligned}$$

(1b)

$$\begin{aligned} p_s - cs_s + r_s \le {\overline{P}}_s \cdot (EU_s+x_s) \quad \forall s \end{aligned}$$

(1c)

$$\begin{aligned} {\underline{P}}_t \cdot u_t \le p_t + r_t \le {\overline{P}}_t \cdot u_t \quad \forall t \end{aligned}$$

(1d)

$$\begin{aligned} u_{t} - u_{t,h-1} = y_t - z_t \quad \forall t \end{aligned}$$

(1e)

$$\begin{aligned} p_{t} - p_{t,h-1} \le RA_t \quad \forall t \end{aligned}$$

(1f)

$$\begin{aligned} soc_{s} = soc_{s,h-1} + cs_s - p_s \quad \forall s \end{aligned}$$

(1g)

$$\begin{aligned} \sum _g p_g - \sum _s cs_s + pns = D \quad : (\lambda ) \end{aligned}$$

(1h)

$$\begin{aligned} \sum _{t/s} r_{t/s} \ge R D \quad : (\mu ) \end{aligned}$$

(1i)

$$\begin{aligned} pns \in {\mathbb {R}}^+, u_t,y_t,z_t,x_g \in {\mathbb {Z}}^+ \quad \forall g,t \end{aligned}$$

(1j)

LEGO is designed as a Mixed Integer Problem (MIP); the non-convexities are due to the integrality of planning variables (i.e., investment ($x_g$) and operational (e.g., unit commitment, UC ($u_t,y_t,z_t$)) decisions. Relaxing integrality on these two sets of variables renders a relaxed-MIP (rMIP) framework that still has physical meaning. This also brings us to some (dual) model outputs such as prices and profits. How do we use LEGO to obtain prices and to calculate generator profits? If integrality in LEGO is relaxed, then the model becomes a linear program (LP), and dual variables are uniquely defined due to the strong duality of LPs. However, in a MIP framework, this is not the case. A common practice Levin et al. (2019) to obtain prices for MIP models, which has also been adopted in this article, is to: (1) run the MIP, (2) fix all integer variables to their optimal value, and (3) then re-run the model as an LP. This method determines equilibrium prices for a given solution. With this in mind, prices are obtained as the dual variables of the corresponding constraints, i.e., the spot market price, $\lambda$, is the dual variable of the demand balance constraint (1h), reserve market prices are obtained as dual variables, $\mu$ of reserve constraint (1i). The technique of fixing unit commitment variables and then obtaining duals can lead to generators running a loss (in the short term) as mentioned in Bothwell and Hobbs (2017). This has led to the use of “make-whole” payments in the US. However, in this paper we want to focus on total cost recovery, including investment costs (not only operating costs). Therefore, make-whole payments would likely have limited impact on our analysis.

Defining capacity credits for renewables Bothwell and Hobbs (2017) or storage technologies Mertens et al. (2021) is a challenging and ongoing topic of research. While exploring all possible policy measures for remunerating installed capacity is outside the scope of this paper, we have included on specific example of such a policy scheme. Inspired by Gerres et al. (2019), we also have introduced a firm capacity constraint in LEGO:

$$\begin{aligned} \sum _g FC_g \cdot {\overline{P}}_g \cdot (x_g + EU_g) \ge D^{peak} \cdot FP \quad : (\nu ), \end{aligned}$$

(2)

where g is the index for generators, $FC_g$ is the firm capacity coefficient. ⁷ by technology and FP is the percentage of firm capacity required by the system (e.g., 110% here) both have been taken from Gerres et al. (2019), ${\overline{P}}_g$ is the maximum power output per generator, $x_g$ is the discrete investment variable, $EU_g$ is the number of existing generators (which we do not consider in this case), and finally, $D^{peak}$ is the hourly system peak demand. Essentially this constraint enforces that firm system capacity is at least 110% of system peak demand. In the rMIP framework, the dual of this constraint, $\nu$, yields a firm capacity price in €/MW of firm capacity, which is used in firm capacity payments.

In this paper, generator profitability is calculated as follows: spot market revenues minus spot market purchases (the latter can only occur for storage units); reserve market revenues minus reserve market O&M cost; minus operating costs (variable and fixed); minus investment costs; plus firm capacity payments (should there be any). We assume truthful bidding and no strategic considerations from participants.

The main purpose of the case studies in this article is to assess the impact of the degrees of convexity of expansion planning models on (primal and dual) power system results. In particular, we investigate whether the importance of convexity of power system modeling changes, especially when migrating from current to future power systems with a high share of VRE sources. However, this is a model-based exercise, and we are not claiming that the same results will necessarily happen in reality. Moreover, we have not analyzed the impact of price-responsive demand and demand-side management (DSM) on model results. In future power systems with a high penetration of VRES, DSM might play a very important role and have significant impact on market prices. We plan to assess this topic in future research. This is, however, out of the scope of this paper.

The basic data set covers a time horizon of one static year in the future, which has been approximated by seven representative days. Using representative periods is a common practice in expansion planning, as pointed out in Gonzalez-Romero et al. (2020). The complete data set used in this paper is available online⁸ and is based on the StarNet Lite demo version⁹.

Fig. 1

9-bus network, candidate generation units and nodal demand indicated in percent

As a brief overview of the case study, in this paper, we consider a 9-bus power system depicted in Figure 1. This system has 13 existing transmission lines with an 800 MW transmission capacity, no existing generation capacity, and candidate generators from different technologies. The technologies considered include combined-cycle gas turbines (500 MW units), open-cycle gas turbines (400 MW units), battery energy storage systems (50 MW units), solar and wind (100 MW units). Note that when solving LEGO as a MIP, investment decisions are lumpy with the unit sizes specified above; however, when investment decision variables are relaxed, then investments are considered continuous variables. For the case studies in this paper, we use LEGO under a DC-OPF framework and without inertia considerations (for simplicity). We consider operating reserve requirements of 3% of demand and a firm capacity requirement of 110% of maximum system demand. The specific data are available on Github¹⁰. Let us now discuss the different degrees of convexity possible in LEGO. Note that non-convexity issues are likely amplified in a small test-system like this one.

3 Degrees of convexity of power system models

In the previous section, we have discussed how we define profits in the LEGO framework, and what revenue and cost concepts are accounted for. However, in this section, we want to raise another important issue related to expansion planning and profitability: the convexity of the expansion model in question and the corresponding definition of meaningful prices; in particular, should market prices reflect long-term investment costs or not?

Economic theory states, and many works in the literature confirm Korpås and Botterud (2020); Rodilla (2010) that under perfect competition and a cost-minimal generation expansion mix, every technology recovers investment costs from the resulting market clearing prices. An underlying assumption to these results is the need for convexity: convex cost functions, a convex model etc. Real life decisions, however, are not convex. Apart from considering non-convex cost functions, realistic generation expansion models in power systems often have inelastic demand and have a source of non-convexity that is the discrete nature of unit commitment and lumpy investment decisions (among other possibly integer variables). Disregarding the integrality of these decisions (by relaxing them, for example) might confirm theoretical statements. However, with the following example, we would like to showcase that theory might diverge from reality in terms of the profitability of individual generators under different convexity assumptions due to a missing-market problem and dual degeneracy as we illustrate in the appendix. In the paper, we want to quantify these impacts depending on the different degrees of non-convexities, which have other impacts on dual variables.

To that purpose, using the exact same data we run the LEGO model under five different paradigms (most convex to the most non-convex):

• rMIP: the relaxed Mixed-Integer Program (rMIP) version, where both sources of integrality (investment and UC decisions) have been relaxed, thereby rendering a completely convex model. Hence, the Lagrange multipliers of the corresponding constraints are used as prices and reflect both short-run (such as UC) and long-run (such as investment) costs.

• rMIP-SR: the short-run (SR) rMIP is when we first solve the rMIP; then fix investment decisions and re-run the rMIP. Nothing will change in the primal variables; however, since we fix investment decisions, the obtained prices only reflect short-run (operational and UC costs) but not long-run investment costs.

• rUC: this corresponds to the MIP where UC decisions are relaxed whereas GEP decisions are considered discrete. Prices therefore reflect only short-run costs (but include UC costs) but not long-run investment costs.

• rGEP: this model is a Mixed integer Program (MIP) in which only generation expansion (GEP) decisions have been relaxed; therefore, prices only reflect long-run investment costs and variable short-run costs but not UC (fixed, start-up nor shut-down) costs. This is a somewhat artificial case (as GEP is relaxed and corresponding UC are discrete) but it only serves as a transition towards full non-convexity.

• MIP: prices from the full MIP version of LEGO only reflect variable short run costs but no long-run costs nor UC-related short-run costs.

Table 2 Types of variables under different model paradigms

In Table 2 we provide an overview of how specific variables are considered (binary/fixed/relaxed) in each type of model paradigm. Costs associated with variables considered binary or fixed are not reflected in the corresponding dual variables¹¹. In the following sections we analyze both primal and dual power system outputs under all 5 different convexity paradigms to draw meaningful conclusions.

4 Current power system: GEP results and impacts of convexity

In Table 3 we present the GEP investments per technology under the different modeling paradigms. While most technologies under rMIP are within a +/- one unit change, e.g., 2020.3 MW of Wind (rMIP) versus 2,100 MW (MIP) with 100 MW Wind units, 1417.6 MW of Solar (rMIP) versus 1500 MW of Solar (MIP) with 100 MW individual Solar units, 3216.2 MW of CCGT (rMIP) versus 3000 MW (MIP) with 500 MW units, 1185.1 MW of OCGT (rMIP) versus 1200 MW (MIP) with 400 MW units, BESS diverges from this trend (50 MW units). From a relative point of view, BESS is the technology most affected by the level of convexity of the model. In this case, the lumpiness of thermal power plant investments (especially CCGTs) in the model is compensated for by BESS units.

Table 3 Capacity investments (MW) per technology under different modeling paradigms

Before assessing the corresponding profits per technology, note that the payment for firm capacity is obtained as the dual of the firm capacity constraint. This dual, however, only exists in models where the capacity decisions are modeled as continuous variables, i.e., there is no such dual in the MIP model. Also note that in the rMIP-SR case the presented profits include the firm capacity payments from the rMIP case, which are 25,814 €/MW-year of firm capacity.

We first observe that, using marginal cost pricing in a completely convex model (rMIP) allows all generators, including BESS, to recover all costs and obtain exactly zero profit, as shown in Table 4 and predicted by economic theory. In Table 5 we show the detailed elements that have been used to calculate the profits for each technology. However, note that the prices that have been used to calculate profits under rMIP included, among other things, long-run investment costs. In order to assess how profits would evolve if only short-run marginal costs were to be used, we focus on rMIP-SR. All primal variables are exactly the same as in the rMIP case; however, dual variables (prices) change and do no longer reflect long-run dynamics (For an illustrative example the reader is referred to the appendix, where we point out the fact that there is a missing-market problem and dual degeneracy in market prices). In practice, numerical solvers choose the simplest market price solutions (only including SR costs). The result is that the system is 0.8 million € short in total. Most technologies actually incur losses if only short-run marginal prices are used except for solar and OCGTs that break even. This is mirrored by the fact that average spot market prices, as shown in Table 6 are also slightly lower under rMIP-SR because they no longer account for long-run investment costs. However, comparing the rMIP and rMIP-SR case, one might be inclined to believe that accounting for investment costs in the market prices does not matter much, as 0.8 M€ of losses are relatively small in comparison to a 1603 M€ total system cost. In that case, one may argue that the current practice of determining prices based on short-run marginal cost (SRMC) works well for the current system with a significant share of resources with a positive SRMC.

Table 4 Annual profits (M€) per technology under different modeling paradigms

Table 5 Annual profit (M€) elements per technology in rMIP case

In the rUC case (where prices only reflect short-run costs, including UC costs), there is no dual for the firm capacity constraint as GEP decisions are discrete. Hence, we have taken the value for firm capacity in €/MW from the rMIP case and applied it to the investments here¹², just to be able to compare the numbers to other cases. Under this case, OCGT breaks even exactly, BESS, wind and solar are net losers, and CCGTs obtain profits. In total, the market makes almost 15 million € of profit, due to the fact that average prices increase substantially with respect to the rMIP case. This is because prices reflect that, to provide an additional MW, a new unit might have to be switched on.

In the rGEP case (where prices do not reflect UC costs), we observe that technologies that do not have discrete UC variables actually break even exactly. The conventional generation that involves UC variables (such as CCGTs and OCGTs) end up making profits. The reason for this are the prices. Average spot prices increase with respect to the rMIP case. Under rMIP it was possible to dispatch a fraction of a conventional plant. Hence, the price only reflects this additional fraction. Under rGEP this is no longer possible, as UC decisions are considered discrete. Hence, the model prices reflect investment costs rather than start-up or fixed costs, which end up being more expensive when having to price dispatching an additional MW. Again, the rGEP case is a somewhat artificial case.

Finally, considering the most realistic expansion planning model, the MIP, prices no longer reflect long-run investment costs nor UC costs. Note that the firm capacity constraint does not have a dual in the MIP due to the discrete nature of investment decisions. As an approximation, we have used the payments derived in the rMIP framework. Hence, MIP profits contain a firm capacity payment with rMIP prices (without these payments, losses would be even higher). The prices that have been extracted in the MIP only reflect short-run variable costs (no GEP nor UC costs are included), which means that they are lower than prices observed in the other cases. In general, all technologies incur losses, and only the solar becomes positive after the firm capacity payments, while the remaining technologies are not profitable. These results seem to indicate that the combination of integrality and disregarding investment costs in prices has a negative effect on the system’s overall profitability. While the rMIP-SR was only 0.8 M€ short, the full MIP is 90.2 M€ short.

Table 6 Spot market prices (€/MWh) under different modeling paradigms

In order to demonstrate that the total cost recovery (operation & investment costs) issues are not triggered by the firm capacity constraint only, we have repeated this case study omitting the firm capacity constraint. We obtain the following results of total annual profits: 0 M€(rMIP); -108.6 M€(rMIP-SR); and, -165.7 M€(MIP). While optimal investments are different, the fact that SRMCs are not enough to recover investment costs in this case, remains true.

The takeaway of these results is that convexity of expansion planning decisions matters. While it might be more convenient to resort to a simplified but convex rMIP model where shadow prices can be easily calculated, we have shown that as soon as one steps away from convexity and towards more realistic models, pricing issues occur, which can lead to an arbitrary distribution of losses and profits. However, in the cases that we have observed, BESS have been on the losing end of the pricing implications of non-convexity. Allowing for long-term costs to be reflected in market prices also has an impact, although our results indicate that this effect is very limited in the current system.

5 Future power system (100% VRE): GEP results and impacts of convexity

In this section, we assess expansion planning and profitability for the future power system assuming 100% VRE sources. Currently, many power systems all over the world are transitioning towards decarbonization. The number of conventional plants (with UC-type decisions and large individual units) is declining. The number of renewable energy resources and energy storage systems, which do not require UC variables and have small individual units, is increasing. Therefore, one might argue that convexity in mathematical formulations is becoming less of an issue. In order to discuss this in more detail, we repeat the above experiment but now enforce a 100% renewable penetration by not including thermal candidate units.

Table 7 Capacity investments (MW) per technology under different modeling paradigms under 100% VRE penetration

Indeed, as can be observed in Table 7 in a system with a 100% renewable mix, the total investments between the fully convex model (rMIP) and the non-convex (due to lumpiness of investments) model (MIP) differ very little. The difference in total system capacity is only 66MW which is 0.2% of total capacity. Note that the rUC case is obsolete here since we do not have any UC variables and it hence coincides with MIP, while the rGEP case coincides with rMIP.

Before we discuss profits, we have to analyze prices, given in Table 8. In the rMIP prices reflect long-run investment costs, and average prices are quite high, i.e., 103.6€/MWh; however, when only short-run costs are accounted for (in the rMIP-SR and the MIP cases), prices only reflect variable costs, which are on average 3.05€/MWh. Note that in this particular case, there is no scarcity, so the SR prices are not inflated by a large penalty for non-supplied energy (NSE)¹³. Figure 2 contains a price duration curve for the rMIP case. Note that this price is a nodal average weighted by demand, and shows that there are 700 hours in which investment costs impact spot market prices, i.e., the spot price is above 6€/MWh which is the highest operating marginal cost in the generation fleet. On the other hand, in the rMIP-SR and MIP cases there is no price instance that exceeds 6€/MWh. This, leads to the apparent problems with long-term cost recovery that we analyze in the remainder of this section.

Table 8 Spot market prices (€/MWh) under different modeling paradigms under 100% VRE penetration

Fig. 2

Average price duration curve (overall curve and detail) in rMIP model under 100% VRE penetration

Profits per technology under a 100% renewable penetration are presented in Table 9. Table 10 contains the BESS profit breakdown for the rMIP case. Note that, in this case, the firm capacity constraint is not binding in the rMIP, and hence there is no firm capacity remuneration. This constraint requires a minimum firm capacity of 110% of hourly system peak demand; however, this amount is way less than the total capacity required actually to satisfy demand overall. In particular, in a 100% renewable system, only BESS and potentially wind (if there is any) can serve demand during the night hours. Therefore, a large amount of BESS capacity needs to be installed, to be charged during the day to provide sustained energy through the night. The amount of BESS capacity necessary to achieve this, i.e., almost 9GW, by far exceeds the 110% of peak demand of 4.5 GW. This result raises the question of whether a firm capacity constraint, as it is proposed in the literature, really serves its purpose in a 100% VRE power systems, in which this constraint is inactive.

While every technology recovers costs in the fully convex rMIP model, this does not apply in the rMIP-SR or the MIP. In summary, if market prices do not reflect long-run investment costs, then no technology recovers costs¹⁴. This is much more problematic in the 100% VRE power system than in the current power system. While with long-run costs, i.e., the rMIP case, all technologies recover costs in both the current and the future power system, when long-run costs are no longer reflected in prices the missing money problem is large in a 100% VRE power system, even in a relaxed model such as rMIP-SR. Comparing profits under rMIP with rMIP-SR clearly shows the impact of no longer accounting for investment costs in prices, which is much worse (2508 M€ losses) in the 100% VRE system than in the current one (only 0.8 M€ losses).

In the MIP model, and for BESS in particular, there were a total of 13.73 M€ losses for 600 MW installed in the current power system, which are relative losses of 22.9 M€/GW. In the future power system, there are relative losses of 64.9 M€/GW, which is worse by a factor close to 3¹⁵. This seems to indicate that there might be a problem with the assumed market design, as short-term spot and reserve market revenues are not enough to offset costs. Finally, as mentioned previously, there are no firm capacity payments in the 100% VRE case as meeting system energy demand is the main investment driver here.

Adequate capacity remuneration for low-carbon power systems is a very interesting but still ongoing topic of research Bothwell and Hobbs (2017); Mertens et al. (2021). The firm capacity constraint implemented here is one particular example or such a policy measure. However, we want to stress that the firm capacity constraint does not affect the profit results of the 100% VRE case presented here (as it is not binding). This would suggest that, here, the long-run cost recovery problems are not triggered by the policy instrument.

Table 9 Annual profits (M€) per technology under different modeling paradigms under 100% VRE penetration

Table 10 Annual profit (M€) elements per technology in rMIP case under 100% VRE penetration

6 Conclusions

As long as we have a significant fraction of conventional thermal plants in our technology mix, the convexity of expansion models is more important in order to approximate the optimal technology mix adequately. In the future power system, relaxing integrality in investment variables actually yields a fairly accurate approximation of the optimal mix. However, our analysis of convexity in GEP formulations reveal challenges regarding prices and profitability, and whether or not long-run investment costs are to be reflected in market prices. Disregarding long-term costs in market prices cause relatively mild profitability issues in the current power system in our analysis, even though they deteriorated in combination with integrality. However, in the future power system, our results indicate that only accounting for short-term costs in prices causes a serious missing money problem, and correct remuneration of generators becomes more challenging. In future work, we will investigate adequate pricing and compensation schemes in fully decarbonized electricity markets. Moreover, we want to assess the impact of price-responsive demand and demand-side management on our results.

Appendix

In this appendix we want to showcase, for a simple example, what is happening with long-run marginal cost and short-run marginal cost for power system planning models with inelastic demand. For simplicity, we consider one one time step, only one generator, no UC decisions or constraints, and continuous variables. With that, problem (1) reduces to problem (3) with positive variables p, x:

$$\begin{aligned} min \quad C^{OM} \cdot p + C^{INV} \cdot x \end{aligned}$$

(3a)

$$\begin{aligned} 0 \le p \le x \quad : ({\underline{\mu }}, {\overline{\mu }}) \end{aligned}$$

(3b)

$$\begin{aligned} p = D \quad : (\lambda ) \end{aligned}$$

(3c)

As the above problem is convex, we replace it with its KKT conditions (4), where (4a)-(4b) correspond to the derivatives of the Lagrangian, (4c)-(4e) are the complementary slackness conditions, (4f) the non-negativity of Lagrange multipliers and (4g)-(4h) the original constraints of the problem.

$$\begin{aligned} C^{OM} - {\underline{\mu }} + {\overline{\mu }} - \lambda = 0 \end{aligned}$$

(4a)

$$\begin{aligned} C^{INV}- {\overline{\mu }} = 0 \end{aligned}$$

(4b)

$$\begin{aligned} p \cdot {\underline{\mu }} = 0 \end{aligned}$$

(4c)

$$\begin{aligned} (p-x)\cdot {\overline{\mu }} = 0 \end{aligned}$$

(4d)

$$\begin{aligned} (p-D)\cdot \lambda = 0 \end{aligned}$$

(4e)

$$\begin{aligned} {\underline{\mu }} \ge 0, {\overline{\mu }} \ge 0 \end{aligned}$$

(4f)

$$\begin{aligned} 0 \le p \le x \end{aligned}$$

(4g)

$$\begin{aligned} p = D \end{aligned}$$

(4h)

If we consider investments x to be a variable (with its corresponding derivative of the Lagrangian), we get that $C^{INV}= {\overline{\mu }}$ from(4b). Substituting this into (4a), we get that marginal cost (or market spot price) $\lambda$ is equal to $C^{OM} + C^{INV} - {\underline{\mu }}$. If we consider a non-trivial solution, i.e., $p>0$, then complementary slackness yields that ${\underline{\mu }}=0$, which yields that market price $\lambda = C^{OM} + C^{INV}$ includes short-run operating costs and long-run investment costs. It is trivial to see that with such a market price, both operating and investment costs are recovered.

Now let us briefly analyze the case, where we fix investment decisions and re-run the model. The resulting optimization model is almost identical to (3) with the only difference that investments x are now considered parameters. To indicate this, let us write investment capacities as capital letters, i.e., X. If we were to take corresponding KKT conditions of this problem, they would be:

$$\begin{aligned} C^{OM} - {\underline{\mu }} + {\overline{\mu }} - \lambda = 0 \end{aligned}$$

(5a)

$$\begin{aligned} p \cdot {\underline{\mu }} = 0 \end{aligned}$$

(5b)

$$\begin{aligned} (p-X)\cdot {\overline{\mu }} = 0 \end{aligned}$$

(5c)

$$\begin{aligned} (p-D)\cdot \lambda = 0 \end{aligned}$$

(5d)

$$\begin{aligned} {\underline{\mu }} \ge 0, {\overline{\mu }} \ge 0 \end{aligned}$$

(5e)

$$\begin{aligned} 0 \le p \le X \end{aligned}$$

(5f)

$$\begin{aligned} p = D \end{aligned}$$

(5g)

Again, assuming a non-trivial solution ($p>0$) we derive that ${\underline{\mu }}=0$. With that, market price $\lambda$ equals $C^{OM} + {\overline{\mu }}$. Let us distinguish two cases now: first, if $p<X$ then complementary slackness would yield that market price is only the short-run operating cost $C^{OM}$; second, if $p=X$ the Lagrangian multiplier ${\overline{\mu }}$ is not uniquely defined. That is the crux of the matter. In the long-run problem (3) investments x are also binding ($p=x$), but since x is a variable, the Lagrange multiplier ${\overline{\mu }}$ reflects investment costs. Now, in the short-run problem, investments are considered constant. Therefore, Lagrange multiplier ${\overline{\mu }}$ no longer reflects investment costs. As a matter of fact, there are infinite solutions for the pair $(\lambda , {\overline{\mu }})$ that satisfy the KKT conditions. This a missing-market problem in the sense that the operational problem (5) has no information about the investment cost and thus it cannot reflect them in the dual/price. There also exists dual degeneracy. In practice, numerical solvers yield ${\overline{\mu }}=0$ and $\lambda =C^{OM}$, which is a valid solution to the KKT conditions. However, as a consequence investment cost recovery is not achieved in this case.