Asymmetric risk and fuel neutrality in electricity capacity markets

Abstract

In many liberalized electricity markets, power generators can receive payments for maintaining capacity through capacity markets. These payments help stabilize generator revenues, making investment in capacity more attractive for risk-averse investors when other outlets for risk trading are limited. Here we develop a heuristic algorithm to solve large-scale stochastic equilibrium models describing a competitive market with incomplete risk trading. Introduction of a capacity mechanism has an asymmetric effect on the risk profile of different generation technologies, tilting the resource mix towards those with lower fixed costs and higher operating costs. One implication of this result is that current market structures may be ill-suited to financing low-carbon resources, the most scalable of which have high fixed costs and near-zero operating costs. Development of new risk trading mechanisms to replace or complement current capacity obligations could lead to more efficient outcomes.

Main

Since the inception of liberalized electricity markets in the late 1990s, merchant investment into new generation capacity in the United States has been dominated by gas-fired units. The simplest explanation for this trend points to marked improvements in the efficiency of combined-cycle plants along with, more recently, low natural gas prices brought about by the shale revolution. Without attempting to compare the magnitude of the two effects, we propose a second explanation: distortions brought about by capacity markets in the absence of more complete risk trading.

Three alternative paradigms govern investments in generation capacity: vertical integration, energy-only markets and installed capacity (ICAP) markets¹. In traditional vertically integrated systems, investors are guaranteed a rate of return by a regulatory process, which effectively shifts most risk to ratepayers and creates the incentive for excess investment. One goal behind deregulation was to encourage a more efficient allocation of this risk². The theoretically ideal competitive market provides the basis for a second paradigm, energy-only markets. Under the energy-only design, generators earn all their revenue through the sale of energy and ancillary services. Prices rise substantially during times of scarcity, which allows generators to recover their fixed costs. In practice, a lack of price-responsive demand, combined with inconsistencies between the market clearing process and actions taken by operators on the grounds of reliability, hampered the formation of efficient scarcity prices^3,4. An administrative response is to introduce an operating reserves demand curve that reflects the probability that the system operator will need to take emergency actions (for example, voltage reductions or rolling blackouts) to prevent a cascading failure⁵. As the probability of such actions approaches one, the price grows to an estimated value of the lost load, which is typically two orders of magnitude larger than average prices (for example, US$9,000 MWh^–1 for the Electric Reliability Council of Texas). With the exception of the Electric Reliability Council of Texas, wholesale market operators in the United States opted for the third paradigm. Under the ICAP design, load-serving entities are required to either supply or procure an administratively determined level of capacity. Energy prices in these markets are capped at a much lower level (for example, US$2,000 MWh^–1 or lower), and revenue from the sale of energy and ancillary services is supplemented by payments for capacity. The intent of these capacity payments is to balance the ‘missing money’ that results from the cap and other price-suppressing actions to restore the outcomes that would be achieved in an ideal energy-only market^6,7,8.

We are not the first to suggest that capacity markets bring about a distortion of the technology mix. Indeed, this assertion is common enough that the overview in Cramton et al.⁷ deems it ‘a deep confusion concerning capacity markets’ that ‘takes many forms and contains contradictory views’. In this context, our contribution lies in formalizing a mechanism by which distortion could arise.

To illustrate this possibility, we developed a stochastic equilibrium model of the type first described in Ehrenmann and Smeers⁹, as well as an algorithm to solve large-scale instances of the problem. The fuel neutrality of capacity markets relative to an ideal energy-only market relies on the assumption embedded in the classic optimization framework that the cost of capital is exogenous to the market design. The model in this article relaxes that assumption, which enables us to take into account the stabilizing effect that the introduction of capacity mechanisms has on generator revenues. Our numerical examples describe a simple system with three generation technologies: a baseload technology with high capital costs but low operating costs (for example, nuclear), a renewable technology with a similar cost profile but variable output (for example, wind) and a peaker technology with low capital costs but high operating costs (for example, gas). Results on these test systems suggest that an increased reliance on existing capacity market structures without the emergence of other forms of risk trading is likely to work against technologies with low operating costs, a category that includes all of the most scalable forms of low-carbon generation.

Resource adequacy and risk trading

Separate from restoring the missing money required for proper investment incentives, advocates for capacity markets point to the benefits that the ICAP design provides in reducing risk. By replacing highly volatile scarcity prices with a more consistent monthly or annual payment, capacity mechanisms provide a financial hedge for both generators and loads. In the traditional analysis, capacity payments have the impact of a call option with a strike price equal to the price cap for the market^2,7,10. Loads make an upfront payment to generators, in exchange for which the effective price they see in the spot market does not exceed the price cap. As energy markets do not produce an accurate underlying price in times of scarcity, real-world capacity markets do not precisely replicate an option. We discuss some of the implementation challenges this causes in Supplementary Note 1. In the ideal case, however, incentives on the margin are maintained: loads must pay high spot prices if they consume more than they have procured in advance, and generators have to buy back their position if they are unable to perform.

It has long been acknowledged that risk aversion could lead to a suboptimal investment in generation capacity^9,11,12. However, prior studies do not demonstrate any consistent technological shift. In energy-only markets, peaking units face substantial risk due to their dependence on the occurrence of high prices in a small number of hours. At the same time, several authors emphasized the natural hedge enjoyed by gas generators, which often set the price of electricity^13,14,15,16. The ability to trade risk is a key element of this discussion. With complete risk trading, along with the standard assumptions of competitive markets, an equilibrium would achieve the (risk-aware) social optimum^17,18,19,20. The connection between equilibrium and optimum implies that the impact of risk aversion on the generation mix can be in any direction depending on the types of risk that most concern market participants. This theoretical result, however, depends on the presence of ‘complete’ markets, which include a security that corresponds to every possible future state of the world. In this context, our focus is less on incompleteness than on asymmetry: the introduction of a capacity market amounts to a trade against a subset of future world states that peaking plants are particularly concerned about. In the PJM interconnection (PJM), for example, a unit with a marginal cost of US$10 MWh^–1 could expect to earn 17% of its operating profits from the capacity market, whereas a unit with a US$100 MWh^–1 marginal cost could expect 90% (Supplementary Note 2). Along these lines, our results suggest a different interpretation of the natural hedge of gas. By itself, the hedge does not confer an advantage. It is only when combined with a capacity mechanism, which removes risk related to the frequency of scarcity events, that the hedge becomes an advantage.

The size of this advantage depends on the availability of other outlets for risk trading. In practice, the evidence suggests that markets in risk are far from complete²¹. A natural resolution to the problem occurs if investors in generation are able to sign long-term contracts with loads, who are exposed to price risk in the opposite direction. With the support of state and corporate procurement programmes, a robust market has developed for renewable generators that seek long-term power purchase agreements²². In areas with competitive retail markets, however, contracts rarely exceed two years outside these programs²³. Small retailers are unable to commit to long-term contracts without risking customer defection in periods when wholesale prices drop below the contracted rate¹¹. The phenomenon of insufficient demand-side interest in hedging is seen as characteristic to commodity markets beyond electricity²⁴, especially in the presence of transaction costs²⁵.

If they are unable to secure a power purchase agreement, merchant generators are often able to hedge risk on timescales of five to seven years through contracts with financial traders able to bear some of the risk²⁶. However, these bilateral deals cannot match the liquidity of centralized capacity markets, which leads some to propose formalized exchanges^27,28. Owing to the proprietary nature of these arrangements, it is a challenge to assess the liquidity in this market. Nevertheless, the available evidence suggests some endogeneity between cash flows and the cost of capital. Among publicly available ratings, debt issued for projects in the energy-only Electric Reliability Council of Texas market receive ratings of B and CCC, whereas projects in PJM typically receive a higher BB rating²⁹. Separate evidence comes from renewable support schemes in Europe. Although some programmes (for example, feed-in tariffs) lead to predictable revenue streams, others (for example, quotas) result in payments that vary with market conditions. The relative ease of financing stable revenues can lead to a greater deployment for the same amount of government support^30,31.

Even if the size of the effect is small at present, insufficient risk trading may have large consequences for long-term market design. In general, long-run models predict that the incorporation of solar and wind will lead to lower average electricity prices but greater volatility, which prompted many to consider how capacity mechanisms may need to evolve to guarantee resource adequacy^{1,32,33,34,35}.

Two-generator example

To illustrate the impact that incomplete markets in risk may have on the capacity mix that arise in equilibrium, we developed two simple test systems. The first features two technologies: a baseload resource with a high capital cost but a low and certain operating cost, and a peaking resource with a low capital cost but a high and uncertain operating cost. The second example, developed in the next section, adds a renewable resource with variable availability. In the context of current US markets, these three technologies can be thought of as existing nuclear plants, many of which are considering retirement in the near term, and new gas plants and wind farms, which are being constructed at a rapid rate. With that said, the parameters were chosen to best illustrate the impact of risk trading, rather than to recreate or predict market outcomes. In keeping with US market design principles, the focus is on economic efficiency, with no explicit environmental goals within the market. Support mechanisms for carbon-free generation are assumed to be reflected through the investment and energy costs associated with each technology. Unlike most US markets, we did not implement a mandatory capacity market: resource decisions are driven directly by the value of load expressed by consumers. Instead, we mimicked a capacity market through a call option with a strike price of US$1,000 MWh^–1. If the cost of capital were exogenous to the market design, this would have no effect on the resulting capacity mix^7,8. We also considered a future settling at US$50 MWh^–1.

For the experiments, the nominal demand is based on the load duration curve of PJM in 2017, during which demand ranged from 58 to 146 GW. The first example focuses on two generation technologies, baseload and peaker, and two sources of uncertainty, demand and the fuel cost of the peaker. For both technologies, 90% of ICAP is available in every time block. Fuel prices and demand shifters are modelled as random variables with 100 equally likely year-long scenarios for the dispatch problem. We considered two contracts: a call option with a strike price of US$1,000 MWh^–1 and a future settling at US$50 MWh^–1. The probabilistic model is discussed in Methods, and additional details on the scenario assumptions are given in Supplementary Note 2 and Supplementary Tables 1 and 2.

To characterize the risk attitude of market participants, we used a weighted sum of the expected value of surplus, with weight β, and conditional value at risk (CVaR) of the α-level tail of surplus, with weight 1 − β. CVaR is a coherent risk measure that lends itself to inclusion in a mathematical optimization model^36,37. Lower values of α and β imply a greater risk aversion, with either α = 1 or β = 1 implying a risk-neutral agent. In real-world contexts, market operators have limited ability to ascertain or influence the risk preferences of market participants. Accordingly, our results do not focus on the effect of risk aversion itself, but instead on the availability of risk trades, a phenomenon that market operators and regulators have a greater potential to address. Along these lines, our priority when choosing α and β was not necessarily to match the risk preferences of real-world market participants, but to offer a clear illustration of the impact of incomplete markets. Tests with a number of values for these parameters yielded the same directional results.

Using risk parameters α = β = 0.5 for all the market participants, the socially optimal mix (that is, the complete trading case) includes a baseload capacity of 89.7 GW and a peaking capacity of 77.6 GW. We tested cases with no trading, each contract individually, and both contracts available, and compared the results to this social optimum. Table 1 shows the capacity mix that arises in each case, the financial trades made by investors in each technology and the decrease in consumer surplus relative to the complete trading case. The Methods section describes the associated mathematical model, solution algorithm and metric measuring proximity of the algorithm’s solution to an equilibrium.

Table 1 Equilibria in two-generator example

Incomplete trading leads to a lower ICAP and significant degradation of surplus. To achieve an equilibrium close to the complete trading ideal, both contracts must be available.

The most striking result in Table 1 occurs in the Option Only case, in which the baseload technology is absent from the equilibrium mix. This observation is particularly important because, as discussed above, the options contract is designed to have the same financial impact as the capacity markets currently utilized or under discussion in many real-world markets. A hint as to the mechanism at work is provided by the trades made in the Both Contracts case. The peaker technology prefers to sell options, selling 78.4 GW of options contracts against only 1.2 GW of futures. The baseload technology exhibits the opposite preference, selling 80.7 GW of futures but no options. Tables 2 and 3 show the reason for these trading preferences. Table 2 shows the payouts for the futures contract in the Both Contracts case. The 50 bold cells in the table represent the worst-case realizations of demand and fuel cost for the baseload technology in the No Trading case. As α = 0.5 in this example, these 50 scenarios are the worst-case scenarios identified by the CVaR calculation for the baseload investors. Analogously, Table 3 shows the payouts for the option contract in the Both Contracts case along with the worst-case combinations of demand and fuel costs for the peaker technology. The difference in the bold cells between the two tables reflects the risk exposure of the two technologies: although investors in the peaking technology are only concerned about lower-than-expected demand, investors in the baseload technology are concerned about both lower-than-expected demand and lower-than-expected fuel cost. As we discuss in Methods section, employing CVaR is equivalent to modifying the nominal probability distribution under a risk-neutral objective function. In this numerical example, the modified probability mass function for each agent corresponds to placing probability $\frac{3}{{200}}$ on the bold scenarios and $\frac{1}{{200}}$ on the regular scenarios in the respective tables (in place of the nominal $\frac{1}{{100}}$ on all scenarios).

Table 2 Alignment of futures payout to risk faced by baseload technology

Table 3 Alignment of option payout to risk faced by peaker technology

To improve their overall risk profile, investors in each technology choose to sell contracts with payouts that align with favourable underlying outcomes. Higher payouts for the futures contract correspond to non-bold cells in Table 2, whereas higher payouts for the options contract correspond to non-bold cells in Table 3. In cases for which only one trade is available, the equilibrium shifts towards the resource with a risk profile that is better balanced by that trade. Accordingly, moving from the No Trading case to the Future Only case increases the baseload capacity from 76.1 to 89.2 GW. The effect is more drastic in moving from the No Trading case to the Option Only case, which brings the peaking capacity from 88.9 to 166.7 GW and eliminates the baseload technology from the mix.

An additional interesting feature of Table 1 is that, under the Option Only case, investors in peaking technology sell a volume of options that exceeds the ICAP. As the assumed availability is 0.9, it is clear that these units cannot physically back up the financial trade. Real-world capacity mechanisms typically require physical assets. From a liquidity standpoint, a related issue is that the two-stage model developed in this article enables generators to hedge over their entire operating life, whereas real-world capacity commitments are often much shorter (for example, one year in PJM).

It is a mistake to object that a short delivery period eliminates the financial hedging property of capacity markets: more important than the cash flow for the agreed period is the volatility of future cash flows. The option payouts in Table 3, which reflect the distribution of annual revenues during scarcity periods that might be expected in an energy-only setting, range from US$0 to 138 kW^–1 yr^–1, with an average of US$41 kW^–1 yr^–1 and a coefficient of variation of 1.12. The presence of a capacity market reduces this volatility substantially. In PJM, for example, capacity market prices have ranged from US$28 to 60 kW^–1 yr^–1 in the six years since the introduction of Capacity Performance³⁸, with an average of US$47 kW yr^–1 and a coefficient of variation of 0.24. That said, both the physical back-up requirement and the short-term nature of real-world commitments make the level of hedging achieved in Table 1 unlikely. Similar to the strategy in de Maere d’Aertrycke et al.³⁹, we mimicked the lack of liquidity by limiting the allowed volume of options trades to a percentage of ICAP for each technology and recalculated the equilibrium mix. As shown in Fig. 1, the amount of baseload capacity in equilibrium declines as liquidity in the options market increases, to reach zero when trade volumes equal 60% of ICAP.

Fig. 1: Decline of baseload capacity with increased options trading.

Total ICAP grows slightly with the availability of options contracts. As these contracts are better suited to the risk profile of the peaker technology, baseload capacity falls to zero.

Three-generator example

In the second example, we added a variable generation resource with four scenarios to govern its availability. The availability scenarios for the variable technology are distinguished in how they are coupled with the load duration curve. Although each scenario has an average availability of 37.5% over time, the four are temporally correlated with a fixed load in the following manner: strongly positive, weakly positive, weakly negative and strongly negative. Availability for the baseload and peaker generators remains at 0.9. Scenarios for fuel and demand are retained from the two-generator example, which results in a total of 400 equally likely year-long second-stage scenarios. Investment cost and risk parameters were updated to ensure that all three technologies are represented in both the socially optimal mix and the No Trading equilibrium. Additional details on these assumptions are given in Supplementary Note 4. In this example, the socially optimal mix includes a baseload capacity of 31.3 GW, peaking capacity of 104.9 GW and variable capacity of 139.2 GW.

In addition to the option and futures contracts used in the previous example, we define a unit contingent contract that matches the availability of the variable generator⁴⁰. The interaction between availability and demand in determining the occurrence of scarcity conditions means it is difficult to determine in advance which combinations of random variable realizations will represent the worst-case scenarios for market participants. However, the intent is to structure one contract that matches the risk profile of each technology: futures to the baseload technology, options to the peaking technology and unit contingent to the variable technology. The equilibrium results are split into two tables, oriented in order of increasing surplus. Table 4 shows the capacity mix that arises when at most one trade is available. Comparing the bold cells within each row, it can be seen that introducing any of the trades individually results in more capacity of the corresponding technology. The market share of the baseload unit collapses after the introduction of the unit contingent contract. As in the two-generator example, introduction of the option contract (analogous to current capacity markets) results in a complete exit of the baseload technology.

Table 4 Equilibria in the three-generator example for cases with 0 and 1 contract

Table 5 shows the results for the cases in which two or three contracts are available. Here the bold cells demonstrate the effect of introducing the third contract. In all three pairwise comparisons, the addition of the third contract shifts the capacity mix towards the corresponding technology.

Table 5 Equilibria in the three-generator example for cases with 2 and 3 contracts

Although the directional pattern holds when moving from one to two available trades, the magnitudes are in some instances weaker. In the most extreme example, the Future case and the Future + Unit case result in precisely the same equilibrium. With that capacity mix, the worst-case outcomes for the consumer involve a low output for the variable technology. Accordingly, the consumer is reluctant to purchase unit contingent contracts and none are traded.

The trades that emerge in the All Contracts case echo the results of the two-generator example. Each technology prefers contracts of the trade adapted to its risk profile: investors in the baseload technology sell 38.5 GW of futures, investors in the peaker sell 96.7 GW of options and investors in the variable technology sell 91.8 GW of unit contingent contracts. Unlike the two-generator example, in which the Both Contracts case reached an equilibrium close to the social optimum, there remains a US$0.6 billion yr^–1 gap between the All Contracts case and the complete trading solution. Additional contracts are required to bridge this divide.

Conclusion

This article argues that the introduction of a capacity market will tilt the technology mix that arises in equilibrium towards resources with higher operating costs. The syllogism proceeds as follows. The financial impact of a capacity mechanism is to replace volatile scarcity prices with more regular revenues. The higher a unit’s operating costs, the more its cost recovery in an energy-only market relies on scarcity prices. Therefore, the introduction of a capacity market has a stronger impact on the risk profile of technologies with higher operating costs.

The magnitude of this effect in real-world markets is difficult to evaluate due to its dependence on the risk tolerance of market participants, their evaluation of potential future scenarios and the availability of other outlets for risk trading. However, this article’s numerical experiments reveal the potential for substantial shifts, as both examples exhibit a significant increase in capacity of the peaking technology after the introduction of an options contract. The struggles of US nuclear units, which have been unable to find counterparties for long-term contracts without state support despite their potential role as a hedge against future increases in natural gas and carbon prices, suggests that the mechanism demonstrated in this article may be at work in current markets. This result is particularly important in the context of efforts to reduce carbon emissions. The majority of energy in low-carbon systems is likely to be provided by some combination of hydroelectric, nuclear, wind and solar resources, all of which are characterized by high capital costs and low operating costs. Accordingly, capacity markets as currently structured may work against efforts to decarbonize.

To achieve an efficient capacity mix requires the ability to share risk. Integrated resource planning, which can be made versatile enough to incorporate whatever sources of risk are viewed to be most salient, is one way to do so. A second, modelled in this article, is an energy-only market accompanied by the robust trade of instruments adapted to the risk profile of customers and various technologies. Neither of these strategies is without challenges. Capacity markets can offer a partial solution to the problem of risk sharing. The results of this study, however, reveal the consequences that choices made in the design of these markets may have for the resource mix that arises in liberalized electricity systems.

Methods

Modelling framework

Here we formulated a multiple-agent variant of a two-stage stochastic program for capacity expansion. In the first stage, risk-averse investment and financial trading decisions are made by a number of agents. Spot market prices are determined through a perfectly competitive dispatch over time blocks, $t \in T$, performed by the system operator in the second stage. The ultimate source of all generator revenue in the model is energy sales; financial contracts settle based on the prices realized in the second stage. Instead of writing the optimality conditions for each of the subproblems and reformulating as a complementarity problem^9,41,42, we anticipate the algorithm that we describe later in this section and focus on the problems solved by each agent.

Our modelling framework builds most directly on the approaches of de Maere d’Aertrycke and co-workers^41,]42, which describe a risk-averse capacity equilibrium with incomplete markets. We make two modifications of note, which we describe in detail after introducing the model. Incomplete trading has significant ramifications, both computationally and conceptually. The computational issue is that, instead of a comparatively simple optimization problem, the problem is formulated using a construct called a multiple optimization problem with equilibrium constraints⁴³. Although modest-sized instances can be solved by the PATH solver⁴⁴, numerical tests on larger instances can fail to converge. To help address this issue, we proposed a heuristic decomposition algorithm similar in spirit to that of Höschle et al.⁴⁵, which allows the identification of equilibria through a series of convex quadratic programs. Although the test systems used in this article are kept simple, this decomposition approach allows an easier extension to a richer set of technologies and risks. The conceptual concern is that the uniqueness of the equilibrium solutions cannot, in general, be guaranteed. We revisit this topic after introducing the algorithm. The optimization models are implemented in AMPL⁴⁶ and solved using CPLEX⁴⁷.

In the three-generator example, uncertainty comes from three elements. The random variable D⁺, with scenarios indexed by $s \in S$, reflects an upward demand shift in all the time blocks. The random variables A_gt, with scenarios indexed by $r \in R$, denotes the availability of each generator in a given time block. The random variables $C_{g}^{\mathrm{EN}}$ and D⁻, with scenarios indexed by $f \in F$, comprise generator fuel costs as well as the downward demand shift perfectly correlated with higher prices. These three sources of randomness are assumed to be independent and to have finite support. In the two-generator example, randomness is modelled in the same way except that it does not include the random variables A_gt. Each market participant, $a \in A$, uses a coherent risk measure ρ_a in its decision problem. As shown in Artzner et al.³⁶, this corresponds to choosing the worst-case distribution in a risk set, which takes the form of a convex subset of probability measures on $F \times R \times S$. The risk measures we specifically chose are convex combinations of expected value and CVaR of the surplus (Philpott et al.¹⁸ and de Maere d’Aertrycke et al.³⁹ give additional examples of this construction). The CVaR portion of the calculation identifies the worst (100α) percentage of second-stage outcomes for the market participant in question, and calculates the average surplus only among those scenarios. Thus, starting from a shared nominal distribution, each participant performs a risk-averse optimization that implicitly assigns a higher probability to the combinations of fuel, availability and demand scenarios most harmful to their interest. The difference in expected profit when evaluated using this adjusted distribution versus the nominal distribution corresponds to the risk premium required by investors. This endogenous representation of the risk premium stands in contrast to the classical capacity expansion framework, in which investment cost is annualized according to an exogenous cost of capital.

Dispatch

The economic dispatch (ED) problem was formulated as a convex quadratic program, the outcome of which depends on the realization of the random variables that govern demand, availability and fuel costs. The problem splits operations into simple time blocks of varying length, which reflect the different levels of load seen throughout the year without explicitly representing operational considerations, such as unit commitment decisions, which in general preclude equilibria, or ramping constraints, which can exacerbate the issue of multiple equilibria. In general, the inclusion of these constraints leads to a greater need for capacity, but their implications for the resource mix are not well understood⁴⁸.

Notation

Sets:

$g \in G$, set of all generation technologies

$t \in T$, set of time blocks

$f \in F$, set of scenarios for fuel prices

$r \in R$, set of scenarios for generator availability profiles

$s \in {\cal{S}}$, set of scenarios for demand shifters

Parameters:

B, value of non-price-responsive load (US$ MWh^–1)

L_t, length of time block t (h)

$D_t^{{\rm{fix}}}$, baseline level of non-price-responsive load in time block t (MW)

$D_t^{{\rm{res}}}$, amount of price-responsive demand, which bids at a value declining linearly from B to 0 (MW)

Scenario-specific parameters (that is, realizations of random variables):

A_grt, availability of generator g in time block t under availability profile scenario r (%)

$C_{fg}^{{\rm{EN}}}$, marginal cost for generator g under fuel price scenario f (US$ MWh^–1)

$D_f^ -$, downward shift in demand in all time blocks under fuel price scenario f (MW)

$D_s^ +$, upward shift in demand in all time blocks under demand shifter scenario s (MW)

Variables:

y_gt, production by technology g in time block t (MW)

$d_t^{{\rm{fix}}}$, amount of non-price-responsive demand cleared in time block t (MW)

$d_t^{{\rm{res}}}$, amount of responsive demand cleared in time block t (MW)

Capacity expansion decision (fixed in dispatch model):

x_g, quantity installed of generation technology g (MW)

Formulation

The ED problem, with H_frs denoting the total surplus given scenarios f for fuel price, r for generator availability and s for demand, is stated as:

$$({\rm{ED}})_{frs}\quad H_{frs} = \quad \mathop {{{\mathrm{maximize}}}}\limits_{y,d} \quad \mathop {\sum}\limits_{t \in T} {L_t} B\left( {d_t^{{\rm{fix}}} + d_t^{{\rm{res}}} - (d_t^{{\rm{res}}})^2/(2D_t^{{\rm{res}}})} \right)$$

$$- \mathop {\sum}\limits_{t \in T} {\mathop {\sum}\limits_{g \in G} {L_t} } C_{fg}^{{\rm{EN}}}y_{gt}$$

(1)

$${{{\rm{subject}}\,{\rm{to:}}}}\quad d_t^{{\rm{fix}}} + d_t^{{\rm{res}}} + D_s^ + - D_f^ - = \mathop {\sum}\limits_{g \in G} {y_{gt}} \qquad \forall t \in T$$

(2)

$$0 \le y_{gt} \le A_{grt}x_g\qquad \forall g \in G,t \in T$$

(3)

$$0 \le d_t^{{\rm{fix}}} \le D_t^{{\rm{fix}}}\qquad \forall t \in T$$

(4)

$$0 \le d_t^{{\rm{res}}} \le D_t^{{\rm{res}}}\qquad \forall t \in T$$

(5)

Equation (1) includes a linear term for the value of non-price-responsive load, a quadratic expression for the value of price-responsive load and a linear term for the cost of producing power. The responsive load is the first departure from the model in de Maere d’Aertrycke et al.⁴¹. Its presence does not carry any philosophical significance: it is introduced for computational convenience, to help avoid the inevitability of multiple optimal dual solutions in the linear version of the model discussed in Abada et al.⁴². The power balance constraint in equation (2) splits demand into four segments: $d_t^{fix}$, $d_t^{res}$, $D_s^ +$ and $D_f^ -$. The second, more meaningful departure from de Maere d’Aertrycke et al.⁴¹ is the absence of the terms $D_s^ +$and $D_f^ -$ from the objective function. This choice implies that consumers do not experience any gain or loss of surplus as a result of these exogenous shocks to demand, which has important consequences when comparing our numerical results to those of de Maere d’Aertrycke et al.³⁹, which are based on the model in de Maere d’Aertrycke et al.⁴¹. Conceptually, the question is which events in the sample space $F \times R\times S$ should customers be most concerned about. Upward shocks to demand could come from, for example, a higher-than-expected economic growth that leads to more productive uses of electricity. In this case, consumers might not be concerned about high-demand scenarios, as shortages would be balanced by an increase in surplus from consumption. An upward shock could also come, however, from an increased use of air conditioning due to higher summer temperatures, which consumers might experience as neutral or even a loss. By excluding these shocks, we ensure that consumers concentrate on the high-demand scenarios in which shortages occur more frequently. This definition of consumer surplus appears subsequently in the consumer model (CON). The optimal values of the ED model across all the scenarios define the social surplus (SOC) used in the complete trading model.

Contracts

In conjunction with the capacity investment decisions made in the first stage, investors can sign contracts with consumers that settle based on the energy price in the second stage. Assume a small set of contracts $K$ is available to trade. Let λ_frst be the price of energy in time block t given fuel price f, profile r and demand shifter s, that is, the dual variable that corresponds to the power balance constraint in equation (2) divided by the length L_t, and let $\eta _{frs}^k$ represent the payout of contract $k \in K$ given this scenario. A call option gives the purchaser the right to buy a unit of energy for a predetermined price if the spot price exceeds that level. Thus, if contract k indexes a call option that covers all the time blocks with strike price λ^k, we can calculate:

$$\eta _{frs}^k = \mathop {\sum}\limits_{t \in T} {L_t} \max \{ 0,\lambda _{frst} - \lambda ^k\}$$

(6)

If k instead indexes a futures contract that covers all time blocks at price λ^k, the payout is calculated as:

$$\eta _{frs}^k = \mathop {\sum}\limits_{t \in T} {L_t} (\lambda _{frst} - \lambda ^k)$$

(7)

We also define a unit contingent contract that tracks the availability of generator g, a construct often used for variable technologies. When k indexes a unit contingent contract for generator g, the payout is calculated as:

$$\eta _{frs}^k = \mathop {\sum}\limits_{t \in T} {A_{grt}} L_t(\lambda _{frst} - \lambda ^k)$$

(8)

Market participants

Each market participant solves a convex quadratic program, choosing a quantity of financial contracts that maximizes its risk-adjusted surplus. Generator capacities, outputs of the dispatch model for each scenario and the prices and payouts of contracts are fixed parameters in these models. We define notation here that can be shared between the consumer and generator models, that is, among all market participants.

Notation

Sets:

$a \in A$, set of market participants (generators and consumers)

$k \in K$, set of contracts

Parameters:

α_a, tail probability at which CVaR is evaluated by market participant a, 0 < α_a ≤ 1

β_a, weight given to the expected value in the risk measure for market participant a, 0 ≤ β_a ≤ 1

γ, weight on the term that penalizes imbalance in the contract volumes between market participants

$\underline v _a^k,\bar v_a^k$, minimum and maximum volume of contract k to be purchased or sold by market participant a (MW)

p_frs, nominal probability of scenario (f,r,s)

$C_g^{{\rm{INV}}}$, investment cost for generator g annualized at risk-free rate (US$ MW^–1)

Provisional parameters (that is, values calculated by other agents):

λ_frst, price of energy in time block t under scenario (f,r,s) (US$ MWh^–1)

π_fgrst, operating profit for technology g in block t under scenario (f,r,s) (US$ MW^–1)

$\eta _{frs}^k$, payout of contract k under scenario (f,r,s) (US$ MW^–1)

ϕ^k, price of contract k incurred in the first stage (US$ MW^–1)

Variables:

$v_a^k$, volume of contract k purchased or sold by market participant a (MW)

VaR_a, value at risk for market participant a (US$)

$u_{frs}^a$, surplus for market participant a under scenario (f,r,s) (US$)

$u_{frs}^{a + }$, auxiliary variable used in the calculation of value at risk (VaR) (US$)

Consumer model

We distinguished consumer and generator agents by using $c \in A$ and $g \in A$ in place of the generic agent a, and we assume a single consumer; that is, in terms of our notation, $A = G \cup \{ c\}$. Given the assumption of perfect competition, the resulting decisions are equivalent to those of a larger number of small, identical load serving entities. Defining ρ_a to be the risk measure associated with market participant a, the consumer’s problem is stated as follows:

$$({\rm{CON}})\quad \rho _c =$$

$$\mathop {{{\mathrm{maximize}}}}\limits_{v_c,u^c,u^{c + },{\rm{VaR}}_c} \quad (1 - \beta _c)\left( {{\rm{VaR}}_c - 1/\alpha _c\mathop {\sum}\limits_{f \in F} {\mathop {\sum}\limits_{r \in R} {\mathop {\sum}\limits_{s \in S} {p_{frs}} } } u_{frs}^{c + }} \right)$$

$$+ \beta _c\left( {\mathop {\sum}\limits_{f \in F} {\mathop {\sum}\limits_{r \in R} {\mathop {\sum}\limits_{s \in S} {p_{frs}} } } u_{frs}^c} \right) - \gamma /2\mathop {\sum}\limits_{k \in K} {\left( {\mathop {\sum}\limits_{a \in A} {v_a^k} } \right)^2}$$

(9)

$${{{\rm{subject}}\,{\rm{to}}}}\:\quad u_{frs}^c = - \mathop {\sum}\limits_{k \in K} {v_c^k} \left( {\phi ^k - \eta _{frs}^k} \right)$$

$$+ \mathop {\sum}\limits_{t \in T} {L_t} B\left( {d_t^{{\rm{fix}}} + d_t^{{\rm{res}}} - (d_t^{{\rm{res}}})^2/(2D_t^{{\rm{res}}})} \right)$$

$$- \mathop {\sum}\limits_{t \in T} {L_t} \lambda _{frst}\left( {d_t^{{\rm{fix}}} + d_t^{{\rm{res}}} + D_s^ + - D_f^ - } \right)\quad \forall f \in F,r \in R,s \in S$$

(10)

$${\rm{VaR}}_c - u_{frs}^c \le u_{frs}^{c + }\quad \forall f \in F,r \in R,s \in S$$

(11)

$$0 \le u_{frs}^{c + }\quad \forall f \in F,r \in R,s \in S$$

(12)

$$\underline v _c^k \le v_c^k \le \bar v_c^k\quad \forall k \in K$$

(13)

The consumer maximizes a convex combination of CVaR and expected value of surplus, subtracting a proximal term that penalizes imbalances between the contracts bought and sold by the market participants. In equilibrium, this third term of equation (9) must equal zero. Although similar in appearance, the inclusion of this proximal term does not technically yield an augmented Lagrangian, because the market clearing condition for financial contracts is not part of the consumer’s true objective function. Constraint (10) calculates the surplus for the consumer in every scenario that results from the purchase of contracts in the first stage and energy in the second. Constraints (11) and (12) dictate the value of auxiliary variables used in the CVaR calculation. Constraint (13) dictates a minimum and maximum volume of each contract. Although we discuss specific choices of these parameters in Supplementary Note 3, an algorithmic advantage of including these constraints on trading is to guarantee that all the subproblems are bounded. Note that v_c^k is a decision variable in model (CON), but the contract volumes for the generators, which appear in the objective function’s final term, enter model (CON) as provisional parameters.

Generator model

We modelled a single investor in each generation technology $g \in G$. As in the case of the consumer, investment decisions can equivalently be represented as a large number of identical firms. The investor’s problem is stated as:

$$({\rm{GEN}})_g\quad \rho _g =$$

$$\mathop {{{\mathrm{maximize}}}}\limits_{v_g,u^g,u^{g + },{\rm{VaR}}_g} \quad (1 - \beta _g)\left( {{\rm{VaR}}_g - 1/\alpha _g\mathop {\sum}\limits_{f \in F} {\mathop {\sum}\limits_{r \in R} {\mathop {\sum}\limits_{s \in S} {p_{frs}} } } u_{frs}^{g + }} \right)$$

$$+ \beta _g\left( {\mathop {\sum}\limits_{f \in F} {\mathop {\sum}\limits_{r \in R} {\mathop {\sum}\limits_{s \in S} {p_{frs}} } } u_{frs}^g} \right) - \gamma /2\mathop {\sum}\limits_{k \in K} {\left( {\mathop {\sum}\limits_{a \in A} {v_a^k} } \right)^2}$$

(14)

$${\mathrm{subject}}\,{\rm{to}}\:\quad u_{frs}^g = - C_g^{{\rm{INV}}}x_g - \mathop {\sum}\limits_{k \in K} {v_g^k} \left( {\phi ^k - \eta _{frs}^k} \right)$$

$$+ \mathop {\sum}\limits_{t \in T} {{\mathrm{\pi}} _{fgrst}} x_g\quad \forall f \in F,r \in R,s \in S$$

(15)

$${\rm{VaR}}_g - u_{frs}^g \le u_{frs}^{g + }\quad \forall f \in F,r \in R,s \in S$$

(16)

$$0 \le u_{frs}^{g + }\quad \forall f \in F,r \in R,s \in S$$

(17)

$$\underline v _g^k \le v_g^k \le \bar v_g^k\quad \forall k \in K$$

(18)

The consumer and generator models differ only in the calculation of scenario surplus in equations (10) and (15). For simplicity, we employed the operating profit π_fgrst, calculated as the dual variable to the maximum generation constraint in the dispatch model, equation (3), multiplied by availability A_grt.

Decomposition approach

With the assumption of perfect competition, investors in technology g will build new capacity until the risk measure ρ_g = 0 in equilibrium. Thus, the challenge is to identify values for the ICAP x and contract prices ϕ that balance the market in all financial contracts and result in zero risk-adjusted profit for all generation technologies, given the prices and operating profits that arise from the dispatch model. We refer to this equilibrium problem as (EQ).

Algorithm 1 shows the solution approach that we employ. The algorithm chooses a capacity mix, finds the implied dispatch solutions, identifies the prices that balance the markets for financial contracts (to tolerance ε) and then updates the capacity mix based on profitability. With capacity decisions made in the outer loop, one interpretation of the algorithm is that it simulates the process of entry and exit in competitive markets. The use of a proximal term in the objective function for the market participants, as well as the sequential updating, invites comparison to the alternating direction method of multipliers^49,50, an avenue pursued on a model similar to ours in Höschle et al.⁴⁵. As in Höschle et al.⁴⁵, convergence is not guaranteed, but was achieved in all the numerical tests after limited experimentation with different step sizes σ and μ. We compare our approach to that of Höschle et al.⁴⁵ in Supplementary Note 5.

Algorithm 1

Input: An instance of (EQ) defined by models (ED), (CON), and (GEN).

Output: Near-equilibrium solution to (EQ)

 Define σ, μ, δ, ε > 0; let $\rho _a = 0:\forall a \in A$; initialize x, ϕ

 loop

 $x_g \leftarrow \{max { 0,x_g + \mu \rho _g/C_g^{{\rm{INV}}}}\} \quad \forall g \in G$

 solve (ED)_frs; update $\lambda _{frst},\pi _{fgst},\eta _{frs}^k:\forall (f,r,s) \in F \times R \times S,k \in K$

 solve (CON)

 solve $({\rm{GEN}})_g\quad \forall g \in G$

 while $\max _{k \in K}|\mathop {\sum}\limits_{a \in A} {v_a^k} | > \varepsilon$ do

 $\phi ^k \leftarrow \phi ^k + \sigma \mathop {\sum}\limits_{a \in A} {v_a^k} \quad \forall k \in K$

 solve (CON)

 solve $({\rm{GEN}})_g\quad \forall g \in G$

 end while

 if $\max _{g \in G}|\rho _g{\mathrm{|}} < \delta$ then

 return x and ϕ

 end if

 end loop

The potential for multiple equilibria poses a challenge for the interpretation of the numerical results⁵¹. In an effort to avoid this possibility, we omit intertemporal constraints from model (ED) and maintain a constant merit order through all the scenarios. Under these assumptions and with no financial trading, uniqueness can be shown for the model in Abada et al.⁴². In both numerical examples, capacity is initialized at the same, socially optimal starting point for all the cases within each example, with the goal to avoid a spurious result. Contract prices are initialized at their expected payouts using the nominal probability distribution. Starting from alternative points in ad hoc tests did not uncover alternative equilibria.

Complete trading

For comparison, we also constructed a model that assumes complete trading. In this setting, an equilibrium can be identified through a risk-averse optimization problem without the need for equilibrium constraints, using the intersection of the risk sets of all market participants^17,18,19,20. This can be represented as a single large-scale optimization problem that comprises investment decisions and the dispatch model for each scenario. To be concise, we employed as a variable H_frs to indicate the surplus that arises from the dispatch using the chosen amount of capacity x. Replacing this variable with the objective function from (ED)_frs and inserting the constraints in (ED)_frs for each scenario recovers the complete model. With subscript i denoting societal risk preferences, the complete trading model is written as:

$$({\rm{SOC}})\quad \rho _i =$$

$$\mathop {{{\mathrm{maximize}}}}\limits_{x,y,d,H,u^i,u^{i + },{\rm{VaR}}_i} \quad (1 - \beta _i)\left[ {{\rm{VaR}}_i - 1/\alpha _i\mathop {\sum}\limits_{f \in F} {\mathop {\sum}\limits_{r \in R} {\mathop {\sum}\limits_{s \in S} {p_{frs}} } } u_{frs}^{i + }} \right]$$

$$+ \beta _i\left[ {\mathop {\sum}\limits_{f \in F} {\mathop {\sum}\limits_{r \in R} {\mathop {\sum}\limits_{s \in S} {p_{frs}} } } u_{frs}^i} \right]$$

(19)

$${\mathrm{subject}}\,{\mathrm{to}}\quad \quad u_{frs}^i = - \mathop {\sum}\limits_{g \in G} {C_g^{{\rm{INV}}}} x_g + H_{frs}\quad \forall f \in F,r \in R,s \in S$$

(20)

$${\rm{VaR}}_i - u_{frs}^i \le u_{frs}^{i + }\quad \forall f \in F,r \in R,s \in S$$

(21)

$$0 \le u_{frs}^{i + }\quad \forall f \in F,r \in R,s \in S$$

(22)

Convergence

As is characteristic of the alternating direction method of multipliers, the algorithm exhibits slow convergence near the equilibrium. For cases in which the termination criterion δ in Algorithm 1 is not met, we report the solution found at iteration 50,000 of the outer loop. As a measure of proximity to equilibrium, we compute the value:

$${\mathrm{max}} _{\{ g \in G:x_g > 0\} }\quad \left| {\frac{{\rho _g}}{{C_g^{{\rm{INV}}}}}} \right|$$

(23)

for each case, where ρ_g is the risk measure calculated by problem (GEN)_g.