Roughly a year ago, Spain and Portugal went dark when the electrical grid of the entire Iberian Peninsula failed. While the grid operators did a heroic job of restarting the grid quickly, there were obvious questions about what had led to the blackout in the first place. A preliminary report suggested that a combination of grid-level voltage oscillations and early disconnections was the main factor.
Over the weekend, the European grid coordinator, ENTSO-e, released its final, detailed report on the event. While it’s largely consistent with the preliminary conclusions, the report provides much more detail about what went wrong and, more significantly, offers a clear picture of how the Iberian grid operators could make changes to prevent a similar event in the future.
Oscillations
The expert committee that prepared the report had access to a wealth of data, including status logs from most of the major hardware on the Spanish and Portuguese grid, often recorded with sub-second precision. There’s also data from the two major interchanges between the Spanish grid and those in France and Morocco. The group even obtained data from two manufacturers of the small inverters used for rooftop solar about the performance of their hardware on the day in question.
This allowed the committee to track how all the major facilities on the Iberian grid behaved in the critical hours before the blackout and compare their behavior with what was expected under the rules governing grid operations.
One of the key issues the committee examined was oscillations in the grid, as noted in the preliminary report. These occur because, while the grid typically displays steady, average behavior, that stability comes from many independent pieces of hardware that may drift from the average. Based on the exact setup of the grid at any given moment, it’s possible for the drift of multiple components to move in harmony, reinforcing any deviation and creating a measurable effect.
The grid naturally forces deviations back toward the average, but this response can overshoot, leading to deviations in the opposite direction. This can produce oscillatory behavior, as all the hardware that’s in harmony continually adjusts and goes too far.
One of the two oscillations the grid experienced on the day of the blackout stems from a known phenomenon in which both the eastern and western edges of the European grid oscillate relative to its center. This has a known frequency, and it clearly showed up in the data.
There was a second oscillation, however, and the committee used grid data to trace its origin back to a single interface between the grid and a generating station in Spain near the Portuguese border. The group concludes it was likely a fault in an inverter that takes the direct current generated by photovoltaics and converts it to the alternating current used by the grid.
These oscillations did not cause the blackout. But oscillations in general increase the risk of problems; while the average behavior of the grid may remain within an acceptable range, the extremes of the peaks and troughs of the variations in frequency or voltage can force hardware outside its normal operating range and potentially lead to a disconnect from the grid. So grids have procedures for dampening oscillations, and those went into effect in Spain.
Over-reactive
To understand what comes next, we need to review a key point: An alternating current grid delivers two types of power. One of those, active power, is easy to understand, as it does useful things like charge your batteries or turn a motor. The active energy takes a one-way trip through the grid before being converted to some sort of work. The second, reactive power, is power absorbed by components on the grid but not used, doing things like charging capacitors or powering electromagnets. It can potentially be returned to the grid at some point.
(If you want to learn more about reactive power, check out this video from Practical Engineering.)
In Spain, the process for dampening the oscillations increased the amount of reactive power on the grid, raising the voltage on the main transmission lines. There are ways of removing active power from the grid, and a number were either active or available to the grid operators.
Spain’s high-capacity transmission grid runs at 400 kiloVolts and is meant to remain within about 20 kV of that number during normal operating conditions. A clear increase in voltage followed the first set of oscillations, but over about 15 minutes, the situation returned to normal. The system only stayed there for about five minutes before the second oscillations hit, though, and then began rising again, this time peaking at over 420 kV.
Levels on the grid as a whole quickly started recovering toward the normal 400 kV, but many generating systems registered voltages above their operating ranges and began disconnecting from the grid. Within two minutes, voltages shot to well outside of the normal operating range, and the blackout became inevitable.
What went wrong
The committee is clear that there was no single cause of the blackout (in fact, the report includes a “cause tree” encompassing 16 major events). But several issues stand out as obvious. First, a variety of generating facilities can also consume reactive power and could have helped keep the voltage within a tolerable range. But Spanish grid policy assigned fixed values to any renewable generating sources rather than allowing them to adapt to contingencies. As this was an early spring day, the majority of the power on the Spanish grid came from renewable sources, leaving the system with less flexibility.
Similarly, some equipment is specifically designed to remove reactive power from the grid, and the Spanish grid had a large number of so-called “shunt reactors” available. But these were operated manually, and events during the grid’s collapse happened so quickly that roughly two-thirds of the grid’s shunt reactor capacity remained available as the blackout became inevitable.
Another major issue arose from the Spanish grid’s narrow window between recognizing problems and systems beginning to drop offline. The main transmission lines were meant to operate at 400 kV but also tolerate some variability around that.
“The overvoltage alarm thresholds at 400 kV nodes in Spain were, depending on the node, at thresholds of 420 kV, 430 kV, or 435 kV,” the report notes. Some of the hardware, however, is allowed to disconnect when voltages reach 430 kV. “The safety margin between the allowed voltage operating range and the voltages at which generators could disconnect was low [five kilovolts] or non-existent,” as the report puts it.
The situation was made worse because some of the hardware that disconnected first didn’t properly respond to the high-voltage readings. Grid policy dictates how much the voltage needs to exceed normal ranges and for how long that excess must last before a disconnection is allowed.
The committee looked at 19 pieces of hardware that disconnected during a critical 12-second period immediately prior to the grid’s collapse. Only four seemed to behave exactly as required by grid policy; nine clearly did not. Collectively, these nine removed nearly 1.9 gigawatts of generation from the grid.
What about renewables?
At the time of the collapse, many were quick to point the finger at the high penetration of renewable power in Spain. It’s definitely true that the hardware associated with renewable generation played a major role, but that was inevitable. Spring typically brings lower heating and cooling demand, along with decent renewable energy output, which tends to be the cheapest form of power on the grid when available.
But while renewable power was heavily involved, most of the factors weren’t specific to the form of generation. Many factors, like the use of shunt reactors, have nothing to do with generation at all. Others, such as the premature disconnections and the oscillation-generating problem, come from the hardware that sits between renewable generation and the grid and reflect either faulty hardware or poor configurations. And the policy that limited the amount of reactive power that could be absorbed by renewables plants is just that: a policy issue.
Most of these issues can be improved through a combination of software and policy changes, and the report makes some suggestions along those lines.
The inertia provided by generators with lots of spinning metal—think hydro or natural gas turbines—is generally thought of as improving the stability of the grid, but this analysis suggests that even tripling the amount of inertia would have only dampened the system’s oscillations by about 3 percent. So it’s not clear that having more traditional power online would have helped.
That said, there is one area where potential problems were clearly assigned to one form of renewable generation: rooftop solar. The problem there is less that the hardware wasn’t following policy and more that there’s no real policy being followed. Red Eléctrica, the Spanish grid operator, estimates that it has about 6.5 GW of small-scale (< 1 MW) solar on the grid, with 75 percent (4.9 GW) connected to low-voltage, consumer-level grids. The committee got data from two inverter manufacturers, which collectively track the performance of about 15 percent of that capacity.
This data shows that a substantial fraction (over 12 percent) of one of the manufacturer’s hardware dropped off the grid during the first oscillations and reconnected a few minutes later. Shortly after that, over 20 percent disconnected again during the voltage peak that occurred about two minutes before the blackout. In contrast, the fraction of the second manufacturer’s hardware that dropped off the grid never exceeded 10 percent.
All of this suggests that small-scale generation could have been seeing hundreds of Megawatts of production drop off and hop back on the grid in the minutes leading up to the blackout, with the exact numbers highly dependent on the inverter manufacturers—and that the grid operator has a limited window into their actual behavior. This is a case where increased regulation is probably needed.
Putting learning into practice
The report is encouraging in that it identifies numerous fixes that should be fairly easy to implement, including greater automation of shunt reactors, wider safety margins between alarms and disconnection, and better alignment between grid policies and hardware behavior. And it doesn’t appear to identify any critical issues that will require a rethink of Spain’s approach to getting its grid to net zero.
Economics will also likely help the situation. Spain presently has little in the way of battery capacity, which can perform multiple functions in stabilizing the grid. But the continued growth of renewables will increasingly lead to the overproduction that makes batteries economically viable.
The biggest question appears to be how quickly Spain can implement some of the recommendations in the report.
