Spikes in demand for power and unexpected dips in supply have plagued electricity generators and their customers for decades. The solutions have been crude. More than a decade ago North American power companies started paying big consumers to switch off machines and devices to ease the load on creaking grids. In 2003 French producers did the same to cope with a heat wave.

In some ways the problem has worsened. The rise in the use of renewable power, especially in Europe, has led to surges of supply on sunny and windy days and unpredictable lulls in conditions of cloud and calm. But that is a big opportunity for "demand-response" companies, which use computing power and clever algorithms to divert electricity from some consumers, such as factories or greenhouses, to users who need it more.

Pioneered by technology firms rather than power producers, demand-response is strongest in North America, but it is spreading fast. In a control room in London's fashionable Soho district, Ziko Abram of KiWi Power shows off a "virtual" power plant with a capacity of more than 100 megawatts (MW). KiWi pays users for agreeing to switch off cooling and heating, pumps and other equipment when asked. A switch installed on a user's premises might be programmed to cut off power to freezers, for example, when they are cold enough. In other cases, KiWi negotiates the cutoff with the consumer. It sells the spare capacity thus created to the National Grid.

Along with competitors such as Flexitricity, KiWi also buys the right to use standby diesel generators in hospitals, government buildings and elsewhere. Rather than waste fuel by testing these machines every week, these institutions let KiWi's technology switch them on when the grid requires. In power-poor South Africa an American firm, Comverge, has created a market in which companies can sell the electricity they choose not to consume.

Such services cope with spikes in consumption, which are mostly foreseeable (typically 18 hours in advance). Fluctuations in wind and cloud can cause gluts or shortages within minutes. The conventional way to offset this is through gas, hydro or modern coal-fuelled power stations, which can be switched on or off quickly. But getting consumers to change their habits a little is potentially much cheaper.

The prospects in Europe have brought a flurry of acquisitions. The world's biggest demand-response firm, EnerNOC, a publicly traded American company, has bought a dozen foreign providers since 2005. Nearly 20 percent of its $383 million in revenues come from abroad. In February it bought Entelios, the biggest German company, with 600MW of capacity, and Activation Energy, the leading Irish firm.

The industry has plenty of room to grow. In America it accounts for more than 20 gigawatts (GW), or 2 percent of the total installed capacity, according to Colin McKerracher, an analyst at Bloomberg New Energy Finance (BNEF). It is concentrated in the northeast part of the country. In the European Union, a much larger energy market, capacity is only 5.4GW, but BNEF forecasts it will grow to 15.3GW by 2020. Pulse Energy, a Canadian firm, expects to double its customer base to 2 million by Christmas.

Demand response still faces skepticism. People who run and regulate the network like power stations they can see, not virtual ones.

"Is that really the sort of country we want to be — where you just cut the power off when supply gets tight?" asks an executive at one of Britain's "big six" energy providers (which want easier rules on building new power stations). David Brewster of EnerNOC says changing "100 years of policy and mind-set" is a "slow grind."

Some parts of America already have well-developed markets in capacity, where demand-response providers can bid alongside conventional power producers for supply contracts, typically three years in advance. Their cost advantage makes them increasingly competitive.

Such markets are starting in Europe, too, but rules are still being written. Providers worry that regulators will tilt them in favor of conventional producers on such issues as response time (how quickly power must be switched on and off) and how much aggregation (bundling of small slices of consumption) will be allowed.

To be credible, demand- response must be reliable. Grid managers want to know the power will be there when needed. That may favor the bigger demand-response companies — and the utilities, which could dominate the industry if they chose to enter it. Upstarts claim they have smarter algorithms. They say they are creating prime electricity out of subprime portfolios. That analogy will reassure neither customers nor rule makers.

Copyright 2013 The Economist Newspaper Limited, London. All Rights Reserved. Reprinted with permission.