A long period of outperformance of shares with lower volatility has helped to give rise in recent years to what amounts to a new investment style, low volatility, with more than $15 billion flowing into this type of mutual fund and exchange-traded fund (ETF) in the first seven months of the year, according to Morningstar data. They now account for 13 percent of U.S. ETF assets under management.

A recent period of lagging returns, both in the U.S. and Europe, may be a signal that this run is in the early stages of an extended return to mean. While the long-term performance data of low-volatility vs. high-volatility stocks still looks good, some of the underlying drivers, particularly interest rate movements, may be headed for either a cyclical or secular reversal.

If, as appears likely, the so-called low-volatility effect was driven in large part by falling inflation and momentum flows from trend-following investors, then a potentially vicious unwind may be at hand.

Low inflation's role

Falling interest rates do appear to help the performance of stocks, such as the staples sector, which fall into the low-volatility category. Yet new demand from all of the low-volatility funds and investors has, as you would expect, driven an expansion of the market price for a given dollar of earnings by a low-volatility company.

"Our analysis tilts us towards the bears. Multiple re-rating has indeed driven outperformance," analysts from the Barclays European Equity Strategy team, led by Dennis Jose, write in a note to clients.

"Furthermore, we find that the dominant driver of the re-rating of low-vol stocks was a deceleration in global inflation. However, with inflation having bottomed out, low-vol stocks could hereon underperform, in our view."

Low-volatility funds are a subset of smart beta funds, a set of techniques which tries to improve on index investing by adjusting away from the typical cap-weighted style to take advantage of supposed factors which can drive outperformance.

That low-volatility stocks can outperform higher-volatility stocks is, on the face of it, a puzzle, given that investors usually find high volatility unattractive, implying that the market should offer better returns in compensation.

Underperforming low-vol?

One thing is uncontroversial: the period during which low-volatility shares have outperformed has been coincident with a period of falling and then very low inflation and interest rates. Since January 2002, U.S. low-volatility stocks have outperformed by one percentage point annually, while their European peers have beat the index by three points a year.

An academic study from Holland, released this month, finds that low-volatility stocks are geared to falling interest rates, and that interest rate movements can explain up to 80 percent of the otherwise mysterious outperformance.

"We find that portfolios consisting of stocks with low idiosyncratic volatility are exposed to more interest rate risk than portfolios with stocks that have higher volatility. Our results imply a strong implicit exposure to bonds of low-volatility portfolios, increasing the returns," Joost Driessen and Ivo Kuiper of Tilburg University and independent researcher Robbert Beilo write.

In other words, if you wanted to make the damage that rising inflation and interest rates would do to your mixed portfolio via bonds that much worse, low volatility is the strategy for you.

Pricey shares

Of course, while long bonds have crept higher in recent weeks there is no assurance that we will see rising inflation or rates. Indeed, it is possible that a period of extended secular stagnation is at hand.

Headline inflation in many developed economies, notably the U.S., has risen off its lowest levels. Core inflation in the U.S. is now at 2.3 percent, helped in large part by chunky wage inflation of 3.3 percent. If this short trend is extended, we might expect to see low-volatility shares lose their attractions and underperform.

And low-volatility shares are not only expensive by some measures, their performance has been driven by a re-rating by investors of the value of a given dollar of their earnings. Looking just at European stocks, Barclays finds that, unlike earnings growth and dividends, price/equity multiple expansion for low-volatility shares has been much greater than for the broad market. Data in the U.S. tell a similar story.

So what, do you suppose, will happen to low-volatility inflows during a period of underperformance? Inflows may well turn to outflows, investors being what they are.

And what will happen to valuations if low-volatility funds rather than being big buyers turn sellers?

Rinse and repeat, as they say on the back of the shampoo tubes.

James Saft is a Reuters columnist.