In the fall of 2008, James Weatherall, today a professor of logic and the philosophy of science at the University of California, Irvine, was a PhD student in physics in New Jersey and lived across the river from Wall Street.

When the financial crisis hit, he saw how the media was placing the blame on mathematical modelling, saying physicists, mathematicians, statisticians and computer scientists had completely changed the industry, introducing systemic risks no one could see because they used models no one could understand.

“But I was having a lot of trouble finding anyone who could clearly explain what the models were doing,” he says. “Why, if they were so dangerous, were they so widely used in the first place? In what ways had they failed?”

With his outsider perspective and background in physics and math, Weatherall decided to look into this, writing a book called The Physics of Wall Street: A Brief History of Predicting the Unpredictable.

“The book looks at the history of ideas, moving from physics and mathematics into financial modelling, and looks at examples where the models were used effectively and examples where the models were used ineffectively, or in ways that created risk or volatility or losses,” he says.

There are differences in how someone understands models and how they can be used safely and successfully, adds Weatherall. “And the key idea is that you need to understand that any kind of mathematical modelling in finance or in economics relies on background assumptions that are strictly speaking false, but often approximately true. And the key is to really understand what you’re assuming when you use a model.”

Throughout history, many financial innovations have been positive, says Weatherall. “Things like algorithmic trading and even high-frequency trading, generally speaking, have been beneficial because they create liquidity, they make it easier to make markets, especially in equities; they facilitate trading.”

But there are also examples demonstrating how this innovation has been tied to greater volatility, he notes, citing the quant crisis in 2007. “And so that was a case where, because of no change in market fundamentals, you suddenly saw a lot of volatility. Most of the people who were seriously hurt by that were the quant funds themselves, but it was pretty mystifying for everyone else in the market. You couldn’t figure out why markets were moving the way they were.”

Another example is the financial crisis, which was due to a massive change in collateralized debt obligations and that more complicated practices based on these were used in banking, says Weatherall.

“Now, if you look at these two examples, they’re totally different from one another, even though in both cases you have market volatility or market drawdowns that are due in some sense to financial innovation. And I think that keeping an eye on the difference is important.”

In the first case, people who were at the cutting edge of the new practices suffered most, which makes sense because there’s a risk in being on the leading edge, Weatherall notes.

“The second thing, I think, is where the real danger lies, and that’s reflected in the fact that the ultimate costs to the rest of the market were much higher. And the real danger was that a change in market practices, a kind of innovation, led to the widespread adoption of an assumption that was really opaque to most of the people that were implicitly making it. I don’t think that most people who were rating or trading CDOs were really thinking about how correlations in default rates across different parts of the U.S. or different parts of the world were going to affect their portfolios. And that’s the real problem and that’s the thing I think we really need to be vigilant against.”

While fund managers should think about their own adoptions of new technology, they also need to pay attention to what everyone else is doing and which large-scale trends are creating new kinds of systemic risk which the rest of the market may not be aware, says Weatherall.

Speaking at the Canadian Investment Review’s Investment Innovation Conference on Nov. 22 in Miami, Weatherall will expand on the risks, associated opportunities and issues that come along with financial innovation.

“Financial innovation changes markets,” he says. “Changing markets creates systemic risk, but there’s a way of thinking where you can see how that systemic risk actually has upsides as well, if you’re sensitive to the ways in which it’s going to change markets.”