Using Market Regression to Improve Prediction Accuracy in the NFL

Read this post on our new app, nfeloapp.com

Summary

• Through price discovery, markets produce incredibly accurate predictions about the future

• Even the best models can benefit from regressing their predictions to those of the market

• Scaling the degree of regression with signals that reliably identify situations where the market prediction might be relatively more or less accurate improve predictions even further

• Using these methods, models like nfelo can predict NFL margins better than the market by itself

Markets & Price Discovery

When a market prices an asset below its worth, buyers who know it is mispriced are incentivized to purchase it from sellers who do not. If enough buyers believe an asset is mispriced, sellers may adjust the price higher, either as a response to this market signal or due to classic supply/demand dynamics.

As the price of the asset increases, fewer buyers will believe it to be underpriced and more sellers will believe it to be overpriced. As these forces converge, the asset’s price will stabilize at a point that represents the collective beliefs (ie wisdom) of all market participants. As many well informed and properly incentivized participants place bets on the asset, the market price of the asset will begin to reflect its true worth.

This powerful process is called price discovery and it is the fundamental mechanism through which markets create remarkably accurate predictions of the future.

Models

Models represent another avenue through which the true price of an asset can be determined. Assets have characteristics and historical signals that can be used to project future behavior. A discounted cash flow, for instance, uses a businesses’ existing cash flow characteristics and assumptions about how those cash flows will change in the future to price the business based on the present value of all future profits.

In a sense, models exist in a vacuum from the market. Instead of relying on the collective wisdom of others, models produce an independent assessment of an asset’s value. Though individually models exist in a vacuum, they are the force that collectively produce the market signal. Markets reflect the beliefs of many independent models all assessing the value of an asset in slightly different ways.

Improving Models with Market Signals

No model is perfect. Even the best model is likely to miss dynamics captured by others, and as a result, the best model is unlikely to produce the most accurate price under all conditions.

Because the market represents the collective output of all models, any individual model can stand to gain by incorporating the market’s price as an additional signal. The process of incorporating the market as a signal is often called market regression.

However, market regression presents its own challenges that themselves must be modeled. How much should a model regress to the market? Should a model always regress by the same amount?

Regressing nfelo

In the context of football spreads, market regression takes the projected spread of a model and combines it with the market’s spread. The simplest way to regress a model is to take a weighted average of the model’s spread and the market’s spread.

The best weights to use in this process are those that minimize the prediction error of the regressed model. In this analysis, prediction error is measured by a Brier score where higher values represent more accurate predictions:

By itself, the market’s price is more accurate than the model’s, but when the model and market are combined, the resulting regressed model is more accurate than either in isolation. The optimal weighting for the regression is the weight that produces the high point on the graph above (65%).

The fact that a regressed model is more accurate than the market means that the model itself contains a valuable signal not fully captured by the market. A model that achieves its greatest accuracy when the market’s price is weighted 100% contains no signal. Generally speaking, the more accurate a model is relative to the market, the less it should be regressed.

If past accuracy is predictive of future accuracy, then the amount the model should be regressed for a given game should scale based on the past accuracy of the model relative to the market. Said another way, if the model’s trailing average error for a team over a given window is lower than that of the market’s for that team, the model’s prediction for the team’s next game should be regressed less.

Here, the accuracy of the model relative to the market for a given game can be defined as the average error of the model for the home and away teams minus the average error of the market for the same home and away teams:

With a team’s error defined as the exponential average of squared errors over its last N games:

The relative accuracy of the model can be translated into a scaling factor for the regression weight by dividing by an additional parameter and constraining the result to a number between 0 and 1:

The additional parameters of “N games” and “Base” can be optimized to reduce the error of the error weighted regressed line. This method does yield additional improvements over a flat value for market regression:

The error weighted market regressed line yields a very powerful model that beats the market line by itself:

An important point to note here is that Brier scores incorporate the prediction’s margin of error. When the prediction misses the actual margin by a greater degree, the error is larger. Conversely, performance against the spread (ATS) is a binary measure--a model was either more accurate than the market or it wasn’t. As a result, a model with a Brier score below the market can still have a positive grade against the spread if its predicted margins have more volatility than the market’s. This is the case for both the 538 model and nfelo.

Conclusion

Nfelo is a powerful model in its own right. It surpasses a very strong benchmark model (538) and only slightly underperforms the market in predicting margin. This performance alone yields positive ATS performance on higher confidence wagers.

However, as we might expect, the model is made even more accurate when regressed to the market. Because nfelo possess predictive power, the regressed model outperforms the market on a squared error basis (Brier). 

Furthermore, the regression framework outlined above creates a basis for future improvement. The error weighted regression yields additional improvements because relative error is a reliable signal for identifying situations where the market might possess either more or less predictive power than average. Other signals that reliably identify such situations could, in theory, be used to further improve regression logic. For instance, does the model perform relatively better in primetime games? Does the market perform relatively better when a team is suffering from significant injuries? Does the market perform relatively better when a new coaching staff and play calling scheme is put into place? Etc. etc.

A market regression framework creates yet another means through which to introduce marginal improvements to predictive power. When stacked and combined, these marginal improvements are what make nfelo one of the best publicly available models for predicting NFL games.