How large is your sample size?
The initial sample size used to run the regression analysis was approximately 1,500 “contract seasons.” Each contract season is a single input record. So if a given player’s contract covered five seasons from 2005-2009, this resulted in five different contract seasons for the purpose of creating input records (even if the player was released after three seasons).
Each year, we will be able to increase the sample size based on the decisions that teams make with respect to actual contracts. For example, by April we will be able to update the database with all of the 2015 contract seasons for players who were released during February and March (and therefore register a “No” with respect to the binary outcome used in the regression analysis). In September, we can update the database again with more players who were released at the conclusion of training camp, as well as all of the players who remained under contract through the offseason and into the regular season (who therefore register a “Yes” with respect to the binary outcome).
We only used contract seasons from multi-season contracts, we did not include rookie contracts, and we disregarded contract seasons where the details were not ascertainable to a reasonable degree. Because the goal was to create a model that simulates the decision- making process that front office executives would go through, we wanted to create a sample consisting of “pure” decision-making scenarios. In our view, the purest decision-making process occurs within the context of multi-season veteran contracts. Once we generate a model based on those results, we can generalize to all contracts.
Are you confident that these are the correct inputs? What if you realize afterward that you forgot something important or included something that isn’t important?
No, these are not necessarily the “correct” inputs. It is very possible that we will come up with an idea for a new input variable that will improve the accuracy of the model. Doing so may also lead us to remove or change input variables that are currently in the model. Furthermore, as more input records are entered into the sample size (see previous question), the model will naturally change incrementally over time. This may be particularly true as contracts signed under the new CBA begin to represent a greater portion of the sample; it is possible that teams have designed contracts differently under the new CBA and have adjusted their decision-making processes accordingly.
The most important point of this metric is to change how people think about contracts. We want to convey the message that the characteristics of a contract determine (to a significant degree) the future decisions that a team will make with respect to the contract, and that the decision-making probabilities can be quantified as the characteristics of the contract change. The specific quantification model may (and almost certainly will) change over time, but we think that the approach we have developed will prove to be useful and influential over a long period of time.
Wouldn’t this analysis be even further enhanced by factoring in the time value of money?
In principle, yes, we agree that the time value of money is important and should always be considered. We would hope that agents take this into consideration when advising clients as to which contracts to accept, and we assume that teams at times present this consideration as an argument as to why a given contract proposal may be stronger than that of another team. However, we have not included time value of money calculations into Expected Contract Value for two reasons.
First, we are presenting Expected Contract Value as an alternative way of viewing traditional contract valuation. NFL contracts have traditionally been reported and analyzed without any concern given to the time value of money. We have never seen Adam Schefter tweet the net present value of a contract. So in order to make an apples to apples comparison, we only want to make one change to the way in which contracts are thought about, rather than two changes. If a contract is reported as being for 5 years and $25 million, we want to be able to use Expected Contract Value to make a direct comparison to that $25 million face value. If we also layer on time value of money considerations, the comparison will no longer be direct.
Second, even if we wanted to factor in the time value of money, we wouldn’t know what discount rate to use. Remember, there are two different contexts in which we are applying Expected Contract Value: actual money to be earned by the player, and salary cap space to be allocated by the team. With respect to actual money to be earned by the player, we suppose we could just use a generic discount rate based off of the Treasury Rate, or the average rate of return in the stock market, or something else of that nature. However, these choices all seem pretty arbitrary, and we don’t see any basis by which to calculate weighted average cost of capital as would be used in a DCF type of time value of money calculation.
With respect to salary cap space to be allocated by the team, we also do not know what discount rate we would use. [Specifically Bryce speaking] However, I think developing this could be possible. In fact, I think that developing a useful cap space discount rate is one of the most interesting areas for future research.
Does the cap increase have any influence on contract decisions in your model?
No. The rate of increase in the salary cap is closely related to the amount of cap space that a given team has in a given year, and falls into the category of variables that cannot be projected at the time a contract is signed (along with player performance, player health, team needs, etc.). Expected Contract Value therefore ignores this variable.
With respect to “Expected APY,” why are you using the total number of years as the denominator, rather than the “expected” number of years? Aren’t you mixing units (using an “expected” number in the numerator, but an “actual” number in the denominator”)?
Well first, we acknowledge that the name “Expected APY” is probably misleading relative to what we intended it to mean. The name implies that we are calculating the APY if the contract proceeds as would be “expected” according to Expected Contract Value (player is released once the expected outcome falls below 50%, etc.). What we should have called it is “Average ECV.” This name would imply that we are just taking the Expected Contract Value and then averaging it across the years of the contract. As we described in Part 2 of the introduction, this would solve the over-inclusiveness issue of regular APY, because Average ECV takes into account the relative unlikeliness of the money in later years of the contract actually being earned.
Having said that, we think it could make sense to change the denominator, although not in the way that we interpret this question as suggesting. Let’s assume that a player signs five year contract, and that the expected outcomes for the five contract seasons are as follows: 90%, 80%, 60%, 30%, and 10%. In our description of Expected APY, we indicated that the denominator for a contract with these expected outcomes would be 5, due to there being five contract seasons. I think that this question is suggesting that the denominator be 3, because the expected is that the player will be released after the third season of the contract. However, in order to perfectly match units, the denominator should probably be 2.7 (.9 + .8 + .6 + .3 + .1). The fifth season is “expected” to be 1/10 of a season, rather than no season at all. This gets back to the point illustrated in our discussion of Weighted Average Cap Budgeting in Part 4: conceptually it is necessary to get comfortable with the uncertainty that is presented at the moment the contract is signed.
However, changing the denominator in this fashion seems to remove the apples to apples nature of the comparison to traditional APY, so we aren’t sure if this would be a useful metric to use. Ultimately, this doesn’t really matter. Once the Expected Contract Formula model generates the expected outcomes, the user of the data can create any kind of metric deemed desirable. The primary consideration should be whether or not the metric is useful for the intended purpose.
If the player is released once his expected outcome falls below 50%, won’t that mean that he will earn less than his Expected Contract Value suggests that he will? How do you reconcile this?
This is somewhat related to the last question. Yes, of course there will be many instances where the player will earn less than his Expected Contract Value. On the other hand, there will be many instances where the player will earn more than his Expected Contract Value. In fact, we would guess that very rarely will any player ever earn an amount that comes extremely close to matching the Expected Contract Value. But the point is to present an average expectation based on all of the potential scenarios, weighted to the appropriate degree. It is as if we are imagining that 100 parallel universes are set up; in each of them the player makes it through various numbers of years of the contract, and then we average all of these outcomes together.
More generally though, this gets at the issue of what people find most useful about this metric. Some people may find it more useful to just identify the point at which expected outcome falls below 50%, and then base all of the numbers off of the assumption that the player will be released in that season. Other people may find it more useful to treat a 30% expected outcome as a 30% expected outcome, rather than as an assumption of 0%. Again, the data can be used in whatever way is most helpful to the user’s intended purpose. We will keep in mind the differences of opinion on this topic and attempt to present the data both ways going forward.
Are you sure that a player’s position doesn’t matter? What about kickers/punters and quarterbacks?
Generally, we hold the view that a player’s position does not matter for the purposes of the Expected Contract Value model. However, with respect to kickers and punters, the results may become skewed in the case of extreme ages. For other positions, a player in his age 37 season would be an extreme outlier and generally very unlikely to be kept under contract. But in the case of kickers and punters, this really isn’t an extreme age. So for kickers and punters within the range that other positions generally occupy, the results should largely be the same as for all other positions. But for kickers and punters with extreme ages, the expected outcome will systematically be lower than common sense may dictate.
A reader might respond to the above paragraph by pointing out that aging curves may be different for other positions as well. For example, the reader might assert, running backs tend to decline at a younger age than other positions. However, this isn’t really an issue of position; this is an issue of workload and usage. The running back position might cause more workload per play than the safety position, but on the whole workload is workload. So comparing the positions would involve counting the number of snaps a player plays and multiplying it based on the amount of workload per play to determine how quickly a player might break down. However, this involves projecting playing time, which would fall into the category of variables that Expected Contract Value ignores, along with performance and health. Furthermore, there probably isn’t a significant difference in workload between offensive lineman, defensive lineman, linebackers, etc.
With respect to quarterbacks, we acknowledge that the decision-making process used to evaluate starting quarterbacks is likely different than that used for all other positions. This is a product of positional scarcity. There are fewer than 32 reasonably productive quarterbacks in the world, so a team who has one under contract – even if that quarterback is not “#ELITE” – may be more inclined to keep the quarterback under contract, regardless of the characteristics of the contract. However, we think that Expected Contract Value works fine for backup quarterbacks and fringe starters.
So wait, how much is [Player X] worth? How much should [Player X] get paid? How much will [Player X] get paid?
As to the first question, we have no idea. The goal of Expected Contract Value is not to identify how much value a given player’s contract generates for a team, such as how WAR can be used to do so in baseball. The goal of Expected Contract Value is to forecast decision making. The issue of measuring player performance value is one that must be addressed by creating as close to a WAR equivalent as can be done in the context of football.
As to the second question, it depends on issues of measuring player performance discussed above. Tell us what metric you like for the purposes of measuring performance for a given position, and we could, in theory, build a model that would say how much the player “should” get paid. The accuracy of the model would depend on the usefulness of the chosen metric.
As to the third question, this is a matter of replicating decision-making, so conceptually it is the same idea as Expected Contract Value. It would be an entirely different model with entirely different variables and entirely different outputs, but we think this is very possible to do. However, it is not what we have done with Expected Contract Value.
What are the next steps with this? Who is this most helpful for?
We will use the basic Expected Contract Value results to perform further research into the two primary components of capology: (i) the efficient structuring of individual contracts and (ii) the optimum risk profile of overall team cap ledgers. We are optimistic that Expected Contract Value will serve as a useful tool to bridge the gap between the two.
For teams, Expected Contract Value can have a range of uses, depending on how in-depth they would want to go. At the very least, Either/Or Cap Budgeting can be useful for planning a few years ahead of time to get a good idea of which players will likely be released at what time, therefore impacting how much cap space will be available in which years. The next step would be using Expected Contract Value as a negotiation tool with agents, demonstrating that a “smaller” contract may deliver a higher expectation of earnings than a “larger” contract, based on the comparative structures of the contemplated contracts. In this way, teams may be able to sign players while allocating less cap space toward those contracts. There may be tradeoffs, such as dead money risk, but these are the issues we are referring to in the research mentioned in the previous paragraph.
For agents, the use seems pretty obvious. The goal of an agent should be to maximize the earnings of the client. While signing players to inflated deals may be good business from a marketing perspective, an agent who adopts Expected Contract Value as a contract measurement tool, and who can communicate the concept to clients, may be able to design contracts that will enable clients to earn more money. We suspect that there are contract structures that would be “smaller,” but that would result in a higher amount of expected earnings for the player.
For fans, Expected Contract Value serves as another way to follow the NFL and analyze the ways in which teams act. Not all NFL fans care about contracts and salary cap numbers, which is fine. But, presumably, the NFL fans who read and use Over The Cap do care about contracts and salary cap numbers. Expected Contract Value enables fans to think about contracts and salary cap numbers in a more useful and sophisticated way. Regardless of the chosen endeavor, people will (or at least should) always seek and prefer more useful information than they currently have. By introducing Expected Contract Value, we have enhanced the quality of the information available to NFL fans who care about contracts and salary cap numbers.
Expected Contract Value was created by Bryce Johnston and Nicholas Barton.
Image courtesy of bluegrassfootball.com
Georgetown Law Class of 2014
Email Bryce: firstname.lastname@example.org
Follow Bryce on Twitter: @eaglessalarycap
Georgetown University Class of 2017
Email Nick: email@example.com