Professional Documents
Culture Documents
Evidence from
Pace-Adjusted Measures*
Joseph Engelberg
Grant Goodstein
Abstract: Professional sports teams are often faced with the daunting task of evaluating amateur
talent for the purposes of a draft. How can they best use available data, say of a player in college, to
forecast how that player will perform in the pros? Using data on all NBA draft picks of college athletes
between 2003 and 2012, we find that NBA teams could improve their draft picks by incorporating
pace-adjusted metrics in their selection decisions. Specifically, in linear regressions we find
differential predictability for an NBA players performance. When we use measures of his collegiate
productivity without pace-adjustment, points produced per game, we find no predictability for win
shares or player efficiency rating (PER) after controlling for the players draft number. However,
when we use pace-adjusted measures of collegiate performance, such as points produced per
possession, we find significant predictability for win shares and player efficiency after controlling for a
players pick number. A one standard deviation change in collegiate points produced per possession
(CPPP) predicts a change in win shares commensurate with 14 draft spots. That is, by increasing the
CPPP of its selection by one standard deviation an NBA team would be able to draft a player with the
26th pick that had the average performance of a 12th pick. Taken together, the results suggest NBA
draft markets are inefficient because they do not correctly incorporate all available information when
assigning picks.
* We have benefited from discussions with Christopher Parsons, Ken Pomeroy, Paul Tetlock and Jared Williams.
We thank Ken Pomeroy for his assistance with the data.
Contact: Joseph Engelberg, Associate Professor of Finance, University of California San Diego, (Email)
jengelberg@ucsd.edu, (Tel) 858-822-7912; Grant Goodstein, University of Michigan, (Email)
gmgood@umich.edu, (Tel) 760-585-6882.
I.
Introduction
overall pick in the 2008 draft and Love was the fifth, Loves NBA career has been more successful than
Beasleys by most measures. For example, Loves career win-shares per 48 minutes are .172 compared
to Beasleys .044, and Loves career player efficiency rating is 22.1 compared to Beasleys 15.
The anecdote generalizes to our sample of 440 collegiate players drafted into the NBA between
2003 and 2012 when we compare the differential predictability of collegiate points produced per
game (CPPG) and collegiate points produced per possession (CPPP). We find that CPPG is a better
predictor of draft number than CPPP (as in the previous example), but it does not predict various
measures of NBA performance after controlling for draft number. CPPP, on the other hand, reliably
predicts various measures of NBA performance after controlling for draft number. A one standard
deviation increase in CPPP is commensurate with an improvement of 14 draft spots. That is, by
increasing the CPPP of its selection by one standard deviation an NBA team, for example, would be
able to draft a player with the 26th pick that had the average performance of a 12th pick.
Our paper belongs to a small, but growing, literature which has examined the efficiency of
professional drafts (e.g., Massey and Thaler (2010), Coates and Oguntimein (2010), Berri, Brook and
Fenn (2011), Spurr (2000) and Burger and Walters (2009)). To the best of our knowledge, this paper
is the first to document a fixation on per game measures of collegiate productivity and find evidence
that draft picks could be improved by incorporating per possession measures of collegiate
productivity.
II.
Data
III.
Analysis
We begin with the following question: can collegiate measures of productivity predict NBA
performance after accounting for a players draft number? Or, said another way, is there valuable
information in measures of college productivity that is not fully captured by a players draft number?
Figure 1 illustrates our main result. We first sort each collegiate player into one of four draft
groups: players selected with one of the first 15 picks are in the top group and players selected with
one of the last 15 picks are in the bottom group. Then, within each draft group we further sort into
two groups based on measures of collegiate productivity. In Panel A the measure of productivity is
points produced per game and in Panel B the measure of productivity is points produced per
possession.
From Panel A, a few trends emerge. First, it is clear that draft number is strongly associated
with NBA performance: those drafted in the top group have an average WS48 (0.086) which is more
than double those drafted in the bottom group (0.035). The steady rise of the bars in the graph
indicate the relationship is monotonic across draft groups . Moreover, within each draft group,
collegiate points produced per game have little predictability for win shares. For example, those in the
top draft group with above-median CPPG have an average WS48 of 0.084 compared with 0.089 for
those with below-median CPPG (t-stat -0.59). Likewise, those in the bottom draft group with abovemedian CPPG have an average WP48 of 0.036 compared with 0.034 for those with below-median
CPPG (t-stat 0.23).
From Panel B, however, we find strong predictability for WS48 within draft group when we
sort on collegiate productivity per possession: in every draft group the above-median CPPP group
outperforms the below-median CPPP group. For example, considering the top 15 draft picks, above
median CPPP players had an average WS48 of 0.095 compared to 0.0761 for below-average CPPP
players (t-stat 2.15); for draft picks 16 30 above median CPPP players had an average had an average
WS48 of 0.085 compared to 0.056 for below-average CPPP players (t-stat 3.51); and for draft picks 31
45 above median CPPP players had an average had an average WS48 of 0.071 compared to 0.047 for
below-average CPPP players (t-stat 2.51). Only in the bottom group is the difference between abovemedian and below-median CPPP players insignificant (0.036 vs. 0.034 with a t-stat of 0.18).
In Table 1, we formalize the sorting exercise in a regression framework. In the first two
columns, we ask how CPPG and CPPP predict a players draft number. Formally we estimate the
following two linear regression models:
(1)
(2)
where Draft Number is a players pick number in the NBA draft (1 through 60), CPPG is
collegiate points produced per game, CPPP is collegiate points produced per possession and Controls
consist of draft year fixed effects and position fixed effects. CPPG and CPPP are standardized so we
can interpret the units in terms of standard deviations. The coefficient estimates in columns 1 and 2
suggest teams pay more attention to points produced per game when making their draft selections. A
4
one standard deviation increase in CPPG corresponds to a player being selected 4.2 picks earlier.
However, a one standard deviation increase in CPPP corresponds to a player being selected 2.4 picks
earlier. Because the dependent variable is a count, we also run count regressions (poisson and
negative binomial) in an appendix table and find no substantive change in the results: NBA teams
appear to be more sensitive to points produced per game rather than per possession when selecting
players in the NBA draft.
The remaining columns of Table 1 suggest that this is a mistake. The columns present the
coefficients from several specifications of the following form:
(3)
(4)
where Performance Measure is win shares per 48 minutes (WS48) in columns 3 and 4, win
shares in columns 5 and 6 and player efficiency rating (PER) in columns 7 and 8.1 Controls include
draft year fixed effects, position fixed effects and draft number fixed effects (i.e., 59 separate dummy
variables for each pick number). This last control is worth emphasizing because if NBA draft markets
were efficient we should not be able to find any variable which predicts NBA performance after
controlling for draft number. And yet thats precisely what we find: CPPP reliably predicts each of our
measures of NBA performance after controlling for draft number. For example, column 4 suggests
that a one standard deviation increase in CPPP corresponds to an increase in WS48 of .0131 (t-stat
3.52). To appreciate the magnitude of this result, in an appendix table we consider the same
specification with a linear control for draft number (rather than draft number fixed effects). That
table indicates that moving up the draft one pick improves WS48 by 0.0009 so that a one standard
deviation increase in CPPP is akin to an improvement of .0131 / .0009 14 draft spots. In other
words, by increasing the CPPP of its selection by one standard deviation an NBA team would be able
to draft a player with the 26th pick that had the average performance of a 12th pick. Columns 3, 5 and
7 indicate the predictability for NBA performance is only found with our pace-adjusted metric, CPPP.
When we examine collegiate points produced per game, we find no predictability after controlling for
draft number. Thus, whatever value-relevant information exists in CPPG is subsumed by the draft
number controls.
IV.
Conclusion
Most reports of player productivity come in the form of per game averages (e.g., points per
game or rebounds per game). Such reports can mask crucial differences in pace and possessions
across players and teams. We find evidence that NBA teams fail to fully appreciate these differences
when making their draft choices. The draft picks of NBA teams could be substantially improved by
incorporating per-possession -- rather than per-game -- measures of collegiate performance into their
selection decisions.
1 The number of observations falls to 397 in columns 3 8 because 43 of our 440 NBA draft picks did not play in an
NBA game. In an appendix table we show our results do not change if instead we assign a value of zero (rather than
missing) for these 43 observations.
PANEL A
NBAWinShares(per48minutes)
0.1
0.09
Points
Producedper
Gamein
College(Low)
0.08
0.07
0.06
0.05
Points
Producedper
Gamein
College
(High)
0.04
0.03
0.02
0.01
0
Picks4660
Picks3145
Picks1630
Picks115
PANEL B
NBAWinShares(per48minutes)
0.1
0.09
0.08
Points
Producedper
Possesionin
College(Low)
0.07
0.06
0.05
0.04
Points
Producedper
Possesionin
College(High)
0.03
0.02
0.01
0
Picks4660
Picks3145
Picks1630
Picks115
DependentVariable:
CollegiatePoints
ProducedPerGame
NBADraft
Number
NBADraft
Number
4.197***
NBAWin
Sharesper
48minutes
NBAWin
Sharesper
48minutes
0.0008
NBAWin
Shares
0.9191
NBAPlayer
Efficiency
Rating
NBAPlayer
Efficiency
Rating
0.3620
(0.8790)
(0.0041)
(0.7650)
(0.2519)
CollegiatePoints
2.4084***
0.0131***
2.0538***
0.7055***
ProducedPerPossession
(0.8502)
(0.0037)
(0.5753)
(0.2230)
PositionFixedEffects
YES
YES
YES
YES
YES
YES
YES
YES
DraftYearFixedEffects
DraftNumberFixedEffects
YES
YES
YES
YES
YES
YES
YES
YES
NO
NO
YES
YES
YES
YES
YES
YES
Observations
440
440
397
397
397
397
397
397
0.0422
0.0095
0.1077
0.1431
0.3435
0.3571
0.2486
0.2655
AdjustedR
NBAWin
Shares
DependentVariable:
CollegiatePoints
ProducedPerGame
CollegiatePoints
ProducedPerPossession
DraftNumber
PositionFixedEffects
DraftYearFixedEffects
DraftNumberFixed
Effects
Observations
AdjustedR2
NBADraft
Number
NBAWin
Sharesper
48minutes
NBAWin
Sharesper
48minutes
NBAWin
Sharesper
48minutes
NBAWin
Sharesper
48minutes
YES
YES
0.0010
(0.0038)
0.0010***
(0.0002)
YES
YES
YES
YES
0.0009
(0.0028)
YES
YES
NO
NO
YES
YES
NO
NO
440
440
440
0.1725
440
0.2180
397
0.1198
397
0.1519
NBADraft
Number
NBADraft
Number
NBADraft
Number
YES
YES
0.1596***
(0.0394)
YES
YES
NO
NO
440
440
0.1427***
(0.0316)
YES
YES
0.0841***
(0.0308)
0.0821**
(0.0345)
0.0114***
(0.0026)
0.0125***
(0.0033)
0.0009***
(0.0002)
YES
YES
References
Berri, D., Brook, S., and Fenn, A., 2011, From College to the Pros: Predicting the NBA Amateur
Player Draft, Journal of Productivity Analysis, 35, 2535.
Burger J., and Walters, S., 2009, Uncertain Prospects: Rates of Return in the Baseball Draft,
Journal of Sports Economics, 10(5), 485501.
Coates D., and Oguntimein, B., 2010, The Length and Success of NBA Careers: Does College
Production Predict Professional Outcomes? International Journal of Sport Finance, 5(1),
426.
Hollinger, J., 2003, Pro Basketball Prospectus: 200304, Brasseys Inc, Washington, DC.
James, B., and Henzler, J., 2002, Win Shares. STATS, Morton Grove, IL.
Kubatko, Justin. "Calculating Win Shares Basketball-Reference.com - Basketball Statistics and
History. http://www.basketball-reference.com/.
Massey C., and Thaler, R., 2010, The Losers Curse: Overconfidence vs. Market Efficiency in the
National Football League Draft, Working Paper, University of Chicago.
Spurr,
S.,
2000,
The
Baseball
Draft:
Study
of
the
Ability
to
Find
Talent,