I analyzed all of the various mock drafts I was tracking in my compilation (see https://jazzfanz.com/showthread.php...-mock-draft-compilation-v-2-0-(final-version) ) to see which ones were best. Note that I know a few of them changed a bit after my final rankings, but I didn't go back and update my lists to reflect their truly last versions.
Due to my methodology (see below) I analyzed first round only mocks separately from ones which had both rounds. In the "both rounds" list I also included my "instant runoff" rankings which I've been posting, along with a ranking that gave each player the median of all the mocks and one that gave each player the mean.
Here are the results (see my compilation thread for links to the actual mocks):
Mocks which included both rounds
1 and 2 (tied) DraftExpress.com--this was far and away the best of all of the published mocks--and median (I was surprised this did better than instant runoff, and was surprised this was the equal of draftexpress)
3 mean
4 instant runoff
5 DraftTek
6 The Ringer (Kevin O'Connor)
7 and 8 (tied) NBADraft.net and Walter Football
9 NBA.com (Scott Howard-Cooper)
10 mynbadraft.com
11 nbadraftroom.com
12 netscoutsbasketball.com
13 basketballinsiders.com -- this was far and away the worst of all the mocks. Horrible, truly horrible.
Mocks which included first round only
1 FoxSports (Andrew Lynch)
2 CBS Sports (Gary Parrish)
3 SI.com (Andrew Sharp)
4 and 5 (tied) SportingNews.com (Sean Deveney) and USA Today (Adi Joseph)
6 CBS Sports (Reid Forgrave)
-----
Analysis details for those who care:
To determine which ones were the best I decided to do things a little more sophisticatedly than just looking at where a player was actually chosen compared to where the mock draft predicted. Instead I decided to think of it like this: for a given draft position, how far down the list of players remaining in the mock draft was the player who was actually selected? This was the "error" for that position. (For players not predicted to be drafted, I lumped them into a 61st or a 31st position for both round mocks and for first round only mocks, respectively.) Then I added up all the errors for all draft positions to obtain an overall error for each mock. The mocks were then ranked by lowest error.
Because I knew that how I defined the error would probably affect things, I also tried three other error definitions. The second method was to use a "squared error", i.e. the error for a given player was taken as the square of the remaining players ranked above him by the mock. Missing a player by 2 positions gave 4x as much error as missing by 1 position. The third method was to give increased error to players at the top of the draft. I did this via a 1/n normalization function, i.e. I divided each error by the player's position, so the top pick by one player would have given 30x the error of missing on the 30th pick by one player. Finally, my fourth method was to use the 1/n normalization function combined with squared error.
Then I looked at how all the mock drafts did in terms of those four different error functions (rankings, as well as numerical error values) to draw my conclusions.
Due to my methodology (see below) I analyzed first round only mocks separately from ones which had both rounds. In the "both rounds" list I also included my "instant runoff" rankings which I've been posting, along with a ranking that gave each player the median of all the mocks and one that gave each player the mean.
Here are the results (see my compilation thread for links to the actual mocks):
Mocks which included both rounds
1 and 2 (tied) DraftExpress.com--this was far and away the best of all of the published mocks--and median (I was surprised this did better than instant runoff, and was surprised this was the equal of draftexpress)
3 mean
4 instant runoff
5 DraftTek
6 The Ringer (Kevin O'Connor)
7 and 8 (tied) NBADraft.net and Walter Football
9 NBA.com (Scott Howard-Cooper)
10 mynbadraft.com
11 nbadraftroom.com
12 netscoutsbasketball.com
13 basketballinsiders.com -- this was far and away the worst of all the mocks. Horrible, truly horrible.
Mocks which included first round only
1 FoxSports (Andrew Lynch)
2 CBS Sports (Gary Parrish)
3 SI.com (Andrew Sharp)
4 and 5 (tied) SportingNews.com (Sean Deveney) and USA Today (Adi Joseph)
6 CBS Sports (Reid Forgrave)
-----
Analysis details for those who care:
To determine which ones were the best I decided to do things a little more sophisticatedly than just looking at where a player was actually chosen compared to where the mock draft predicted. Instead I decided to think of it like this: for a given draft position, how far down the list of players remaining in the mock draft was the player who was actually selected? This was the "error" for that position. (For players not predicted to be drafted, I lumped them into a 61st or a 31st position for both round mocks and for first round only mocks, respectively.) Then I added up all the errors for all draft positions to obtain an overall error for each mock. The mocks were then ranked by lowest error.
Because I knew that how I defined the error would probably affect things, I also tried three other error definitions. The second method was to use a "squared error", i.e. the error for a given player was taken as the square of the remaining players ranked above him by the mock. Missing a player by 2 positions gave 4x as much error as missing by 1 position. The third method was to give increased error to players at the top of the draft. I did this via a 1/n normalization function, i.e. I divided each error by the player's position, so the top pick by one player would have given 30x the error of missing on the 30th pick by one player. Finally, my fourth method was to use the 1/n normalization function combined with squared error.
Then I looked at how all the mock drafts did in terms of those four different error functions (rankings, as well as numerical error values) to draw my conclusions.
