Thursday, 23 July 2015

Corsi Plus-Minus: Individual Player Value Accounting for Teammates

***Reminder, if you are just interested in looking at the data/visualizations you can check out the separate page here instead of scrolling through this entire article.

Hockey stats have existed for about as long as the game itself. Simple boxscore stats such as goals and assists can be traced back almost a full century now. These stats have helped informed fans/coaches/managers of the value held by some players. Around the 1950’s the Montreal Canadiens began to track a player’s plus-minus. This stemmed from the idea that simple boxscore stats fail to capture many important elements of a game. Plus-minus was a good start to tracking impact that is not realized in traditional boxscore stats. Unfortunately, Plus-Minus has been recently shown to be quite incomplete and lacking by modern evaluation standards. 

The most famous stat to come out of the modern hockey analytics movement is probably Corsi (or just simply all shots). Corsi has numerous benefits over goal based metrics. It accumulates faster leading to more reliable information and is actually more predictive of who will be better in the future. Raw Corsi% (or CF%) tracks the share of shots (all shots, not just shots-on-goal) directed at the oppositions net versus how many are directed at a player’s own net when said player is on the ice, the higher a player’s CF% the better. This metric suffers from a key drawback however, each player’s CF% is heavily dependent upon the quality of his on-ice teammates. This is how a depth player on a good team (ex. Shawn Horcoff) typically has a higher CF% rating than a star player on a bad team (ex. Taylor Hall).

Issues with Current Metrics

Above, I have outlined some current issues with standard CF%, but these are not new issues. People have been aware of the effect playing on a good/bad team can have on a player for a while now, so next progression in the history of hockey stats resulted in Corsi%Relative. To calculate the CF%Rel of a Player Y on Team X is done as follows, Team X’s CF% while Player Y is on the ice minus Team X’s CF% while Player Y is not on the ice. This still runs into issues of comparing players on vastly different teams. Consider the example below:

Whose stat line is more impressive? It seems as though Malkin makes a pretty good team into a great team when he is on the ice while Hudler turns his team from awful to just bad. Which is more valuable? Or are they equally valuable? This is an improvement upon raw CF% but still shares many of the same problems. 

dCorsi fairly prevalent metric constructed by Stephen Burtch. dCorsi is calculated for offense as, Player X’s Actual Corsi For minus Player X’s Expected Corsi For. The real trick here is calculating what a player’s expected Corsi For is. I won’t dive too deep into this stat since it is not mine and you can read up more on it here or reach out to Stephen Burtch via Twitter here. Expected Corsi is calculated using a multivariate regression using five independent variables:
  • A dummy variable for each Team/Season
  • Time on Ice per game 
  • Team total time on ice that the player in question wasn’t on the ice
  • Offensive Zone Start%
  • Neutral Zone Start%
dCorsi moves the right direction here, especially with the dummy variable accounting for which team a player was on (if you don't know what a dummy variable is, just try to hold on until the Methodology section where I do my best to explain it). Unfortunately this doesn’t seem to help address the largest issue with CF% which is how heavily dependent an individual player’s CF% is on their teammates. 

The next evolution of CF%Rel is CF%RelTM, which stands for Corsi For % Relative to Teammates and is calculated by subtracting a player's average teammate CF% (calculated by weighting a player's teammates' CF% without him by their TOI spent with him) from his observable CF%. This sounds good but runs into issues of collinearity. Collinearity occurs in hockey data because of the way coaches use their players. 

The most famous example is probably with the Sedin twins on Vancouver. During the 2014-2015 regular season, of all 1100 minutes of 5v5 ice-time each the twins played in that season, 92% of their minutes were played with each other. This can greatly boost their teammate relative stats because their CF% is being boosted by playing with there also talented brother and is being compared to when one is instead playing with their lesser teammates. When Daniel wasn’t playing with Henrik last season his CF% was 29% but that was only about 77 minutes of ice time (or 8% of his seasonal total), unfortunately CF%RelTM will then weight this 29% heavily because of how much Daniel and Henrik play together. These rare instances where one player plays without the other can have disproportionately large effects on both player’s ratings.

This issue also occurs between players who are never on the ice together such as Jarret Stoll and Anze Kopitar on Los Angeles. Stoll played 905 minutes at 5v5 last season, how many of those were with Anze Kopitar? 1 minute and 30 seconds or 0.16% of Stoll’s total ice-time. When we look at Jarret Stoll’s most common teammates (the ones who will be weighted highest by CF%RelTM) we find that when they aren’t playing with Stoll they are playing with Kopitar. This unfairly punishes Stoll for playing the same position on the same team as one of the top centres in the game. Kopitar boosts Stoll’s teammates CF% when Stoll is on the bench while never providing the same boost to Stoll’s own CF% simply because they are never on the ice together.

This finally brings me to explain Corsi Plus-Minus, how it is calculated and why I believe it is a better metric for isolating a player’s contributions independent of the strength of that player’s teammates.

Methodology

Using the play-by-play data provided by NHL.com I was able to look at every even-strength shift that took place from 2007-2015. From this data we set up a multivariate regression where our dependent variable is rate at which a Corsi event took place, our independent variables are which players were on the ice during that shift and each shift is weighted by how long it was. This model does not account for a player's zone starts for a variety for reasons (see. here here) and it also does not account for the strength of a player's opponents (see. here). Simply, there has been a lot of research to show that both of those components might not be as relevant in helping determine a player's value. 

For those of you that might not be familiar with multivariate regressions here is a relatively simple and hopefully helpful example to help you grasp the concept. 

y=β01X12X23X3
  • y - Dependent Variable - How well you will do on your test? Measured in points
  • x - Independent Variables
    • X1 - How long did you study for? Measured in hours
    • X2 - How long did you play video games? Measured in hours
    • X3 - Did you go to the study session? Yes/No?
      • This is an example of a dummy variable 
  • B - Coefficients - The value of each independent variable 

So if you had a sheet of data filled out with how well everyone did on their test (dependent variable) along with the information on the 3 independent variables and then ran that data as a multivariate regression you would get something that looks like this (reminder I made all these numbers up): 

y = 75 + 2X- 3X+ 6X3

For the purposes of my methodology, the regressions coefficients are what we will be focusing on. Looking at coefficient β1, for every extra hour you studied (holding all other independent variables the equal) you can expect to score 2 points better on the test. Looking at the dummy variable, X3, it can either be a 1 (yes you did go to the study session) or 0 (no you did not go to the study session). If you go to the study session (holding all other independent variables the equal), you can expect to do 6 points better on your test.

Now back to how this relates to my methodology. Picture every player as a dummy variable in our data, that player was either on the ice during a shift or they weren’t. Running our regression will then give us coefficients that tell us (holding all other players equal) how much of an impact Player X has on his team’s CF%, a value I have coined Corsi Plus-Minus (CPM). Two of these regressions are run for each season, one for offence and one for defence. 

However, due to the collinearity nature of our data, as was touched upon earlier, we would be better served to use a ridge regression (also known as Tikhonov Regularization) to help account for this collinearity. This method adds a penalty factor to the regression for results being far away from the mean. This penalty factor, called lambda, is chosen based on a 10-fold cross-validation. This helps remove a lot of the noise accompanied with such a process. Players falling in the bottom 25% of the league’s playing time (measured by games played instead of TOI to not bias against forwards) are group together and treated as a single player to help reduce volatility in the results that would be caused by their extreme values. While randomness can still have an effect, the damage is less so due to this regularization. 

Repeatability and Predictiveness 

I ran some basic correlation tests to see how repetitive of a skill Corsi Plus-Minus (CPM), Offence CPM (OCPM) and Defence CPM (DCPM) is from year-to-year, players must have played at least 20 games in back-to-back years to be included in the correlation. I compared CPM to the other metrics (CF%RelTM and dCorsi) I mentioned earlier just as a means of reference. A correlation (Pearson R) rangers from -1 to 1. The closer to either -1 or 1 the stronger the relationship is, the closer to 0 the weaker the relationship.


I also ran some correlations to see how well CPM does at prediction future GF%. Reminder, negative correlations are a good thing for DCPM/dCA, so focus on the numerical values and not necessarily positive or minus signs.

  • OCPM -> GF/60
  • DCPM -> GA/60
  • CPM -> GF%




Reliability vs. Validity 

As I said above, having all 3 different metrics here is strictly for comparative purposes. Just because CF%TMRel is higher in most categories doesn't automatically qualify it as a better metric. I will summarize key parts from this Columbia article with regards to reliability and validity. 
“Reliability refers to a condition where a measurement process yields consistent scores (given an unchanged measured phenomenon) over repeat measurements.”  

This is a measure that quantifies how much random fluctuations can interfere with getting consistent results. As we have seen above, the collinearity of CPM samples decreases CPM's reliability. Compared to single-year With-Or-Without-You stats, CPM seems to be quite reliable.
“Validity refers to the extent we are measuring what we hope to measure (and what we think we are measuring).”  

CPM is completely valid, because it is directly measuring the result. Stats such as CF%, CF%Rel and CF%TMRel are not as valid, because they measure a proxy rather than the subject directly. So even while they may seem reliable and predictive they are not hitting the target of determining individual contributions.

Final Notes

Corsi Plus-Minus is just the next step in accurately determining a player's true value. There are many different versions of this methodology that could still be applied instead of Corsi, including goals, shots on net, Fenwick and xG. There is also a Bayesian version where instead of assuming every player starts the season with a rating of zero, we can tell the model what rating a player had the prior season so that it can more accurately estimate that player's true value. This version wouldn't help describe what happened during a given season quite so accurately but it could help provide an even great idea of a player's value. I have looked pretty heavily into this version of CPM and hopefully might release it shortly. In a follow-up post I hope to explore further correlations (ex. Y -> Y+3) as well as how CPM relates to time-on-ice and salary. 

The data is posted below in both as a spreadsheet and in the form of some Tableau visualizations. I also have noticed an error with the Comparison tab in my original Tableau that I am unable (for some  unknown reason) to update, so I created a new Tableau (posted below the spreadsheet) that now includes player salary data.
  • OCPM/DCPM/CPM are rate stats (per 60 minutes of ice-time)
  • The Impact version of these stats apply a player's ice-time to determine their actual impact in a given season.

Thanks to C. Tompkins for helping me with the Tableaus, as well as War-On-Ice and Puckalytics for the data. If you are interested in reading about topics like this in further depth, I will direct you to previous work done by Brian McDonald (here, here & here) that was an instrumental guide in my work.

Please let me know if you have any thoughts, questions, concerns or suggestions. You can comment below or reach me via email me here: DTMAboutHeart@gmail.com or via Twitter here: @DTMAboutHeart  




17 comments:

  1. I really like this work, and how readable this is. I have a couple questions, which may or may not be smart.

    1. Does this assume standard OLS assumptions (constant variance, normally distributed residuals, etc.)? If so, are those assumptions valid?

    2. To what extent does coaching still play a role here? In theory, it seems like it could be somewhat accounted for because you're comparing players on the same team to one another through the dummy variables. I'm a Leafs fan, and the first thing I noticed when looking at Phil Kessel is that his DCPM dropped drastically in 2012/2013, and stayed at the same level for all subsequent seasons. I'm curious if being coached by Carlyle had that effect. A similar thing happened to Nazem Kadri, but his most recent year was a bounceback.

    3. If I'm not mistaken, the way this adjusts for teammates is due to the regression, right? So in theory, the coefficient for player A is measuring his impact after accounting for the impact of players B,C,D,E, etc.

    Thanks again for the great work. This looks really promising.

    ReplyDelete
    Replies
    1. Thanks. I will try to answer your questions to the best of my ability:

      1. I believe so yes. It is still a linear model but with reduced variance.

      2. Coaching would definitely still play a role here. If player's are being asked to do tasks that are ineffective (or don't fit their skill sets) such as dumping the puck the puck in instead of carrying it into the zone this could hurt their production on the ice. I think coaching can be over-stated unless you are looking at extreme cases (both good and bad), where I believe they can definitely have an effect on certain players. I would consider Carlyle to fall under the category of an extreme case.

      3. Exactly. I think the simplest way to picture it is that if you have 5 average players their CPM will be zero. If you were to swap one of these players out for a non-average player, CPM shows what effect this new player would have.

      I went a little all over the place with question 2 but hopefully those answers helped and if you have anymore don't be hesitant to ask.

      Delete
    2. Thanks so much for the clarifications. Great work! I look forward to reading more.

      Delete
    3. I really want to thank Dr Ero for saving my marriage. My husband really treat me bad and left the home for almost 1 month i was sick because of this, then i contacted eromosalelovespell@outlook.com for help, he told me that my husband is under a great spell of another woman. then he cast a spell of return back of love on him. And he came back home and ask for forgiveness and today we are happy again,Don't give up just yet, the different between 'Ordinary' & 'Extra-Ordinary' is the 'Extra' so make extra effort to save your marriage/relationship if it's truly worth it.eromosalelovespell@outlook.com website http://eromosalspiritualtemple.webs.com or call him with +2348161850195

      Delete
  2. Excellent article. Well researched, well written and well laid out.

    I think an important test would be to do the same tests you have done above but to do so only with players that have changed teams. If we are measuring individual contribution when teammates change we want to see reliability maintained. I have been wanting to do this sort of analysis but just haven't gotten around to it yet. Not sure how easy it would be for you to do with the data you have but if you could have a look that would be awesome.

    ReplyDelete
    Replies
    1. Thanks David, much appreciated.

      I do think I have the data handy for such an analysis, though I will have to figure out what to do with players who played on multiple teams in one season. I am planning on doing a follow up post where I look into more tests like that (ex. salary/TOI analysis, auto-correlation Y -> Y+2, etc...) so this will definitely be a welcomed addition.

      Delete
  3. Question for you. On your DCPM graph is a positive number a good result for the player or is the negative number the good result for the player? So on the Blackhawks graph are both Hammer and Rundblad considered having the best or worst defensive production?

    ReplyDelete
    Replies
    1. A positive number for DCPM is a good result. The regression would give negative is better but I flipped the signs so that positive is always better for both OCPM and DCPM.

      Delete
  4. Great work! I like that you take the time to explain the statistics behind the methodology. The more we can help the everyday fan understand the statistics, hopefully the less resistant they will be toward them. I'll definitely have come back to study your article further; a bit rushed right now. (I'm going to have to start using Tableau. The visuals do such an effective job of representing the data.)

    ReplyDelete
    Replies
    1. Thanks. That's great to hear. If you have any questions don't be hesitant to ask.

      Delete
  5. I read this post and its tell us how solve accounting questions with your class fellows and how to help each other when you see and problem in answer thanks for sharing reword my paragraph .

    ReplyDelete
  6. We can writing a video game script which is useful as questions and it help each other for better understanding. It is really nice post.

    ReplyDelete
  7. Great work. I'm developing a model myself which I hope also solves some of the problems you mentioned but using a machine learning approach, i.e., avoiding regression analysis altogether (find me on Twitter @latent_variable). This post does a great job contextualizing and articulating the problems with current player evaluation models. Look forward to reading more.

    One semantic point: I think what you're calling multivariate regression is actually multiple linear regression. Multivariate regression usually implies more than one dependent variable, see http://www.ats.ucla.edu/stat/stata/dae/mvreg.htm

    ReplyDelete
  8. Thank you for your miracle Doctor Osemu Okpamen

    This article is dedicated to the Doctor Osemu Okpamen. I have been married with my wife for 5 years and recently she broke up with me and it hurt me deeply when she told me to leave her alone and that she does not love me anymore when i was always faithful and honest to her. I tried all the ways to get her back buying her what she wants like i always did and she still left me heart broken and she even has a new boyfriend which destroyed me even more until a friend of mine from high school directed me to this genuine spell Doctor called Osemu Okpamen. This man changed my life completely. I followed everything he told me to do and my wife came back begging for me back. I was stunned everything happened exactly like he told me. I had faith in everything he told me and everything was true. Also he was there every moment until i got my happiness back and he also provides spells that cures impotence, bareness, diseases such as HIV/AID E.T.C You can contact him via email at { Doctorokpamenspelltemple@yahoo.com } or visit his website http://www.doctorokpamenspells.com. He will help you in anything you need and quick to answer once you contact him.

    You can also call me for more info +1 (914)-517-3229.

    ReplyDelete
  9. I really want to thank Dr Ero for saving my marriage. My husband really treat me bad and left the home for almost 1 month i was sick because of this, then i contacted eromosalelovespell@outlook.com for help, he told me that my husband is under a great spell of another woman. then he cast a spell of return back of love on him. And he came back home and ask for forgiveness and today we are happy again,Don't give up just yet, the different between 'Ordinary' & 'Extra-Ordinary' is the 'Extra' so make extra effort to save your marriage/relationship if it's truly worth it.eromosalelovespell@outlook.com website http://eromosalspiritualtemple.webs.com or call him with +2348161850195

    ReplyDelete
  10. This is my first time i visit here and I found so many interesting stuff in your blog especially it's discussion, thank you.The real marriage killer

    ReplyDelete
  11. I was in total despair when I found Dr. Todd. My life was going terrible and I didn't know if I was coming or going. I had just gone through a rough divorce, wasn't making enough money to sustain me and my children, and my 17 year old son had just gone to jail for the first time. When I talked to him, I immediately found a sense of peace. He was very honest with me and I could feel that. He also told me that everything would be okay. After my work began, things began to change. My bills were all caught up, the relationship I was in became much stronger, I was never FLAT broke, and my son was released from jail earlier than we expected!! I also completely got over the failed marriage and began to move on. And, received a better position at my job which will cause an $800 per month increase!! I felt completely comfortable with the work that was being done because I was always encouraged by Dr. Todd. manifestspellcast@gmail.com is the BEST!!!!

    ReplyDelete