I just can’t stop thinking about this Ben Rothenberg piece, aptly titled “Unforced Error Is Unloved Statistic Among Tennis Players“. You probably read it already — it came out during the first week of Indian Wells. If you haven’t, do take a look. Rothenberg tells the story of how this particular statistic came to be, and he has quotes from Andy Murray, Sam Sumyk (Azarenka’s coach), Kevin Fischer (senior communications manager for the WTA) and Roger Federer about these pesky unforced errors, as well as statistics in general. Here you have three of these quotes:
“I think if you have two or three different people recording unforced errors, you’re going to get two or three different figures.” – Kevin Fisher
“I’m not interested on the statistics, because I’m not sure it’s accurate.” – Sam Sumyk
“The forced error, unforced error count is always extremely tricky.” – Roger Federer
I was not surprised when I read this, but it did amuse me. Why so much resistance towards a statistic that is so basic? Sure, we can all agree that a large part of the unforced errors in a match is attributed in a subjective way. The obvious exception is the double fault, which is the lone objective element of the unforced error count. However, after years of watching tennis, it doesn’t seem to me that an unforced error is such a mysterious thing as to generate this much skepticism among some of the pros. I’d probably argue that if you have 100 passionate tennis fans tallying the unforced errors of a random match, they would probably do a good job determining what is an unforced error and what isn’t (provided the match doesn’t involved their favorite player, naturally).
Anyway, since I don’t have the funds to have 100 people tally unforced errors, I decided to run a little experiment on my own. During today’s match between Sorana Cirstea and Angelique Kerber, I tallied the unforced errors myself. In addition, I could take advantage of the new live stats provided by TennisTV during their streaming of Masters 1000 events, which even break down the unforced errors by whether it was a forehand or backhand mistake. Looking at those numbers will help zero in on where the discrepancies lie at the end of the match.
What is my hypothesis going into this experiment? That any difference between my tally and the official tally will be minimal, if any.
Without further ado, here is what transpired in the first set, which Cirstea won 6-4 (the blue highlight on game number three means that Kerber broke serve in that game, and the red highlights on games six and 10 mean that Cirstea broke serve in both of those games):
As you can see, I tallied 13 unforced errors for Kerber, and 14 for Cirstea. Here is the official tally from TennisTV:
Notice that TennisTV gave Kerber one more unforced error than I did, while they assigned Cirstea two more unforced errors. Regarding the latter, I can guess where at least one of these came from: in one of the first games of the match, Cirstea hit a smash into the net. I gave her a pass on it, since it wasn’t a straighforward smash — Cirstea had to backpedal a bit to hit it. However, I can see how that can be tallied as an unforced error.
Now for an important disclaimer: using TennisTV’s nice new controls that allow you to go back (or spring forward) 10 seconds in a match, I was able to review several points to see if I got it right. Naturally, the folks tallying the stats live don’t have that luxury. Huge advantage for me.
Anyway, let’s move on to the second set, which Cirstea won 6-0 (the red highlights on games two, four and six just mean that Cirstea broke serve in those games):
And here is my final tally:
The broadcast didn’t produce a segregated second set graph, but they did produce a final stats graph:
Notice that my tally is only off by one unforced error for each player. That’s not bad, right? Now let’s see what the live TennisTV stats (the ones only users can see and are not part of the broadcast, though they come from the same source). The red highlights are mine:
Notice that I actually have the same numbers as the Tennis TV crew for forehand unforced errors for both players. The official tally gave Kerber an extra backhand unforced error, while it seems like I gave Cirstea one backhand error too many. Notice that the two overhead smashes Cirstea missed have not been tallied as either forehand or backhand unforced errors. I judged that only one was an unforced error and tallied it as a forehand unforced error.
Regardless, you can clearly see that the difference between tallies is tiny.
Conclusion
As expected, there was a variance, but it was quite minimal. Hence .. .why the resistance to a statistic that could be very useful — if not for players, for coaches? Is it because the nature of the stat, which illustrates an error, a mistake, something that shouldn’t have happened, might affect the psychological state of a player? I can see that. Tennis is such a mental game, and it’s not like you can sub someone in if a player is having a horrific day with unforced errors (as it happens in basketball when a certain player commits way too many turnovers or misses way too many shots). Also, focusing on errors might erode a player’s confidence.
However, wouldn’t you want to see these numbers as a coach? I keep thinking of the Indian Wells matches between Juan Martín del Potro and Andy Murray, and then the semifinal between Del Potro and Djokovic. In both of those cases, the higher ranked men had an inordinate amount of unforced errors on their backhand side, which is the bedrock on which their baseline game is built on. Wouldn’t Marian Vajda want to know this information, so that he can later watch the match again and try to figure out what happened during all those backhand unforced errors?
The unforced error stat is not a solution to anything. We can all agree on that. But it is a handy tool to figure out if a player is struggling with a particular shot. Then you can start asking why that is happening: it could be that the player is not moving their feet, that they took their eyes off the ball, that they didn’t put enough spin on the shot, etc. That’s for the coach to assess, but it’s sometimes visible to anybody who’s watching the match, too.
As viewers, it’s always fascinating to see the winner/unforced error count. It usually tells you something about how the match went. There’s the famous example of the 2007 Australian Open semifinal between Fernando González and Tommy Haas, which I wrote about here. In that match, González was credited with 42 winners and just three unforced errors. You might have heard that the Australian Open has a reputation of being kind with the unforced error count. As I watched that semifinal again, I paid close attention to the unforced errors, and the official tally was indeed off … by just two. Fernando González committed five unforced errors (all in the second set, amazingly) in the entire match. And had 42 winners. That tells you all you need to know about how well González played.
In the same vein, you can see just how good Sorana Cirstea was in that second set today: the official tally had her committing only one unforced error, while I had her committing only two. Either way you look at it, Cirstea’s play was remarkable.
Regardless, the main lesson here is that when you see those unforced error counts on your broadcasts, or when you use TennisTV, you can be reasonably confident that they are a fairly close reflection of what is happening on a tennis court. Looking at the numbers might lead you to identify certain trends, as well as to ask many questions.
And that’s the best part of any statistical analysis, no?
I don’t think UFEs, in and of themselves, tell you very much about a match. They’re a bit like the first serve percentage, probably the single most quoted statistic. The amount of times a player gets the ball into play on their first serve doesn’t tell you how effective the first serve is, and it also doesn’t tell you how effectively a player is serving in advantaged, neutral and disadvantaged positions in a game.
For example, how often does a player win a point BP down or 0-30 down after getting their first serve into play? Seems to me that that counts for more than getting a cheap point when 40-0 or 40-15 up.
So, back to UFEs: what’s actually happening in the match? Does a play commit UFEs attempting to keep a neutral ball rally going, or while attempting to force the play? How much risk is appropriate (eg for a player ranked 50 playing one of the top 5?) If I’m a coach, I think I worry about whether a player is executing a game plan and hitting the right shots under pressure, not a raw total.
You make a lot of good points (and suggestions for other stuff to track), Andrew. And I agree partially that a raw total for UFEs at the end of the match doesn’t say much. But if you have an UFE count by game, and then segregate FH and BH UFEs…that will inevitably be revealing. Particularly if you add video to the mix and go back and check out a particularly problematic game.
Yes, agree with comments above. Plus, I’ve got to think that that if Sam is sat watching Vika play he is pretty clued into what is working and what is not during her matches. UFE is only one way of explaining what happens during a match and although interesting and potentially valuable, most of us watching a match have a pretty good idea of whats going on too without seeing the stats. They tend to confirm for me what I have been seeing rather than illuminate something new.
tclairel, the key here is which stats are being tracked and which aren’t. Like I told Andrew, I do agree that a raw UFE total at the end of the match is pretty useless. What you would like, either as a coach or a fan, is to know when those UFEs happened, and then have a chance to go back and watch any problematic patches. You might find stuff there that you had no idea about.
For example, I don’t know if you saw that I tracked Ferrer’s FHs and BHs during a match. That’s never counted in any broadcast. I fully expected Ferrer to hit way more FHs than BHs, and instead, I found that he hit just 10 more in the entire match. I also found that he tends to hit more BHs than FHs during return games.
There are all sorts of things that Sumyk could see in statistics…provided they are interesting statistics, and not just raw totals at the end of the match.
UFE is generally a useless stat. Almost every error is somewhat forced; almost every error is somewhat unforced. Also, it doesn’t take into account how many times the player missed relative to the amount of times the player hit the ball in. Thar’a more important imo. It’s strange that there’s no ‘percentage of shots which were errors (forced or unforced)’ stat.
We disagree, Ilija. I do think UFEs are revealing, but not so much as a raw total at the end of the match. Game-by-game FH, BH UFEs, those tend to say things.
About the stat you mentioned…I wrote a post about it today! I do think that it would be great if we could have Forehand and Backhand Efficiency Ratings for every match, for every player. That would be really awesome.
Is a DF always objective? Aren’t there times when the returner puts so much pressure on the serve that it gives the server the heeby-jeebies, and along comes a double?
Enjoyed the read and the comments. Hope you will do one of these for Wimbledon – wasn’t there a lot of talk last year about how the UFE count was artificially low?
I would argue that a DF is always objective, since all the returner is doing is standing there. The server is in complete control of the shot – he/she is not reacting to anything. Yes, there is psychological pressure, but that can’t possibly be measured, I don’t think.
Glad you liked this, Jewell! I will do one for Wimbledon. I read this great piece by Carl Bialik today (it’s from last year’s WTFs), and it does mention that Wimbledon tends to be generous with UFEs. I had heard the same about the AO, so I’ll definitely keep an eye on that once we get to the grass!