Observations and Questions: Investigating offensive playcalling efficiency

This is the second part of an offseason series investigating mysteries about the 2018 Ducks football season, using my game charts and film study of the team last fall (here’s the previous one about the offensive line). This month I’ll be examining offensive playcalling and whether it was suboptimal or not. Two notes before we begin:

First, this article will be limited to what I can study rigorously and objectively: the success rates of various offensive plays in different down & distance situations. It is not concerned with general offensive philosophy, aesthetic preferences, special teams decisions, or anything else the lexically profligate lump into the term “playcalling”. Coach Cristobal has clearly decided to install a downhill, run-first gameplan as an organizing principle to the offense, and while I think it’s legitimate to wish he had done otherwise, evaluating that decision is outside the scope of this article. Fans who mislike this gameplan and criticize OC Arroyo’s playcalling for it are making the classic error of confusing strategy and tactics.

Second, my methodology in general is to chart every play prior to garbage time in each game including the bowl, but in this series I will also be excluding the entirety of the first three non-conference games as well as the Civil War. All four of those teams were essentially FCS-caliber, and dropping them boosts the average S&P+ ranking of the remaining nine opponents to a healthy #45.2 out of 130. Furthermore, I’m excluding a relatively small number of “weird” downs: trick plays, bad snaps, extraordinarily good or bad luck, 1^st & very short, 3^rd & very long, etc. What’s left is a healthy sample of about 500 downs that’s robust and representative of normal play.

The quantitative approach I’m using is testing if Oregon’s playcalling passed two tests. In each down & distance situation:

1. Is the playcalling balanced? That is, are defenses always guessing, or are they allowed to focus on stopping a particular play and/or disregard another?
2. Is the playcalling optimized? That is, when there are situational disparities in effectiveness between plays, are the plays that are more likely to succeed being called more frequently?

First we’ll review the overall numbers, then get into down & distance situations. In my conclusions I’ll wrap up what I consider to be main strengths and weaknesses of the offensive playcalling, as well as present some questions to consider going into 2019.

---

Overview

Oregon’s overall success rate as I tally this sample was 53.05%. That is essentially to say that on a fair bit, over half of their meaningful plays, they stayed ahead of the chains. For comparison, in similar samples of other teams I’ve studied over the years, that’s about a point below conference-championship level per-play efficiency. The best I’ve ever evaluated was Ohio St in 2014 which was at 59.72%; the worst was TCU in 2015 at 44.76%

Without taking down & distance into consideration, almost every individual play call (e.g., an inside zone run or a bubble screen) is within a standard deviation of the arithmetic mean for success rates. Oregon’s least successful play (minimum 10 attempts) is a left-side flag route, which only succeeded a third of the time; that’s probably not surprising given the Ducks’ outside receiver talent. Their most successful play was a right-side comeback route, which succeeded more than two-thirds of the time; also unsurprising given where #13 WR Mitchell lined up. A close second place was a power-blocked halfback dive right up the middle; considering that this is the bread-and-butter efficiency play, this should be expected as well.

Excluding screens and broken plays (like sacks and throwaways, those are hard to tell what the design was), I placed every play into one of six categories: short/intermediate/deep passes and inside/outside/complex runs. (A “complex” rush is my term, the category into which I put things like stretches and counters where technically the back is running between the tackles but it’s still helpful to differentiate, because they require more sophisticated blocking as well as patience and skill by the back.) Oregon called each of these six categories in roughly similar numbers – the most was inside runs, the least was a tie between deep passes and outside runs. The success rates of each of those six categories were almost indistinguishable, only a few percentage points above or below about 55%.

In sum, there is no play or category of plays that is dramatically different from any other in per-play efficiency. Generally speaking, this offense had nothing in its playbook that was either doomed to fail or guaranteed to succeed.

Screen Passes

The one possible exception to the above summary is screen passes. Only about 7% of snaps in this sample were screens, and given how successful they were that’s probably too low by three to four percentage points. I would make this a point of criticism, but Oregon’s coaches seemed to recognize this and adjust, as they threw a lot more of them as the season went on: in the last four games I examined, Oregon threw more than twice as many screens as they did in the first five games of the sample.

The overall number of screens is too few to break them up into individual types and say anything confidently about which are more or less successful. However, I’m happy to say that the data suggest the “super flanker” I highlighted in my film reviews of week 10 and week 12 was their best screen concept (I’m less happy about the dumb name I use for them). Here’s my favorite one in this sample, it’s an outside screen to #34 RB Verdell with #87 TE Bay as the “super” blocker who’s occupying two defenders at once because neither can get a good angle past him:

1^st Down

Oregon certainly lived up to its run-first offensive identity: they rushed on 1^st down at about a 2:1 ratio compared to passing. They were successful at both running and throwing on 1^st downs and had a combined success rate several points better than the global average, but this comes entirely from a pretty significant ten percentage point jump in efficiency when passing on 1^st down (63% vs 53%).

Within their 1^st down passing distribution, they hit the nice bell curve of risk and reward that’s optimal for most teams: a quarter were short passes at 75% success, a third were intermediate routes at 68% success, and a fifth were deep shots at 57% (another fifth were pocket breakdowns and otherwise improvised plays).

The vast majority of 1^st down passes were out of play-action, RPOs, or rollouts, so the bump in passing success on 1^st downs is probably due in significant part to defenses expecting the run. Therefore, even though there’s a mismatch here -- from their success rates in a vacuum it looks like they should be passing more on 1^st down -- I don’t think this qualifies as poor run/pass optimization. If the passing success bump were bigger or the rushing rates were underwater, then there’d be a strong case for re-evaluating the selection mix (and I’ve made exactly that case about Stanford, Arizona St, and Michigan St this year when those conditions obtained), but as it is, this is just a run-first offense behaving nominally.

Where I do have some tactical criticism is the selection of rushing play types. Despite being just as successful as other kinds of runs, outside rushing (OZRs, outside power, sweeps, etc.) are only used on about 20% of all rushing plays, 1^st down or otherwise. Sweeps in particular are really under-utilized: even though there’s a pre-snap crossing motion or fake sweep on the vast majority of all plays, I only counted six actual handoffs to the man in motion during the entirety of these nine games. While motion was very effective in weeks 4 and 6 (Stanford and Cal), by the end of the year defenses were safely ignoring it much of the time. Of course, that just created more opportunities like this one, where the line isn’t even blocking that well but the skill players do a great job (the official box score marks this down as a pass to #30 WR Redd because of that little six-inch push pass; your faithful film reviewer is not so easily fooled):

2^nd Down

There’s a paradox on 2^nd down: the overall success rate is about a point and a half lower than the global average, and yet the run/pass optimization is the best out of the three downs. It’s when Oregon has the highest percentage of screens and outside runs, has the greatest diversity and even distribution of playcalls, and, where there are discrepancies in situational success rates, they match up to call frequency (that is, the more successful plays are called more often, the less successful less often) pretty much perfectly.

Figuring out why this is requires breaking 2^nd downs into two categories, and the data show playcalling tendencies flip noticeably based on how many yards they have to gain for a 1^st down: the natural dividing line appears to be about six and a half yards. This number is essentially the difference between being ahead of the chains or behind them (happily, I use a 3.5 yard gain on 1^st & 10 as my metric for a successful play; nice to know my data-gathering lines up with what the coaches are actually doing).

When Oregon faces 2^nd & 7 or worse, they cut down on rushing to about 30% of the time, and have a situational rushing success rate of about 43%. Screens and downfield passes make up the rest of the 70%, and they’re successful about 54% of the time. To me this looks like good run/pass optimization: they’re calling the more successful plays on a more frequent basis, but they aren’t cutting rushing out entirely so as to keep the defense honest.

It’s concerning that Oregon gets successful rushes at about 10% lower rate than usual in this down & distance, but given that its playcalling mix in this situation is, if anything, less predictable, this appears to be execution issues more than anything else. I tried pulling individual games out of the sample to try and isolate the problem, and I think I found it: Oregon’s 2^nd & long rushing success rate jumps by eight points when I exclude the week 9 game against Arizona. Readers may recall in my film review that week I wrote about the baffling offensive line problems in that game which inexplicably affected even #68 LG Lemieux, by far Oregon’s most consistent lineman and the only one unaffected by the injury rotations. So I’m willing to write off that weird number as the result of a one-off game and move on.

When Oregon is ahead of the chains on 2^nd down, however, we start seeing significant missed opportunities.

On 2^nd down with six and a half yards to go or fewer, the Ducks reverse their tendencies and rush about 54.5% of the time, with a very impressive 62% success rate. They attempted a downfield pass only 41% of the time in 2^nd & short, which is appropriate given the success rate … but that success rate is terrible, only 39%. That’s their lowest situational success rate of any play type in any down & distance I examined and well below what even a mediocre team averages, much less a championship-caliber one.

This specific failure in the passing game eats up the progress over the average that Oregon gets rushing on 2^nd & short and using good optimization on 2^nd & long, and is why the overall 2^nd down success rate looks fine. Also, since Oregon is so successful passing on 1^st down, and, as we’re about to cover, passes so heavily on 3^rd down, this issue also doesn’t show up in the overall passing numbers. It’s a problem that’s been hiding in plain sight.

3^rd Down

The overall success rate on 3^rd down dips a hair under 50% for this sample, and I think there’s a good case to be made that playcalling gets too conventional on 3^rd down, primarily because passing right to the sticks becomes the predominant call. Short and intermediate passes make up 44% of all 3^rd down playcalls, but I believe the intention is for that number to be higher and it’s obscured by the pocket breakdown frequency jumping on 3^rd down to nearly 18% (it’s about 5.5% on 1^st and 9% on 2^nd). Deep shots make up 15% of 3^rd downs and are successful 62.5% of the time, though I suspect that’s largely because of the surprise factor and Oregon probably wouldn’t sustain that rate if they threw it deep much more often. Very few screens are called on 3^rd down, which is probably appropriate (most fans rightly get upset when the ball is thrown behind the sticks on 3^rd). Rushing is comparatively rare on 3^rd downs, and again outside runs in particular are probably being called too infrequently because the success rate is the same as other types.

The dividing line between tendencies on 3^rd down moves up to five yards to go; although tendencies with exactly four yards to go are something of a grey area, I decided to put those plays in with the short yardages to get roughly even numbers between ahead-of and behind the chains.

When Oregon is facing 3^rd & 4 or better their success rate jumps to 55.5%, somewhat above the global average. However, this seems to be despite rather than because of optimal playcalling: despite only a 44% success rate on downfield passes in this situation, they’re called 63% of the time. Rushing is flipped: an astonishing 76.5% success rate, but only 31.5% of the playcalls on 3^rd & short.

3^rd & 5 or worse is where playcalling is probably far too predictable. While I don’t think anyone would be surprised to learn Oregon mostly passes in this situation, I was shocked to see that it’s 92.5% of the time. And this isn’t restricted to 3^rd downs with a huge amount of territory to make up, either: I recorded zero rushes on all 3^rd & 5 and 3^rd & 6 plays in this sample, and only three on 3^rd & 4. Worse, the passing success rate on 3^rd & long drops to 47%, resulting in a whole lot of punting. However, I can say that at least the individual passing play type selection is nicely varied, with even splits between short, intermediate, and deep passes, and the success rate and play type do match up properly here.

Scott Olmos-USA TODAY Sports

---

Conclusions

Reviewing the playcalling numbers really made sense out of the strategic gameplan of Oregon’s offense. They want to use a high-efficiency play on 1^st down to at least get a modest gain (usually a run between the tackles, sometimes a surprise pass), never get nothing or go backwards. This is the step that’s going very well, and it strikes me as ironic that so many fans are complaining about it.

The second step (the “oop” in this alley-oop) is to take advantage of the playbook being completely open on 2^nd & short: it’s an easy pickup with a safety net of another down if it doesn’t work, and keeping the defense guessing makes it even easier. You can either use another high-efficiency play to keep the drive moving, or go for something big. And true to that principle, this is when Oregon attempts most of its deep shots and potentially explosive plays, with the playcalling distribution matching the individual play type success rate well. This combination should be a recipe for priming and then detonating explosive plays.

Here, I think, is the resolution to the above paradox, and the “doughnut hole” in the middle of Oregon’s offense in 2018: 2^nd & short is also the situation in which the lion’s share of both the well-documented dropped passes and #10 QB Herbert’s misfiring issue crops up. The entire offense is built around efficiently generating these 2^nd & short situations (and it does), and then going for its explosion plays when they’re set up (and those are getting called) … but unacceptably often, the execution is a dud.

I then think they get gunshy on 3^rd down after dealing with this much disappointment on 2^nd, taking a pretty conservative mostly passing approach. I think the data indicates that Oregon should be bolder and more trusting of its rushing attack on 3^rd & medium; at the very least this would give the defense more to think about.

Questions

Here’s what I’d like to see resolved for the 2019 season:

1. Will Oregon recognize how successful it is going outside? The data indicates that screens and outside rushes should be getting called more frequently; that seemed to happen with the former but less so the latter. It seems obvious that if you can get more variety without sacrificing per-play efficiency, there’s only upside here. I think that fan criticism of a “boring” offense predicated on inside rushes is overstated and misunderstands the gameplan, but in this sense there’s a kernel of truth to it.

2. Will the 2^nd down explosive threats emerge? In future articles this offseason I’ll investigate the passing and rushing offenses and attempt to identify why specifically these weren’t blowing up when they should be, but the coaches are surely already aware of this, and looking at the way they’ve been recruiting receivers they have quite a few options here to improve.

3. Will they grow a backbone on 3^rd down rushing? For all the kvetching I’ve read about rushing too much, the data conclusively demonstrate that on 3^rd down they’re actually rushing far too little, but even so I was still surprised just how successful and how rare these are. This seems like the biggest clash between Coach Cristobal’s stated “impose your will” preference and actual play selection; I’d like to see Oregon run on 3^rd & 5 and have the confidence that even if they only pick up a couple yards, they’ll hurry up and pound it again on 4^th down. We only saw them do that a couple of times in 2018 but they’re memorable to fans and demoralizing to opponents.