October 23, 2015

Strength of Schedule matters a lot in NCAA Rankings

By Ryan Harmanis

A Tough Lesson From the Initial NCAA Regional Rankings – Scheduling Matters

Change of plans. I’d intended to conclude my three-part series on conference strength today, but the big news this week involves the rankings that actually matter, so I’m going to address those instead. With only two weeks (three weekends) left, the first NCAA regional rankings—the ones used to determine at-large (Pool C) bids for the NCAA tournament—were released on Wednesday. When compared to the D3soccer.com Top 25, these rankings might appear surprising. In the West, No. 6 Whitworth is not ranked at all, while No. 12 Texas-Dallas is only fourth. In the Great Lakes, No. 4 Kenyon (seventh), No. 8 Thomas More (fifth), and No. 22 Case Western (sixth) are far lower than many expected. Elsewhere, No. 15 Christopher Newport (eighth) and No. 17 Kean (fifth) are deep down the South Atlantic. And No. 13 Eastern slides to fifth in the Mid-Atlantic, while No. 24 MIT is tenth in New England. What gives?

In a word: scheduling. Strength-of-schedule is one of the major criteria for the NCAA tournament committee, and the lesson here is that scheduling really, really matters. D3soccer.com editor Christan Shirk’s excellent article explaining the regional rankings breaks down how to calculate strength-of-schedule (SOS), but I’m just going to touch on its impact. In years past, the committee has not even ranked teams whose strength-of-schedule is below 0.500, almost guaranteeing that those teams can’t receive an at-large bid. That doesn’t appear to be a hard-and-fast line this year, but teams with sub-0.500 SOS ratings find themselves far down the rankings, even if they have gaudy records. And come Selection Monday, teams with low SOS ratings will find it very difficult to get a Pool C bid. In the past two seasons, only three teams have received Pool C bids with SOS below 0.545: Texas-Dallas (0.516) in 2014, Salisbury (0.521), and Gordon (0.517) in 2013.

Now take a look at the D3soccer.com Top 25 and the all-important NCAA regional rankings, and look very closely at strength-of-schedule:

Not coincidentally, all those Top 25 teams with very low SOS ratings are the teams whose regional rankings are far off their D3soccer.com rankings.

The outliers are Calvin, top of the Central at 13-0-1 despite a 0.523 SOS rating, and Carnegie Mellon, unranked in the Great Lakes even with a 0.541 SOS. However, the Central isn’t strong in voters’ minds, as Calvin is the only Top 25 team from that region, so perhaps that isn’t too surprising. Still, I’d keep an eye on someone like Wheaton (Illinois) passing Calvin soon if Wheaton’s winning percentage increases to go along with a dynamite 0.594 SOS. As for Carnegie Mellon, the Great Lakes is over-represented in the Top 25 with six teams, and Carnegie Mellon easily has the lowest winning percentage relative to their SOS.

So what does this mean? Frankly, many of these teams could be in trouble. Take a look at the teams I mentioned above and their SOS ratings: Whitworth (0.468), Texas-Dallas (0.489), Kenyon (0.494), MIT (0.497), Eastern (0.503), Christopher Newport (0.513), Kean (0.513), Thomas More (0.516), and Case Western (0.523). For comparison, the average SOS ratings for Pool C teams was 0.578 in 2014 and 0.570 in 2013, and 34 out of 37 Pool C teams had SOS ratings above 0.545. So if only one or two teams get Pool C bids with SOS ratings near 0.500, many of these teams might be left on the outside looking in unless things improve.

Having said that, three caveats. First, the SOS rating is going to change in coming weeks, so these teams might see their SOS improve to take them out of the danger zone. Kean, for example, could still get five really good opponents with two tough NJAC matches and then three possible rounds in the NJAC tournament. Same goes for Kenyon, with a trip to OWU on the horizon and possible rematches with OWU and DePauw in the NCAC tournament. Second, another important factor, record versus ranked opponents, doesn’t come into play until next week. That could really benefit teams with multiple good wins like Thomas More (2-1-1 versus ranked) and Kean (3-1-0). Finally, every team on this list still has a great opportunity to win their conference’s automatic bid to the NCAA tournament, so they control their own destiny.

Why Does Strength of Schedule Matter So Much?

One consideration is that scheduling seems to have an outsized impact. For example, based on the rankings a team with a 0.800 winning percentage and a 0.575 SOS will be ranked above a team with a 0.900 winning percentage and a 0.510 SOS. In my opinion that makes sense, because balance matters and a really low SOS should hurt a team in the exact same way a low winning percentage does. I want teams to schedule better games, so I prefer to reward the team that does that—even if it picks up a couple extra blemishes—over the team that posts a great record without being tested. And using record-versus-ranked in subsequent weeks allows for a team to state its case even though it may have a lower SOS rating due to a weaker conference or scheduling quirks (cancelled games, opponents having uncharacteristically bad years, etc.).

Another positive thing about these criteria is that they're predictable and hold the committee accountable. Of course, every schedule can fluctuate based on how teams are doing year-to-year. But if you schedule the way teams like No. 18 Loras (0.620 SOS) or Wheaton (Illinois) (0.594) regularly do it’s not a problem, even in a weak conference or with a few teams having down years on the schedule. Point is, teams are aware of the criteria and have been for a long time, so while it’s disconcerting to see regional rankings drastically differ from the polls, it’s not surprising in context.

Finally, I prefer using these objective criteria to subjective considerations. Not because it will always result in perfect rankings, but because the alternatives are worse. Heck, four of the teams with weak SOS ratings were in my Top 10 last week, so it’s hard to say I really think these rankings always reflect the “best” teams. But what qualifies as the “best” team? I’m one person who gets to make a subjective determination based on my personal opinion on a weekly basis. I don’t have to be accountable with my rankings, I can just make a decision and give a short explanation. The committee doesn’t have that luxury. It needs to provide predictability, so teams know what’s required to make the tournament, and accountability, so teams can look at who received Pool C bids and know it was done in some systematic, fair way. The minute we allow subjectivity into the system—because the committee is convinced that Team X is better than Team Y, even though Team Y has the better resume based on the criteria—we open up a huge can of worms. Who’s the committee chair? Which coaches are voting? Does someone have a bone to pick with another team? What makes us so sure that the team with a 0.500 SOS is better than the team with the 0.600 SOS, especially if the records are in the same ballpark? Suddenly, we’re introducing considerations that are not based on evidence, but rather on subjective opinion or even off-field concerns. With that in mind, it seems that the NCAA has decided that the benefits of predictability and transparency outweigh the costs of occasionally ranking a team lower than what “seems” correct.

I understand that this isn't much solace for teams who might get the short end of the stick because of their schedule. Is it the best possible system? Probably not. But this isn’t Division I football or basketball, where you can have a committee that watches every relevant team play almost every game. That’s just not feasible at the Division III level, so the NCAA has made what appears to be a conscious decision to focus on objective factors.

This is definitely a situation to monitor and one that has seen quite a bit of lively discussion over on the D3sports forums. I’m very open to feedback and/or ideas as to how we could tweak the system, even though I have no power to do so. If you have suggestions, ideas, or comments, please feel free to contact me, as I’ll be re-visiting the regional rankings each week leading up to Selection Monday. In any event, I very much hope things will work themselves out and that we avoid seeing a team that deserves to be in the NCAA tournament miss out.

Until next week, where we’ll continue this discussion and (finally) wrap up the conference strength debate heading into conference tournament week.

Ryan’s Boxscore Top 10

The reasoning is slimmed down this week, but that mainly reflects the fact that only two teams picked up blemishes in the past week and others didn’t do enough to crack my Top 10.

1. Amherst (13-0-0, D3soccer.com No. 2) – Not much to add, except to congratulate the Lord Jeffs on winning the NESCAC with two games to go in dominating fashion. One thing to watch: how does a team that has only given up two goals all year react if (when?) they’re down 1-0 in the second half of a playoff game. But, to this point, Amherst is a clear number one.

2. Franklin and Marshall (14-0-0, No. 1) – The Diplomats are rolling, and the offense has found some life in recent games. The big one is tomorrow’s matchup with Haverford, which looks like it might decide the Centennial title.

3. Calvin (15-0-1, No. 4) – Not much to add until the Knights suffer a loss or run the table in the MIAA. Only four goals against, so the same issue with playing from behind could crop up down the road.

4. Whitworth (12-0-1, No. 6) – Whitworth can completely avoid any issues with its schedule by locking up an NCAA bid this weekend. If the Pirates beat Pacific Lutheran (11-4-0) and Puget Sound (9-5-0), they’re in as the NWC’s automatic qualifier.

5. Kenyon (12-1-0, No. 3) – Big win over No. 23 Denison sets up a showdown with OWU next week. Assuming neither team slips tomorrow, Kenyon will need to win to claim the NCAC title. Kenyon’s last conference title occurred in 2007, courtesy of a 1-0 win at OWU. Will history repeat itself?

6. Thomas More (13-1-1, No. 8) – Another perfect week in conference for the Saints, although they did offer up a wild 6-3 win over Westminster (Pa.) last weekend. That dominating 5-1 win at OWU is looking better by the game.

7. Trinity (Texas) (13-2-0, No. 5) – Eight straight shutouts now for the Tigers, although the conference title is Colorado College’s to lose. I’m sincerely hoping we get a third game between the two teams in the SCAC tournament.

8. Montclair State (15-2-0, No. 10) – The body of work is just too strong to ignore at this point, as even the losses are to very good teams. Bottom line, the NJAC is the deepest I can remember, and the Red Hawks are in pole position to win it outright.

9. Brandeis (12-2-1, No. 8) – Tough loss for the Judges at Washington U. (10-2-3), but the lack of scoring punch was bound to be a problem at some point. More worrying than the outcome is that Brandeis only had one shot on goal. Luckily, Brandeis still controls its own destiny to win the UAA, and a Pool C bid has to be a near certainty at this point.

10. Elizabethtown (14-1-1, No. 11) – What a season Elizabethtown is having. The Blue Jays have the Landmark title wrapped up, so now health and the automatic bid become the focus of the next two weeks.

Trending Up: Lycoming, Ohio Wesleyan, UW-Whitewater, York (Pa.)

Trending Down: Christopher Newport, Loras, Gettysburg, Rochester


Comments or feedback for the author? E-mail Ryan Harmanis.

Ryan's Ruminations


Ryan's Ruminations will go beyond the box scores to offer analysis and opinion on major storylines around the country.  Ryan will provide in-depth analysis of the current season and insight into important aspects of Division III soccer, augmented by fun and compelling stories about players, coaches, teams, and games.



Ryan Harmanis

Ryan Harmanis played for Ohio Wesleyan from 2007 to 2010 where he was a three-year captain. Following graduation, Ryan continued to follow the D-III landscape before joining D3soccer.com in 2013. He combines an analytical background with a passion for writing and the game of soccer. [see full bio]

Questions or comments?

»  E-mail Ryan Harmanis