Among the authors’ assertions is a claim that the graduation rate is “a standard means of evaluating college effectiveness.” They then discuss graduation rates at a number of HBCUs which they suggest compare unfavorably to the national average. It may be true that graduation rates are often cited when discussing college quality; however, no expert trained in education policy would assert that these are useful measures of college effectiveness.[i]
The reason for this is quite simple: colleges don’t start out with the same students.
It is almost taken for granted today in discussions of K-12 policy that it would be foolhardy to assert that differences in a school’s outcomes result from the school being more or less effective without considering characteristics of the students who attend. Because evaluations of college effectiveness are less common, it is perhaps less widely known that measures of college effectives require the same consideration to be credible.
Yale has a graduation rate of 98% while the University of West Georgia has a graduation rate of 43%. If the 18 year olds walking on to the campuses in New Haven and Carrollton were randomly selected, it might be fair to attribute this difference to the schools’ relative effectiveness.[ii] But no one would be willing to make such a silly assumption. We all know that Yale freshman arrive at school with backgrounds, skills, and experiences that differ markedly from the typical student at the University of West Georgia.
Even if we consider just one characteristic about students (the SAT score they got prior to applying to college), seventy-two percent of the variation in college graduation rates is explained. The graph below shows the relationship between average SAT score and Cohort Graduation Rate for U.S. colleges and universities.[iii]
While some colleges graduate more students than one would expect based on average SAT score and others graduate fewer, most schools’ graduation rates are pretty well predicted by the SAT scores of the students who attend.
For this reason, if we want to make claims about school effectiveness, we need to compare the graduation rates of schools that serve similar students.
In this light, HBCU graduation rates are just about what one would expect given the students served. In fact, the schools graduate about one percent more students than would be expected if SAT scores are used to predict graduation rates. The graph below shows the average amount by which graduation rates exceed or fall short of expectations separated by HBCU and non HBCU schools (i.e. how much the schools are “beating the odds”).
In addition to this summary information, you can also use the graph below to look at how the graduation rates at specific schools differ from what would be expected based on students’ performance on the SAT.
It is important to remember that SAT scores are only one way that students differ from each other. Students also differ in other ways not picked up by SAT scores (non-cognitive skills, family background, ability to pay tuition, essay writing skills, etc.). In addition, each school has different standards for graduation.[iv] For these reasons, the “beating the odds” measure is best interpreted with caution when attempting to measure school effectiveness. That said, it is certainly an improvement over a comparison of graduation rates that pays no attention to which students attend.
Legitimate public policy questions remain with respect to low graduation rates at colleges and universities. In particular, it is not clear that it is in the best interest of students to provide student loans to those whose pre-college record suggests they are very unlikely to complete their course of study, leaving them with debt but no degree. However, this is not an HBCU-specific issue. Once SAT scores are considered, and schools are compared to others serving similar populations, there is no evidence that HBCUs are less effective than non-HBCUs at getting their students to graduate.
Disclosure: I do research and teach Mathematical Economics at Spelman College, an Atlanta HBCU.
[i] Graduation rates may be useful for other purposes. For example, when seeking to hire candidates, some companies focus on schools with high graduation rates. This choice is different in that companies do not care whether candidates gained their skills at the school or prior to enrolling. They simply care about the total skills accumulated upon graduation. This is quite different from using graduation rates to make claims about college effectiveness.
[ii] In some ways this would still be problematic because it means something different to graduate at one institution vs. another because standards vary from school to schoool.
[iii] This visual was adapted from work done by John Keltz for his Numbers Box blog. Its uses 2013-14 data. Visit the blog for more info on these schools including pell-grant eligibility and individual school profiles.
[iv] The model used to predict scores is a linear model for simplicity. In a small number of schools at the right side of the distribution, this results in predictions that exceed the possible graduation rate of 100%.
