How Not to Criticize College Rankings

When a college ranking is released or updated, backlash inevitably follows. Usually, colleges who performed poorly lead the charge, but it’s also a reaction against a root issue. Higher education is so diverse that it’s impossible to objectively compare schools.

Which is great! Diversity gives students a wide swath to pursue their education. Universities can achieve different goals. Would the system be better if students were limited to Harvard or the local community college? Yet, as a result, any metric or ranking will be of limited use in comparing some schools. Community colleges, liberal arts colleges, and state research universities don’t expect the same from students, and it is tricky to measure them on an equal footing.

So, when  Jordan Weissmann at Slate noted a new return on investment (ROI) metric released by, criticism of using an ROI metric came swiftly. Unsurprisingly, Weissmann found many colleges with a negative ROI; that is, the graduate would be better off if he or she didn’t attend. Weissmann then wrote a rebuttal to the criticism.

On a certain level, not addressing the concept of college leaves those discussions fruitless. College falls into three categories: economic, hedonic, and intellectual. It’s used for job training, but it’s also viewed as a consumption good to enjoy, and a process of self-fulfillment in the pursuit of truth and knowledge.

Payscale’s ROI figure treats college as job training. Weissmann evaluates Payscale’s usefulness with that lens.  The best argument against that is not “college isn’t just about getting a job.” It’s to point out the limitations of one metric. In fact, a ROI metric is relatively weak due to data limitations. Colleges resist transparency, especially for information about graduate earnings, and Payscale uses self-reported data. That’s problematic, but more useful than anything legally available at this point until higher education institutions provide the information. Otherwise, we’ll remain standing on the quad surrounded by fog.

I’m not sure Payscale claims to present an all-encompassing ranking; criticisms of their rankings reveal more about the critics than the target.


Camilla Johnson

Above: “( College falls into three categories: economic, hedonic, and intellectual. Job training, a consumption good to enjoy, and self-fulfillment pursuing truth and knowledge. )”

The consumption good is apparent; four young years of partying under almost no restrictions.

The intellectual goal is laughable. Supposedly, 18 year olds are saying to themselves “I am curious, and college is the best way of satisfying that curiousity”, at a cost of about $100K counting tuition and lost income.

Job training doesn’t quite capture the desperate desire to prove that one is employable, and the motivation for parents to fund the opportunity for their children. Amazing to me, academics have sold the idea that you don’t really know something unless you have “taken a course” in it.

Schools can give entrance exams covering anything, but businesses are discriminatory if they do the same. Schools can take money for providing semesters of work experience and internships, while businesses are exploitive for paying their “students” some small amount for doing the same things. Schools strongly suggest that college is the way to success. Any complaints get the reply “You attended for your own reasons. We promised nothing.”

The value of college is the degree. That is the proof to the future employer that the student once learned something complicated, even if he has since forgotten. The degree is sold dear. A student must take a minimum number of courses (pay a minimum amount of tuition) before taking exams which show knowledge.

If colleges were in the knowledge business, rather than the degree buisiness, then exams would be administered to anyone regardless of classes purchased. Classes would be optimized to deliver knowledge as a most efficient way to pass the exams, and there would be individual merit badges for exams in each course.

Unfortunately, colleges are taking advantage of their monopoly and charging more than the market will bear, when you count defaulted student loans. Colleges do not disabuse the student of the wildly optimistic idea that their degree will get them $1 million more in lifetime earnings.

The law says that a company cannot give an employment test unless it has been shown to be non-discriminatory in effect, that it doesn’t screen out people of color at a different rate than people of pallor.

So, employers don’t create their own tests or use standardized tests. Companies rely heavily on college degrees to give them some little information about the quality of candidates. Interviewers talk randomly about whatever they want, using personal judgment to decide if the candidate is “a good match”. This is supposed to be less discriminatory!