When a college ranking is released or updated, backlash inevitably follows. Usually, colleges who performed poorly lead the charge, but it’s also a reaction against a root issue. Higher education is so diverse that it’s impossible to objectively compare schools.
Which is great! Diversity gives students a wide swath to pursue their education. Universities can achieve different goals. Would the system be better if students were limited to Harvard or the local community college? Yet, as a result, any metric or ranking will be of limited use in comparing some schools. Community colleges, liberal arts colleges, and state research universities don’t expect the same from students, and it is tricky to measure them on an equal footing.
So, when Jordan Weissmann at Slate noted a new return on investment (ROI) metric released by Payscale.com, criticism of using an ROI metric came swiftly. Unsurprisingly, Weissmann found many colleges with a negative ROI; that is, the graduate would be better off if he or she didn’t attend. Weissmann then wrote a rebuttal to the criticism.
On a certain level, not addressing the concept of college leaves those discussions fruitless. College falls into three categories: economic, hedonic, and intellectual. It’s used for job training, but it’s also viewed as a consumption good to enjoy, and a process of self-fulfillment in the pursuit of truth and knowledge.
Payscale’s ROI figure treats college as job training. Weissmann evaluates Payscale’s usefulness with that lens. The best argument against that is not “college isn’t just about getting a job.” It’s to point out the limitations of one metric. In fact, a ROI metric is relatively weak due to data limitations. Colleges resist transparency, especially for information about graduate earnings, and Payscale uses self-reported data. That’s problematic, but more useful than anything legally available at this point until higher education institutions provide the information. Otherwise, we’ll remain standing on the quad surrounded by fog.
I’m not sure Payscale claims to present an all-encompassing ranking; criticisms of their rankings reveal more about the critics than the target.