I was in Athens this past June, at an EU-sponsored conference on rankings, which included a very intriguing discussion about the use of reputation indicators that I thought I would share with you.
Not all rankings have reputational indicators; the Shanghai (ARWU) rankings, for instance, eschew them completely. But QS and Times Higher Education (THE) rankings both weight them pretty highly (50% for QS, 35% for THE). But this data isn’t entirely transparent. THE, who release their World University Rankings tomorrow, hides the actual reputational survey results for teaching and research by combining each of them with some other indicators (THE has 13 indicators, but it only shows 5 composite scores). The reasons for doing this are largely commercial; if, each September, THE actually showed all the results individually, they wouldn’t be able to reassemble the indicators in a different way to have an entirely separate “Reputation Rankings” release six months later (with concomitant advertising and event sales) using exactly the same data. Also, its data collection partner, Thomson Reuters, wouldn’t be able to sell the data back to institutions as part of its Global Institutional Profiles Project.
Now, I get it, rankers have to cover their (often substantial) costs somehow, and this re-sale of hidden data is one way to do it (disclosure: we at HESA did this with our Measuring Academic Research in Canada ranking. But given the impact that rankings have for universities, there is an obligation to get this data right. And the problem is that neither QS nor THE publish enough information about their reputation survey to make a real judgement about the quality of their data – and in particular about the reliability of the “reputation” voting. Read the whole article.
This article was originally published on the Higher Education Strategy Associates website.
