University rankings and altmetrics

I was on holiday last week, and we drove down to Victoria for a few days. On the way there I listened to the radio and someone (I forget who) was talking about the THES rankings and making the very good point that individually the various ranking tables don’t mean a lot as they all use different algorithms and tray and measure different things. but that if a university scores consistently high in a number of these tables it suggests that in some way it is better than one with either inconsistent scores or consistently low scores.

Better here means that its is good at teaching, good at research, and is effective in promoting this.

So with altmetrics, impact rankings and the rest. Individually the various scores don’t mean a lot, but collectively they are an indicator of engagement. I’ll say engagement because this is still a nebulous topic. There are people who publish highly cited research but don’t promote it. Typically these people are well known in their discipline. Then there are those who communicate well about their field and have an impact through teaching, through social media, and the rest. And of course there’s some people who are somewhere in between.

Like ranking tables, high scores are good. However before dismissing inconsistent scores (high say on social media, low on research impact) we need to actually ask a very difficult question: What are we trying to measure, and how will we know we’ve measured it ?

The first part is probably relatively simple to answer, the second one rather less so, as we need to decide on what we will accept as evidence and what it tells us about engagement …

Written with StackEdit.