Slipping down the rankings

A semester into her arts degree at the University of Auckland, Harriet Pudney transferred to Victoria – if you’re to take at face value the most recent QS World University Rankings, a move from one of the top 100 institutions in the world to a much worse one, with a score of 43.1 to Auckland’s 69.8, and an inferior humanities department. But Harriet, now 23, didn’t base her decision on the rankings.

“Auckland definitely has a better reputation, but I don’t think I would’ve had a markedly better experience there,” she says. “I was aware that in moving to Wellington I was taking a bit of a step down in terms of the supposed quality of the university, but I feel like getting the most out of uni is as much about attending class and getting to know your lecturers as it is the ranking of the school itself.”

From the three best-known international rankings, put out by QS, Times Higher Education, and Shanghai Jiao Tong University, you can draw two conclusions about New Zealand’s universities – firstly, that Auckland is the best in the country by some margin, and secondly, that most of them are getting worse year on year.

New Zealand universities' rankings in the last three global tables

	QS '11	QS '12	QS '13	THE '11	THE '12	THE '13	ARWU '11	ARWU '12	ARWU '13
Auckland	82	83	94	173	161	164	201-300	151-200	201-300
Otago	130	133	155	201-225	226-250	226-250	201-300	201-300	201-300
Victoria	237	237	265=	251-275	251-275	276-300	401-500	401-500	401-500
Canterbury	221	238	238	301-350	301-350	301-350	401-500	401-500	401-500
Waikato	357	374	401-410	301-350	301-350	301-350	-	-	-
AUT	-	451-500	471-480	-	-	-	-	-	-
Massey	329=	308	343=	-	-	-	401-500	401-500	401-500
Lincoln	-	-	481-490	-	-	-	-	-	-

But how important you think the rankings are depends on how much you’re prepared to overlook their obvious shortcomings. Even the institutions themselves, most of which are quick to point to the rankings as indicators of excellence when they do well in them, struggle to sum up what they reflect or convey. They’re a measure of excellence, but a flawed one. They influence universities’ strategic plans, but don’t direct them. They matter, but they don’t.

“You can’t ignore them, but you have to view them with some skepticism, because what they’re trying to attach a single number to a university somewhere in the world, made up of hundreds of different subjects, lecturers, staff and academics is multidimensional,” says Roy Crawford, chair of Universities New Zealand and vice-chancellor of Waikato University. “Of course, when one does well at them, you’re very pleased to use that as part of your bragging rights – and if you haven’t done quite so well, you can dismiss them as not being relevant.”

Though the three rankings broadly assess the same measures of university performance –research quality and quantity, international outlook and reputation, graduates’ employment prospects and teaching quality – they’re informed by a variety of sources, and prioritised differently. The QS World University Ranking, for example, refers to its own surveys of academics and employers, as well as to the more typical citations databases and academic journals, to calculate its final scores. (Its World University Rankings by Subject are out today.)

“Since the number of respondents is very high, our reputational surveys are able to pick up tangible differences in how institutions are regarded among the international academic community and, perhaps of paramount relevant to students, among employers,” says Ben Sowter, head of research at QS, via email from London. “Clearly these are useful things for students to be aware of, even if one doesn’t regard them as the most important criteria.”

But the QS ranking is often criticised for its focus on peer review, and in 2010 the Times Higher Education, with which it collaborated from 2004 to 2009, split to publish its own table following concernsthat surveys were too volatile a methodology. Today, the THE’s World University Rankings employ 13 “carefully calibrated” performance indicators to group universities in bands, as opposed to giving them a single figure, based on their teaching, and research volume and influence.

Though the THE ranking was first developed with a trade publication audience in mind, editor Phil Baty says students benefit from them as well. “We find that as we are trusted at the highest level of government around the world, students and their families feel reassured that they are working with a robust and highly respected ranking, based on serious data and important, relevant indicators.”

The Shanghai Jiao Tong University’s Academic Ranking of World Universities is based on six indicators, including, uniquely, Nobel Prizes and Fields Medals won by staff and alumni; like the THE ranking, it also groups universities of comparable quality together. Though it’s often regarded as the most credible ranking, priority is given to the natural sciences, and the focus is on research quality and quantity, as opposed to teaching. (Shanghai Jiao Tong researchers didn’t respond to requests for comment.)

Despite their differences in scope, all three measure what can be measured, and that tends to relate to research. Variations in quality across a university’s departments, or the appeal of the city in which it’s based, or its potential for the fabled ‘student experience’, are lost in the process of attaching a number to it – and AUT University vice-chancellor Derek McCormack sees that as a serious shortcoming of the rankings.

“The intangibles you get from a university experience – that transformative reimagining of yourself, the sense of your own capability building, the desire to go out and do something in the profession you’ve trained for, the discipline you’ve become interested in – are hard to measure, so people don’t measure them.”

As such, they have little to offer domestic students, who have access to plenty of sources of information on where to study. Universities New Zealand’s Roy Crawford says, with all eight New Zealand universities now in QS’ Top 500, there is no bad choice, at least for undergraduate study – only the one that’s right for you. “Students need to make their own minds up if they’re comfortable in the environment,” he says. “That’s the most important thing, much more so than rankings that are established on the far side of the world.”

Often, you find there’s very little difference in terms of the score of universities that might be 10 or 20 places away in the actual ranking chart

“I don’t think a 17- or 18-year-old should look at the rankings and think, ‘This is where I want to be 20 years from now, so this is where I’m going to do my undergraduate study’,” says Daniel Haines, president of the New Zealand Union of Students’ Associations. “At an undergraduate level, they’re all world-class, so anywhere you study is going to be fine.”

But though domestic undergraduate students don’t have much reason to refer to the rankings, they’re as useful a frame of reference as any for those looking at New Zealand universities from overseas.

Victoria University student Taylor Hughson, 21, spent seven weeks in the Middle East over the New Year, and found the University of Auckland was widely understood to be “the best” in the “land of education”. On a trip to a remote beach on the border of Oman and Yemen, a 40-something Dhofari man wearing traditional garb told Taylor that it was his dream to study at Auckland; an English teacher in Najafabad, Iran, seemed “perplexed and a little let down” to discover that Taylor went to Victoria, when it was “not so good” as Auckland.

“I tried to explain that most New Zealanders wouldn’t see this as the case, but he wasn’t convinced,” says Taylor. “Each time it struck me as strange that anyone in Iran or Oman could tell me about our university system, and where each institution stood in international rankings. Some people particularly interested in the subject were, like the English teacher, disappointed that I went to Vic and had robbed them of their chance to meet a real, live Auckland student.”

As prospective international students refer to the rankings before committing to study here, our universities’ slide could have consequences for our multi-billion-dollar “export education” sector. That’s despite the fact, as AUT’s Derek McCormack points out, a drop in the rankings doesn’t necessarily equate to a drop in quality.

“You could be improving but still slipping down the rankings – because more universities are now participating, and they’ve got to fit in somewhere,” he says. “Often, you find there’s very little difference in terms of the score of universities that might be 10 or 20 places away in the actual ranking chart.” (McCormack mentions favourably the QS Stars rating, which gives universities an overall score without pitting them against other institutions.)

Tertiary Education Minister Steven Joyce says, while New Zealand universities’ increasing scores show they’re “making progress”, their drop in the rankings reflect increasing competition in the sector globally. He, and others, point to Asian universities’ investing to be at the cutting-edge of the sector. “It’s not just the New Zealand universities; the Australian and European universities are finding that suddenly there are a lot more Asian universities that are able to compete with them ... and it’s only going to happen more.”

Joyce says the answer is for our universities become more efficient, and to continue to build on their ties to institutions overseas, not least through active recruitment of international students. “Today’s young people are citizens of the Asia-Pacific region much more than they are citizens of New Zealand.”

But the reason those new Asian universities are climbing the ranks is “massive amounts of money”, says the Tertiary Education Union’s Stephen Day. “Our government’s decision to continuously cut, or freeze, tertiary education funding is starting to show in the rankings.” (He acknowledges that the TEU and the Government are “in an ongoing and tedious dispute about whether funding is rising or falling”.) In particular, Day points to New Zealand universities’ deteriorating staff-to-student ratios, which factor into the QS and THE rankings, as being in need of urgent improvement.

This seems, at face value, a straightforward problem: you can keep the same number of students, and hire more staff, or you can have the same number of staff, and fewer students. But neither option is easy for New Zealand universities. Raising entry standards or capping student numbers reduces access to education, but so do fee increases, and universities’ abilities to increase their income are already under considerable constraints.

In the face of fixed or falling government funding, international students’ contribution to both universities’ revenue (close to $309 million in market-price tuition fees in 2012) and GDP ($689 million in 2011) is significant. Their fees also subsidise domestic students’ because universities are able to charge them a market price, while domestic fees are capped at a maximum four per cent increase each year. If international students take the rankings at face value, no university can afford to dismiss them out of hand, no matter how crude a measure they are, or how many interpretations they’re open to.

“If you took out the international students from universities, most of us could barely break even, is the reality of it,” says Stuart McCutcheon, vice-chancellor of the University of Auckland. “If international student numbers fall because the rankings are falling, the sector itself is in considerable trouble.”

Auckland has the biggest international student population in the country, and increased its international fees only in line with inflation last year so as to remain competitive.

After a string of interviews in which the importance of the global rankings and New Zealand universities’ steady decline in them was qualified or downplayed, McCutcheon sticks out. He begins by rattling off a list of stakeholders that refer to the rankings: international students, “very significantly”; domestic students, “increasingly”; alumni; donors; international partner universities; potential staff; governments.

When I respond to his use of “slide” to describe our universities’ movement in the rankings with the party line that other universities are pushing them down, he retorts, “That’s the same thing as a slide, isn’t it?”

Though Auckland University, as the highest-ranked in this country, stands to gain the most from the rankings being taken seriously, it also has the most to lose if the downward trend established over the past five or six years continues. McCutcheon blames this on the funding model for tertiary education, which he says takes a “lowest common denominator approach”.

“Irrespective of the ranking of the uni, the funding system is exactly the same for all universities,” he says. “It doesn’t strike me as a system that in any way rewards excellence, or encourages it.”

But Minister Joyce has no plans to adjust the model in the near future. He says splitting funding for universities between the tuition and research protects the taxpayers’ interests.

“It makes sense to have two funding streams, which you can adjust ... I think it’s important that the money follows the student. I don’t think the system is perfect, and we’ll continue to fine-tune it, but it does ensure that the universities are incentivised to deliver quality research-based learning for their students, and that’s what we want them for.”

Irrespective of the ranking of the uni, the funding system is exactly the same for all universities. It doesn’t strike me as a system that in any way rewards excellence, or encourages it

Like Joyce, McCutcheon agrees that our universities are getting better year on year, but he says that doesn’t diminish the negative impact of the drop in the rankings. “People read them in terms of the trend of the rankings, not in terms of absolute performance or scores,” he says. “If you see New Zealand universities come down the rankings and others go up, you’re going to turn your attention to those other universities as possible places to study, and that’s what puts international education in this country at risk.”

Auckland’s own push for excellence is outlined in its strategic plan to 2020, in which it explicitly sets its sights on ranking among its peer universities in the “Australian Group of Eight, the UK Russell Group and the Canadian U15” an “ambitious” goal that will manifest itself in reduced undergraduate numbers and higher entry standards.

Auckland already raised entry standards above University Entrance for all its courses in 2009.

“We have to reduce our spending in other areas to make sure we can focus as much of our resource as possible on the teaching and research functions,” says McCutcheon. “That means we have to run very lean support mechanisms for our staff, and we are continually having to adjust them to make them leaner still. It means our scholarships aren’t as generous as those in other countries, and we aren’t able to offer as many of them.”

In a tertiary education sector that implicitly prioritises egalitarianism over elitism, Auckland’s goal to be “world-class” presents an access issue: if it’s harder to get into Auckland than it is other universities, what does that mean for disadvantaged students for whom it’s their local institution?

McCutcheon’s aware of the issue, but says underrepresented groups (most significantly, Maori and Pasifika students) won’t be left behind under the new strategic plan. He says, once disadvantaged students have made it through their first year of university, they tend to experience a success rate that’s hard to tell from that of other students for the rest of their qualification; the first step is bridging the gap between their schools and communities, and starting university.

Already, Maori and Pasifika students are able to enrol at Auckland with just University Entrance, a level lower than its guaranteed minimum entry score, as part of its ‘Targeted Admission’ programme; there’s also mentoring and support offered through the Tukana network.

“It’s a matter of overcoming the disadvantages that some of those students – not all of them, by any means; lots and lots of Maori and Pasifika students come in over the guaranteed minimum entry score and fly through – will have experienced,” says McCutcheon. “There is a group of students whose background makes it more challenging for them to demonstrate their innate ability, and we are aiming to help them with that process.”

But, he points out, access issues exist already, at all universities; they’re not uniquely a consequence of raising entry standards. But McCutcheon’s determination to see Auckland move up the global rankings does signal a trend towards differentiation in the sector, which is likely to continue – and without significant investment from the government and the private sector, including students, he can see no alternative.

“One option would be to have a more differentiated university sector, and [another] is to do nothing, and to allow the universities to go down in the rankings,” says McCutcheon. “The option of doing nothing is a pathway to the whole sector being in real trouble – even though it’s the option that by default has been employed in this country for the last 10 years.”

For all their shortcomings, the global rankings have real consequences for the success of our tertiary education sector; whether or not a top-100 university is tangibly better than one with a score of 200-plus is a moot point. If it’s not feasible for all of our universities to keep pace with those in Asia, if not other parts of Oceania, the questions becomes whether we’d be happy for just one or two to do so, even if it’s at the expense of the others.