UNIVERSITY OF AUCKLAND AND VICTORIA UNIVERSITY OF WELLINGTON
[2004] NZHC 1065
•23 March 2004
IN THE HIGH COURT OF NEW ZEALAND AUCKLAND REGISTRY
CIV2004-404-1304
UNDER Part 1 of the Judicature Amendment Act 1972 IN THE MATTER OF
Part 13A of the Education Act 1989
BETWEEN
UNIVERSITY OF AUCKLAND
First Applicant
AND
VICTORIA UNIVERSITY OF WELLINGTON
Second Applicant
AND
TERTIARY EDUCATION COMMISSION
Respondent
Hearing:
23 March 2004
Appearances: Julian Miles QC with Adam Ross and Katherine Anderson for applicants
Les J Taylor for respondent Judgment: 5 April 2004
JUDGMENT OF WILLIAMS J
Solicitors:
Chapman Tripp, P O Box 2206 Auckland, for Applicants
Minter Ellison Rudd Watts, P O Box 2793 Wellington, for respondent
UNIVERSITY OF AUCKLAND And Anor V TERTIARY EDUCATION COMMISSION HC AK CIV2004- 404-1304 [5 April 2004]
SUMMARY
Paragraph
Issues [1] Background
[4] (1)
To early 2004:
[4]
(2)
March 2004: Universities learn of Appendix D
[17]
(3) Were the Universities forewarned the International Comparison would be undertaken? [20]
The International Comparison [25]
Grounds of Judicial Review Application and Submissions [40]
(1)Is the International Comparison amenable to
Judicial Review [40]
(2)Substantive Ground of Review [55]
(a)Submissions [55]
(b)Discussion [66]
Result [84]
-----
Issues
[1] The respondent Commission1 is on the point of publishing a report on Performance Based Research Funding2 which evaluates research carried out by all researchers in New Zealand Tertiary Education Organisations3. The PBRF Report currently contains an appendix, Appendix D, and four paragraphs, paras 24-26 and 159, containing an International Comparison ranking the research performance of TEOs against that of British universities.
[2] The applicants, the University of Auckland and Victoria University of Wellington, seek orders by way of judicial review that TEC’s decision to publish the International Comparison is unlawful in an administrative law sense and should be set aside or TEC should be directed to reconsider whether to include the International Comparison in the PBRF Report.
[3] The PBRF Report was initially due for public release on 23 March 2004. The universities issued these proceedings on 17 March and applied ex parte for interim orders barring the release of Appendix D and the accompanying paragraphs. That application was granted later that day after a hearing in which Mr Taylor, counsel for TEC, participated by telephone. Timetable orders were made, a further conference was held on 19 March, TEC applied to rescind the interim orders and, with the co- operation of counsel their clients and the Court, the substantive hearing of the universities’ application for interim orders was able to be arranged for 23 March.
Background
(1)To early 2004:
[4] Until the late 1980s research and tertiary education in New Zealand were almost entirely State-funded, the recipients being Departments of State, the universities and the polytechnics. However, the bouleversement which transformed
1 “TEC”
2 “PBRF”
3 “TEO”
the State and much of the New Zealand economy in the late 1980s included both research and tertiary education.
[5] In research, Departments of State became Crown Research Institutes.4 State research funds were channelled through such avenues as the Foundation for Research Science and Technology and the Marsden Fund. Access to those funds, always less than sought, became contestable between the CRIs and the TEOs. Keen competition grew up for other research money, sourced within New Zealand and overseas.
[6] State-funding for TEOs also underwent a transformation. Funding was paid on an Equivalent Full-Time Student5 basis. That declined in real terms over time. As a result, student fees increased considerably.
[7] So did student numbers. Funding on the basis of EFTS and the consequent competition between TEOs to increase enrolments had the beneficial effect of raising rates of New Zealanders’ participation in tertiary education from their previous low levels to about the OECD average but, despite the best efforts of TEOs and their staff, the emphasis on quantity inevitably compromised quality.
[8] By the late 1990s all involved concluded the system of funding New Zealand research must change, not least to contribute to the aims of the “Knowledge Economy”.
[9] As a result, in April 2000 Government established the Tertiary Education Advisory Commission6 with the broad aim of identifying ways in which New Zealand could develop a more co-operative and collaborative tertiary sector. TEAC’s report of November 2001, “Shaping the Funding Framework”, recommended the introduction of PBRF for TEO research.
[10] In May 2002 the Ministry of Education released its “Tertiary Education Strategy 2002/07” and in July established the PBRF Working Group to advise the
4 “CRI”.
5 “EFTS”.
6 “TEAC”
Ministry and TEAC on the detailed design and implementation of PBRF for TEOs. That document incorporated Cabinet directives designed to encourage research excellence in this country.
[11] Over the period up to June 2002, Prof Boston, an academic with a lengthy interest in tertiary education policy, had been engaged by TEAC and the Ministry of Education to analyse the United Kingdom equivalent of PBRF which had been operating since 1986 and was the first of its kind in the world. It involved Research Assessment Exercises7 where expert panels assess the quality of research produced by academic staff in UK TEOs. In June 2002 Prof Boston completed a report entitled “Designing a Performance-Based Research Fund for New Zealand”.
[12] After the PBRF Working Group was established in July 2002 to advise the Ministry and TEAC on the detailed design and implementation of PBRF for TEOs it conferred and consulted widely in accordance with “Shaping the Funding Framework” and the “Tertiary Education Strategy”. In its work, the PBRF Working Group considered a number of overseas models but, according to Prof Conder, its Chair, ultimately adopted a model unique to this country. In December 2002 it published its report “Investing in Excellence”. Its recommendations, too, were endorsed by Cabinet. As Prof Conder describes it, “Investing in Excellence” set out the following principles :
… the PBRF should be guided by the following principles:
19.1Comprehensiveness: the PBRF should appropriately measure the quality of the full range of original investigative activity that occurs within the sector, regardless of its type, form, or place of output;
19.2Respect for academic traditions: the PBRF should operate in a manner that is consistent with academic freedom and institution autonomy;
19.3Consistency: evaluations of quality made through the PBRF should be consistent across the different subject areas and in the calibration of quality ratings against international standards of excellence;
19.4Continuity: changes to the PBRF process should only be made where they can bring demonstrable improvements that outweigh the cost of implementing them;
7 “RAE”.
19.5Differentiation: The PBRF should allow stakeholders and the government to differentiate between providers and their units on the basis of their relative quality;
19.6Credibility: the methodology, format and processes employed in the PBRF must be credible to those being assessed;
19.7Efficiency: administrative and compliance costs should be kept to the minimum consistent with a robust and credible process;
19.8Transparency: decisions and decision-making processes must be explained openly, except where there is a need to preserve confidentiality and privacy;
19.9Complementarity: the PBRF should be integrated with new and existing policies, such as charters and profiles, and quality assurance systems for degrees and degree providers; and
19.10Cultural inclusiveness: the PRBF should reflect the bicultural nature of New Zealand and the special role and status of the Treaty of Waitangi, and should appropriately reflect and include the full diversity of New Zealand’s population.
[13] Because of differences between TEOs, the Working Group recommended two levels of reporting, first, against the relevant subject area and then by nominating an academic unit against which scores would be reported. Other information to be reported included average scores for the relevant subject area across all providers, proportions of eligible staff to receive different category ratings, numbers of student places at under-graduate and post-graduate levels, both taught and researched, post-graduate completions, and numbers of eligible staff and proportions of academic staff involved in research and teaching. The scoring system adopted by TEAC was on a scale of 0-7 for each of Research Output, Peer Esteem and Contribution to the Research Environment, with the scores then weighted A B C or R from research of the highest excellence to research unrecognised by the PBRF.
[14] Enter TEC. It was set up pursuant to the Education (Tertiary Reform) Amendment Act 2002 which came into force on 1 January 2003. The amendment inserted a new objects provision for tertiary education (s159AAA) which included high quality learning and research as an objective and enacted a new Part 13A relating to TEC, the functions of which include conducting “applied policy and programme research, monitoring and evaluation” monitoring TEOs’ performance against their profile to assess their overall achievement in relation to the Tertiary Education Strategies and Priorities, providing information and other tertiary-related
services to the Crown and allocating funding to TEOs (ss159F, 159ZA). TEC is required to “work closely with” TEOs (s159G) and its members individually must act in good faith and with reasonable care, diligence and skill (Schedule 13A cl 13(1)).
[15] Immediately on appointment, TEC began designing the final PBRF. Its April 2003 consultation paper “Guidance on the Assignment of Quality Categories” (paras 9, 13) made the point that parts of PBRF were developed from RAE and discussed differences and similarities in the systems. In discussing principles for the reporting framework in its May 2003 consultation paper “Reporting of PBRF Results” the public reporting of PBRF was suggested to be in accordance with a number of principles including (para 14 p6) :
“Facilitating, where possible, international comparability (eg the capacity to compare New Zealand’s performance with that of Britain under the RAE)”
That text was repeated in Part 6 on PBRF reporting and as one of eleven principles (p260) in TEC’s Inaugural PBRF Guide issued in July 2003. That part of the report also detailed the wealth of material which will be publicly reported. But no other passage in Part 6 appears to raise any aspect of International Comparison.
[16] Since the guide was issued, TEOs and their research staff have complied with the process outlined, their research has been submitted in Evidence Portfolios and assessed by peer review panels and moderated and is now ready for distribution to TEOs, panels and each academic unit in each discipline. It is that final stage of the process which has been delayed by this case. Though the universities were content for TEC to publish everything else in the PBRF Report other than Appendix D and the four associated paragraphs, TEC declined for the reasons mentioned elsewhere in this judgment.
(2) March 2004: Universities learn of Appendix D
[17] At a meeting of the New Zealand Vice-Chancellors Committee8 Standing Committee on Research on 4 March 2004, Prof Boston told those present he had
8 “NZVCC”.
been dismayed at the PBRF results and that, as Dr Barnes, a member of the NZVCC Committee put it, “direct comparisons between the PBRF and RAE “would look very bad for New Zealand”. As a result TEC had decided to publish its own International Comparison. NZVCC representatives present, including Dr McCutcheon, its Chair and Vice-Chancellor of Victoria, immediately protested, telling the TEC representatives there had been no prior notification to TEOs that any International Comparison would be undertaken and asserting the validity of any undertaken by TEC was highly dubious given differences in assessment methods and reporting processes. Prof Boston told them they had tried as much as possible to avoid the problems and had included caveats against drawing conclusions from the International Comparison because of assumptions made about participation rates in the British system and the degree of overlap between the two. NZVCC representatives took the view the media and many members of the public would be oblivious to the finer points of the analysis and would simply report that TEO research in New Zealand was below international standards. Any report therefore would be misleading and, in the NZVCC’s view, risked degrading the international reputation of New Zealand universities and their degrees and thus their ability to attract high calibre international students, especially at post-graduate level, high quality staff with significant research publication records, scarce private and off- shore research funding, and maintain and improve their research standards. They suggested the entire deletion of Appendix D.
[18] On 8 March 2004 a meeting was convened specifically to try to persuade TEC to delete Appendix D. TEC representatives declined to withdraw the report because it had already been officially circulated to Cabinet and Government bodies. A TEC media briefing later that day mentioned the International Comparison.
[19]The universities first saw a copy of Appendix D on 15 March 2004.
(3) Were the Universities forewarned the International Comparison would be undertaken?
[20]Prof Conder said :
“At no time did the Working Group ever recommend or anticipate that the data provided under the PBRF assessment process should be used in order to provide a comparison with data obtained under different models overseas.”
[21]Prof Boston disagreed. He said :
“It was and always has been inevitable that New Zealand PBRF results would be compared to overseas results, particularly the UK RAE results … It would be true to say that the precise nature of the analysis included in Appendix D was not specifically contemplated. However, use of the PBRF data for this type of comparison was and is inevitable, obvious and desirable.
Right from the outset, it was envisaged that the PBRF would and should be used to assist with the international benchmarking of New Zealand tertiary education organisations.”
[22] Prof Boston also said the universities should have known the International Comparison would be undertaken. He supported that view by reference to passages from the cited publications. “Shaping the Funding Framework” includes discussion (p95) of the PBRF model having the “potential for use as a means of international benchmarking”. “Investing in Excellence” mentions the need for peer review panels who have the knowledge and expertise to make reliable judgments about quality “against international standards” across the range of disciplines (p14 para 51) and, as noted above, its consistency principle (para 19.3) speaks of its evaluations being rated “against international standards of excellence”. Prof Boston made the point that once New Zealand decided to assess research quality through peer review rather than performance indicators it was “virtually inevitable that the PBRF would draw heavily on the UK RAE” as being the most widely recognised such system and that PBRF drew on RAE for the definition of research, the nature of research outputs, assessment criteria, panel working methods, staff eligibility criteria, assessment periods, auditing processes and reporting.
[23] Prof Boston, supported by Dr Smith, a member of TEC and Deputy Vice- Chancellor (Research Enterprise and International) at the University of Otago, said comparisons were first made between RAE and the emerging PBRF results from mid-November 2003 when provisional PBRF results were rather lower than expected. However, since crude comparisons of raw data would be misleading because the assessment criteria and methodologies differed, TEC undertook a more sophisticated analysis by making assumptions and adjustments to take account of the
differences. Once conducted, the analysis was distributed to the PBRF Steering and Evaluation Advisory Groups, Chairs of the Peer Review and Moderation Panels and Ministry staff. None challenged the methodology or interpretation of the results. One, Mr Bekhradnia, Director of the Higher Education Policy Institute at Oxford University, said he thought the assumptions and adjustments made were reasonable and the methodology “an imaginative and ingenious way of enabling meaningful comparisons to be carried out”.
[24] Both Prof Boston and Dr Smith say that comparison between research performance of TEOs and that of universities and research organisations world-wide is and commonplace both by academics and others interested in the area. Far better, they say, for TEC to publish a detailed analysis setting out the assumptions on which the International Comparison is based and highlighting difficulties in making it and the differences in the systems than leave it for comparisons which will inevitably be made to be on a much less measured and informed basis. As Prof Boston put it :
While there are difficulties in comparing the results of the UK RAE and PBRF, these difficulties are not insuperable. Accordingly, it was decided to undertake a more sophisticated analysis of the data, by making some reasonable assumptions and adjustments which took into account the differences in the two assessment systems. By conducting such an analysis, I believe it is possible to provide a much fairer and more accurate comparison between the research quality of New Zealand and British tertiary education organisations. Admittedly no comparison of this nature can be undertaken with absolute precision, but if undertaken properly it will be significantly more accurate and useful than a simplistic comparison of the raw data.
By setting out the methodology for the International Comparison in detail, it is possible for readers of the PBRF Report to understand the basis for the comparison, and to assess for themselves whether the adjustments and assumptions are reasonable and appropriate.
The International Comparison
[25]So what do Appendix D and the disputed paragraphs contain?
[26] The paragraphs under challenge acknowledge that comparing the PBRF “and broadly similar assessments of research quality in other countries, such as Britain, is difficult because of significant methodological differences” (para 25) but then go on to discuss the data “to the extent that reliable comparisons are possible”. And
although it does say (para 159) that there are “many difficulties” in making the comparison, Appendix D sets out the New Zealand and RAE rankings “depending upon the assumptions made”.
[27] Appendix D itself (in the form put in evidence)9 contains a list of 59 British Universities preceded by 15 pages of detailed and densely-argued material comparing the performance of TEOs with them. The list is followed by three columns of figures which appear to be numerical translations of the graph (Fig.D-1) which shows the “Derived and Actual-Weighted Average Quality Score for Pre-1992 British Universities and New Zealand Universities” at conversion rates between the two of 100%, 75% and 50%.
[28] The introduction to Appendix D discusses various ways of assessing the relative performance of TEOs including comparisons with scientific productivity, research cost-effectiveness and citation rates in learned journals. It mentions other comparators used by institutions such as Shanghai Jiao Tong University’s “Best 500” ranking and deficiencies in that approach and comparing similar kinds of assessment done elsewhere in the world though noting “accurate and meaningful comparisons are difficult” because different systems use different indicators and some comparisons are not public or not published in comparable form.
[29] After concluding that RAE is the best known assessment, Appendix D continues to say that comparison (para 9) “poses many difficulties not least because of methodological differences between the two models”, differences it then goes on to discuss in detail. Different systems are used, different units of assessment, different information which goes to expert panels, different quality standards and different rating categories and inter-panel moderation. After noting the use in the PBRF of some criteria from RAE, Appendix D notes that different quality standards are likely to result (para 15) in the proportions of New Zealand staff scoring highly being smaller than those in British institutions so there is a “potentially significant mis-match between the two sets of quality thresholds”.
9 The version of Appendix D put in evidence may have been incomplete: certainly the Tables contained in the text were all but illegible.
[30] Further, in New Zealand, each individual staff member was assessed whereas such is not the case in Britain. Further again, under PBRF, TEOs may participate though they only have a small research staff whereas they would not qualify in Britain. There are TEOs in New Zealand – theological and Bible colleges are examples – which would not participate in RAE. The criteria for staff participation are broader in this country. That is (para 18) “likely to have increased the proportion of staff with lower Quality Categories” and to “reduce the quality scores of TEOs relative to what they would have been” under RAE. British institutions can exclude staff from participation in RAE but in New Zealand all eligible staff must be assessed. And the RAE results are reported using rating categories, comparison with the PBRF being “by no means straightforward” (para 20).
[31] Appendix D then moves to comparing two means of evaluating the assessments. Applying the RAE scale required assumptions that the New Zealand A B and C ratings equate to the British levels of international excellence, national excellence or falls short of national excellence. The New Zealand requirement for all staff to be rated highly means (para 23) a “higher threshold than typically applies under the RAE”. Not even the best British universities include all eligible staff. Deleting the lower performing categories in New Zealand and repeating the assessment still required (para 26) “considerable care in drawing conclusions”. Reversing the procedure and applying the New Zealand scale to British staff required assumptions about the relationship between the Quality Categories and the RAE standards of research excellence and the proportion of staff in the seven RAE categories that fall into the three PBRF categories of international, national and sub- national excellence even though (para 29) “there is no automatic 1-1 correspondence”. That led Appendix D to use conversion rates of 100%, 75% and 50% between the two sets of quality measures, that is to say to assume that those rated internationally excellent, nationally excellent or sub-nationally excellent in RAE would be rated A, B and C under PBRF and those not submitted for assessment would be rated R. Because RAE categories are broad and cover staff in two or three categories of research excellence, assumptions were then made as to the percentages of staff in each rating category that fell into the three research excellence categories. When all those assumptions are taken into account Appendix D says there are two types of comparison between TEOs and the British institutions. The first compares
the proportion of A-rated staff in both and the second comparing quality scores on an EFTS basis in both. Appendix D says (para 37) “it is not clear what conversion rates might be most appropriate”.
[32] Despite all those qualifications the report reaches the view that a comparison remains possible between world-class researchers in the two countries. That leads to the graph which plots the quality scores for the 59 pre-1992 British universities based on the results of the 2001 RAE with the quality scores of the eight New Zealand universities last year across the three conversion rates (para 42 Fig.D (1)). The conclusion includes the following (para 49) :
… the preceding discussion highlights that comparing the results of the 2003 Quality Evaluation and research assessments in other countries must be handled with very great care. In relation to the British RAE, for instance, it cannot simply be assumed that there is a close correspondence between “international excellence” in Britain and an “A” rating in New Zealand – notwithstanding the use of broadly similar descriptions for these particular quality standards.
[33] Both Prof Conder and Dr Barnes – supported by Prof McCutcheon and Vice- Chancellor of the University of Auckland, Dr Hood – take the view the International Comparison is fundamentally flawed and inappropriate. Both say that amongst the major differences between the two systems is that RAE grades the performance of academic units and, within that, each participating institution can decide which staff members to include and, within that, each staff member included can select up to four items of their best research. Under RAE an academic department can be awarded the maximum score if more than half the research activity submitted achieves “attainable levels of international excellence” and “attainable levels of national excellence” for the remainder irrespective of the number or proportion of academic staff whose work is submitted. Under PBRF every eligible staff member is expected to be assessed over every research output for the past six years with departmental grades calculated as averages of every individual staff grade, not just the high achievers. An academic unit can only be awarded the maximum possible score under PBRF when all its academic staff are assessed as world-class researchers. If a TEO has a high proportion of staff not engaged in research, its PBRF score must decline. PBRF’s research quality criteria are said to be much stricter than those of RAE and RAE has 65 subject area panels against PBRF’s 12.
RAE’s assessment also includes research funding, environment and strategy and other issues not part of PBRF. Both witnesses made the point that not only were TEC’s assumptions in Appendix D not circulated and available for discussion during the consultative peer review phase which led to the creation of PBRF, they have not been tested or justified in any other way. That, they say, particularly applies to the conversion methodology which is undermined by the differences earlier mentioned, including those of British institutions’ choices.
[34] Both Vice-Chancellors made the point that none of the TEOs were asked to consent to the use for the International Comparison of the large amounts of information they supplied to TEC for PBRF purposes. They both say had they been aware of the use to which TEC now proposes to put that information, their universities may not have participated in the PBRF or may have participated differently. At the least, they said TEC should have told the TEOs in advance they intended to conduct the International Comparison and consulted with them as to whether it was possible or desirable and, if so, pursuant to what methodology. They refute TEC’s evidence that it is providing a much better-reasoned analysis for comparison in advance of the widespread and uninformed comparisons certain to follow publication of the PBRF Report even without Appendix D. They say the analysis conducted by TEC as an organ of the New Zealand Government would have much greater weight than any other. It will have the imprimatur of an official comparison when, because of the difficulties and differences earlier mentioned, the International Comparison is undermined and flawed. And they assert – as seems likely – that most readers of Appendix D will look only at the comparison tables. The chance is remote that most media or other commentators, however skilled, will carefully read the preceding text, locate and weigh the TEC caveats, appreciate the effect of the assumptions and thus be able to conduct a knowledgeable assessment of the difficulties inherent in the comparison and thus construe it accurately.
[35] It may be unfortunate that the two tables in Appendix D representing the graphical and numerical result of TEC’s analysis could not be prefaced with a summary of the differences, caveats and assumptions discussed at length elsewhere in the Appendix. But doubtless the universities’ response would be that TEC should never have attempted the International Comparison because it gave them no
forewarning that their sensitive data was to be used for that purpose and the methodology is so flawed and the summary would need to be so lengthy that it would do no more than expose the unwisdom of publishing the International Comparison.
[36] All the universities’ witnesses take the view the deficiencies of the International Comparison and the methods by which it was prepared are such that all the universities will suffer greatly in their competition for students, staff and research funding All of that, they say, would run counter to the Tertiary Education Strategy adopted by Government.
[37] In much of that, the universities are supported by Sir Graeme Davies, Vice- Chancellor of the University of London but a New Zealander whose first doctorate was from the University of Auckland. In addition to a distinguished academic career he was Chief Executive of the Universities Funding Council (1991-93), Chief Executive of the Polytechnics and Colleges Funding Council (1992-93) and Chief Executive of the Higher Education Funding Council for England (1992-95). He confirmed the significant difficulties in comparing RAE and PBRF for the reasons outlined by Prof Conder and Dr Barnes and expressed doubt as to the justification for the assumptions in Appendix D. Alignment of the methodologies between the two systems would have been desirable before Appendix D was prepared he said and he is unaware of any instances of formal comparisons by Government agencies from different jurisdictions of research quality assessments. He concluded :
The English-speaking academic world, despite the large number of participants on an individual basis, is very well connected and intensely competitive. There is a constant struggle between universities to attract the best and the brightest, both in terms of students and faculty. They also compete hard for the scarce private funding that is available to pay for the highest levels of research around the world. Universities and their regulators pay close attention to what their colleagues say about the strength and breadth of their tertiary research capabilities. I am sure the … TEC report will be closely read by other institutional players around the world. The Appendix D would be of very real interest in part at least, because it could provide information for the competitions which I have described above.
While I would not want to overstate Appendix D’s significance, it must be said that the proposed opinions are unique in that they are statements of an official government body concerning the relative quality of a country’s tertiary level of research output as compared with another nations. There
must be a risk in these circumstances of adverse consequences to the applicants which are hard to quantify but are nevertheless real. …
In my opinion the current circumstances suggest that it would be best not to publish Appendix D. If such an enterprise were attempted in the UK, I would expect the relevant tertiary institutions to be consulted about the design and methodology of any comparison in order to seek to ensure that the process was both comprehensive and inclusive. A consensus (with some luck) would emerge. Then the work could be done in an orderly way and the results would then be if not welcomed, at least accepted by the participants.
[38] Prof Boston joined issue with each criticism of Appendix D. For instance, while he agreed the two systems were not identical, he said they draw on very similar types of information and both involve an evaluation of the research outputs of individual academic staff. He defended Appendix D’s acknowledgement of the differences and assumptions and asserted that far from the latter not being tested or justified they were explicitly identified and discussed in the Appendix. Eliminating lower-performing staff from the comparison compensated for the British institutions’ ability to choose the research and individuals they submit. But those strategic choices still meant they included those of international and national excellence and were, he suggested, unlikely to affect the result. “Merely because a comparison cannot be exact”, he said, “does not mean that no comparison can be undertaken.” Prof Boston pointed to Appendix D opting for a conversion rate in all probability towards the lower end of the 50%-100% range. That produced a more favourable comparison for TEOs. He also disagreed with the suggestion that, to be valid, the International Comparison required as much design effort, peer review and transparency as the domestic evaluation, pointing to the amount of effort required for the design and implementation of the PBRF and the submission, without adverse response, of the draft report including Appendix D to many skilled in the area. Meeting Dr Barnes’ point about the averaging of individual grades, Prof Boston said the quality of individual researchers assessed in both systems and the data reported under RAE enables the estimation of broad proportions of staff in each academic unit who fall into the respective categories. He said staff are not required to list all research under PBRF but may list up to 50 outputs with four nominated carrying the greatest weight. And finally, on the practicability of removal, Prof Boston pointed to references to the International Comparison in the preface, in six pages of the report and in at least six peer review panel and moderation reports. Re-drafting to exclude the International Comparison would be reasonably significant.
[39]Finally, as to the assertions of likely damage to TEOs, Prof Boston said :
Once the PBRF results have been published, one of the obvious and likely things to happen is that interested observers such as the media will attempt to make comparisons between the New Zealand results and UK results. There is a substantial risk that such comparison will be simplistic and ill-informed, and suggest that New Zealand institutions compare rather badly against UK institutions. A more measured, and sophisticated analysis – as conducted by the Commission – indicates that New Zealand institutions actually performed much better than the raw data appears to suggest. For this reason, the publication of the Commission’s International Comparison will protect the reputation of New Zealand institutions. The fact that the New Zealand universities compare moderately well with their relevant British counterparts is, in my opinion, preferable to uninformed comparisons which would suggest that New Zealand universities compare rather badly.
Grounds of Judicial Review Application and Submissions
(1)Is the International Comparison Amenable to Judicial Review?
[40] The universities’ amended claim asserts that TEC's decisions to include the International Comparison in the PBRF Report and make it publicly available amounted to exercise of a statutory power of decision. However, the grounds of TEC’s application to rescind the interim orders included an assertion that publication of the PBRF Report did not constitute exercise of a statutory power and the Court accordingly had no jurisdiction to make the interim orders under the Judicature Amendment Act 1972 s8.
[41] Preparation and publication of Appendix D, Mr Taylor submitted, could not come within the definition of “statutory power of decision” in the 1972 Amendment as being a power or right conferred under an Act to make a decision affecting the “rights, powers, privileges, immunities, duties or liabilities” of the universities or the “eligibility of any person to receive or to continue to receive a benefit or licence”. He submitted the International Comparison was no more than an opinion contrasting the New Zealand and United Kingdom research achievements. He submitted those decisions deprived no one of rights and conferred no benefits and were thus not the exercise of statutory power, relying on R v Sloan [1990] 1 NZLR 474, Whale Watch Kaikoura Ltd v Transport Accident Investigation Commission [1997] 3 NZLR 55 and New Zealand Stock Exchange v Listed Companies Association Inc [1984] 1 NZLR 699.
[42] Sloan was an unsuccessful application for judicial review of letters written by inspectors of the Department of Internal Affairs expressing opinions concerning the legality of Mr Sloan’s operation of his gaming machines. The reasons for dismissal of the application included (at 478-479) that the inspectors had no power to decide on the lawfulness of the operation but merely to form an opinion for decision on a prosecution by the Court. Their opinion did not affect Mr Sloan’s rights, powers, privileges, immunities, duties or liabilities. In addition, the inspectors’ powers were no more than administrative or procedural and, thirdly, the Court was asked to assess the merits of the inspectors’ view of legality not merely the process, a decision beyond the 1972 Amendment.
[43] Whale Watch concerned judicial review proceedings brought against the Transport Accident Investigation Commission’s proposal to report its findings and conclusions into an accident. The issue was held non-justitiable on the grounds, first, that the Commission had a statutory duty to publish its report into its investigations and, secondly, because its findings were not determinative of the lawfulness of the conduct but were merely opinions. A subsidiary ground was that the attack was on the merits of the conclusions rather than the process by which they were reached.
[44] In the Stock Exchange case, it was held a decision by the Exchange to suspend a company’s listing was not the exercise of a statutory power of decision. The Court of Appeal held (at 706) it was fallacious to equate the conduct of a statutory body within its statutory sphere with the exercise of a statutory power under the 1972 Amendment.
[45] For the universities, however, Mr Miles QC first pointed to the lack of challenge by TEC that interim orders were necessary for the purpose of preserving the applicants’ position by prohibiting TEC from taking any further action in exercise of a statutory power (Judicature Amendment Act 1972 s8(1)). He then submitted the preferable view on what amounts to exercise of statutory power is now to be found in Mercury Energy Ltd v Electricity Corporation of New Zealand Ltd [1994] 2 NZLR 385, Peters v Davison [1999] 2 NZLR 164 and Royal Australasian College of Surgeons v Phipps [1999] 3 NZLR 1.
[46] In Mercury Energy one of the issues in the Privy Council was whether decisions of the Electricity Corporation were amenable to judicial review. Their Lordships observed (at 388) :
Judicial review was a judicial invention to secure that decisions are made by the executive or by a public body according to law even if the decision does not otherwise involve an actionable wrong. A state-owned enterprise is registered under the Companies Act 1955, it is accountable to its shareholders and carries on commercial activities. The power of the Corporation to determine the contractual arrangements was derived from contract and not from statute. The Court of Appeal concluded that the decision taken by the Corporation to terminate the contractual arrangements by the notice dated 27 March 1992 was no different from any other commercial decision taken by a private body and was not liable to be quashed by judicial review under the Act of 1972. …
A state-owned enterprise is a public body; its shares are held by ministers who are responsible to the House of Representatives and accountable to the electorate. the Corporation carries on its business in the interests of the public. Decisions made in the public interest by the Corporation, a body established by statute, may adversely affect the rights and liabilities of private individuals without affording them any redress. Their Lordships take the view that in these circumstances the decisions of the Corporation are in principle amenable to judicial review both under the Act of 1972 as amended and under the common law.
It does not follow that Mercury is entitled to proceed with its claim for judicial review in the present case. Judicial review involves interference by the Court with a decision made by a person or body empowered by Parliament or the governing law to reach that decision in the public interest. A litigant may only invoke interference by the Court with such a decision if the litigant pleads plausible allegations which, if substantiated at the trial, will demonstrate that the decision was not reached in accordance with law.
[47] Peters was a judicial review alleging error of law in the report of a Commission of Inquiry. Richardson P, Henry and Keith JJ held (at 189) :
To hold that the public interest may require judicial review of the report of commissions of inquiry (their special nature notwithstanding) upon certain of the grounds of judicial review, but then to hold that the Court is never able outside those grounds to intervene for material error of law, an established ground of judicial review, is not in our view sound in principle or justifiable in the public interest. It does not recognise the importance of commissions of inquiry in our constitutional and governmental system. It does not recognise the practical utility that a declaration of error of law may have. It would deny the development of the law in Anisminic and later cases.
If the alleged error of law materially affects a matter of substance relating to a finding on one of the terms of reference it is in general reviewable. The reason for exercising the power of review is the stronger if that error damages the reputation of any person directly concerned in the inquiry.
And Thomas J held (at 201) :
I accept that the Court has jurisdiction to review the report of a commission of inquiry where the commission has made a material error of law in the course of performing its function. To deny the Court that jurisdiction would be to ignore seasoned developments in public and administrative law.
Counsel for the respondent articulated the Court’s jurisdiction far too narrowly. They accepted that a report of a commission of inquiry could be reviewed on two grounds; one, where the commission has acted outside its terms of reference and, two, where the principles of natural justice have not been complied with. In support of these propositions counsel referred to various dicta in Re Erebus Royal Commission (No 2) [1981] 1 NZLR 618 and Re Royal Commission on Thomas Case. There has been, it was argued, no subsequent expansion of these limited grounds of review.
To my mind, this argument reflects an undesirable approach to the application of the doctrine of precedent. Each of these cases related to situations where it was alleged that the commission of inquiry had either exceeded its jurisdiction or breached the principles of natural justice. Not unexpectedly, the dicta of the Courts in those cases relate to those circumstances. Reference to the underlying principle, however, puts paid to any argument based on the literal wording of such dicta. Both acting outside jurisdiction and acting in breach of the principles of natural justice predicate a failure by the commission of inquiry to properly exercise its jurisdiction. Both are examples of the doctrine of ultra vires. It is equally a question of jurisdiction for the commission to fail to exercise its jurisdiction.
[48] Phipps concerned an application for judicial review of the College of Surgeons’ report concerning Mr Phipps’ practice. An appeal against findings that the report was reviewable and should be set aside for procedural error was dismissed, the Court of Appeal holding, at 11-12 :
One broad purpose of the 1972 Act, especially when taken with the 1977 amendments, was to remove technical problems which had until that time bedevilled applications for judicial review by way of the prerogative writs and declarations. Rather, the attention of the parties and of the Court should be focused on the issues of substance, especially the issues of what actual exercises of power are reviewable and on what grounds. In that inquiry the origins of the power and the various characteristics of the decider would often be very important, indeed frequently decisive. But that would not necessarily be so. Over recent decades Courts have increasingly been willing to review exercises of power which in substance are public or have important public consequences, however their origins and the persons or bodies exercising them might be characterised, eg R v Panel on Take-overs and Mergers, ex parte Guinness plc [1990] 1 QB 146 at pp 159 – 160, considered in the Electoral Commission case at p 430. Indeed, that judicially-made law goes back much longer when cases concerned with clubs (such as those referred to in Ridge v Baldwin [1964] AC 40 at pp 70 –
71) are taken into account. The lawmakers have supported that development over recent decades as is seen in the 1972 Act, the 1977 amendment (which
extends beyond powers conferred by statute to powers conferred by or under documents of incorporation) and most recently in the more broadly-drafted provisions of R 626 of the High Court Rules relating to certiorari. (The final provision was not referred to in argument and is not on the face of it directly relevant but it too indicates a willingness to remove technical impediments to the availability of judicial review proceedings.) The Courts have made it clear that in appropriate situations, even although there may be no statutory power of decision or the power may in significant measure be contractual, they are willing to review the exercise of the power including review for breaches of natural justice, the ground argued in the present case. See eg Re Erebus Royal Commission (No 2) [1981] 1 NZLR 618, Mercury Energy Ltd v Electricity Corporation of New Zealand Ltd [1994] 2 NZLR 385 (PC) and Webster v Auckland Harbour Board [1987] 2 NZLR 129.
While the historical development suggests a liberal approach to the availability of review proceedings, the powers of the Court are not, of course, at large. They must be rooted in principle and in the texts of the relevant statutes and rules. To return to the definition of statutory power in s 3, it is important to note that review is available not simply in respect of powers that are conferred by the constitutional document, but also in respect of powers which are conferred under it.
[49] It is pertinent to add that none of the three decisions on which TEC relied were adopted in any of the Privy Council and Court of Appeal decisions just considered and, being for the most part earlier, may be taken to be confined to their facts on this point. Though Whale Watch was upheld on appeal (Whale Watch Kaikoura Ltd v Transport Accident Investigation Commission CA87/97 12 May 1997), the learned authors of McGechan on Procedure take the view the decision at first instance is “questionable authority” in holding that TAIC reports are expressions of opinion only (McGechan para JA3.05.01(2)(g) p4-16). The modern approach to interim orders post Carlton & United Breweries Ltd v Minister of Customs [1986] 1 NZLR 423 tends to be expansionist and flexible (see the cases collected in McGechan para JA8.02(3) p4-57). For all those reasons guidance is more properly derived from Mercury Energy, Peters and Phipps than from the other cases discussed.
[50] When the International Comparison is considered against the background of those authorities, the conclusion must be that TEC’s decision to prepare the International Comparison and propose to publish it in the PBRF Report must amount to the exercise both of a “statutory power” and of a “statutory power of decision” under the Judicature Amendment Act 1972.
[51] In that regard, it is important, first, to note that there is nothing in the Education (Tertiary Reform) Amendment Act 2002 which would make the Commission’s decision to include the International Comparison in the PBRF Report unlawful in the sense of being beyond its functions and powers. Indeed, such was not argued. Preparing and publishing the Report, at least for use within New Zealand, can be seen as part of the Commission’s functions in providing advice to the Minister – and in this case to numerous others as well – as to the Tertiary Education Strategy and the activities and performance of the sector generally and as part of its function of conducting applied policy and programme research and monitoring and evaluating TEOs’ performance for the purpose of assessing their overall achievements in relation to the Tertiary Education Strategy and priorities. It could also, of course, be seen as part of TEC’s allocation of funding to the TEOs.
[52] Secondly, it could not be suggested that TEC was statutorily obliged to publish the International Comparison. It was its decision to create it and publish it. It may have had the laudable aim of trying to forestall less skilled analyses but, even so, it is clear that while it has a statutory power to create and publish the International Comparison, it was its decision so to do.
[53] Thirdly, the universities’ witnesses hold grave concerns as to the effect on the national and international research standing of the TEOs should the International Comparison be published. Prof Boston and the TEC witnesses hold the view the universities’ concerns are overstated but, critically, no witness suggests the universities’ position will not be affected – probably detrimentally – by publication of the International Comparison nor that they will not suffer in competition for research funding, both internally and internationally and in the competition for high calibre students and staff. And should the International Comparison be published in its present form without the universities having adequate opportunity to try to dissuade TEC from publishing it, the universities’ positions will be detrimentally affected without them having an adequate opportunity to try and neutralize the International Comparison. As regards that aspect of the matter, what objective observers may regard as an imbalance in authority needs to be taken into account: while the universities have considerable authority in this area, disinterested observers may regard their efforts at refutation as self-interested and thus of less cogency by
comparison with the impartial views of a State agency, buttressed as they are by detailed reasons supporting the numerical and graphical results despite the caveats expressed and the assumptions made.
[54] All those matters lead to the conclusion expressed that TEC’s decision to publish the International Comparison is the exercise of a “statutory power” and a “statutory power of decision” within the meaning of the Judicature Amendment Act 1972 and the authorities. It risks detrimentally affecting the universities’ position without redress. Publication of the International Comparison is claimed to be in breach of natural justice through inadequate opportunity for consultation and input. And it cannot be denied that publication of the PBRF Report is the exercise of a power by TEC which has important public consequences.
(2) Substantive grounds of review
(a)Submissions:
[55] The universities’ claim asserts TEC’s decision to publish the International Comparison is unlawful in an administrative law sense as being in breach of natural justice and the universities’ legitimate expectation, irrational and based on mistake of fact. The universities’ claim TEC failed to give them adequate opportunity to comment on its intention to use their data for a purpose for which it was not sought or to give them adequate opportunity to challenge the correctness of the material in it. They challenge the use by TEC of their data for one purpose when it was supplied for another as breaching their legitimate expectation. They say publishing the International Comparison is irrational. As valid comparison is impossible and the adverse consequences for them are so considerable, publication is said to be in breach of good faith and the obligation to act with reasonable care, diligence and skill. And finally, the universities assert that it was an error of law for TEC mistakenly to assume valid comparisons between RAE and PBRF are possible. The universities also claim there is a serious question to be tried in the sense that their case has a reasonable prospect of success and that the balance of convenience favours retention of the interim orders.
[56] TEC asserts on the substantive aspects of the case that there is no serious question to be tried and the balance of convenience is against maintenance of the interim orders because the universities’ case is weak, amounts to no more than a difference of opinion and retention of the orders runs counter to the precept of freedom of expression and is contrary to the New Zealand Bill of Rights Act 1990 s14 (though the direct inapplicability of that section was conceded at the hearing). TEC took the view that the public interest in publishing the International Comparison outweighed any disadvantages to the universities and retention of the order would delay release of the PBRF Report and result in wasted expenditure.
[57] Mr Miles submitted it was a breach of natural justice for adverse conclusions to be made against persons who had insufficient notice of what is to be said about them and opportunity to rebut it (Re Erebus Royal Commission: Air New Zealand Ltd v Mahon [1983] NZLR 662, 671). If information is provided for one purpose and is proposed to be used for another, notification must be given plus time to make submissions (Treaty Tribes Coalition, Te Runanga o Ngati Porou and Tainui Maori Trust Board v Urban Maori Authorities [1997] 1 NZLR 513, 522). An established practice of consultation can give rise to a legitimate expectation that such will continue (Te Heu Heu v Attorney-General [1999] 1 NZLR 98, 126 citing Re Westminster City Council [1986] AC 668, 692 per Lord Bridge of Harwich). Irrationality varies according to the facts of each case (Ports of Auckland Ltd v Auckland City Council [1999] 1 NZLR 601, 606). And a decision-making process miscarries if based on material mistake of fact (The Northern Inshore Fisheries Co Ltd v Minister of Fisheries HC Wellington CP235/01 4 March 2002 para [49] p12 Ronald Young J).
[58] In relation to balance of convenience Mr Miles relied on cases where courts have restrained publication of material on an interim basis should such be necessary to preserve an applicant’s position. Te Waka Hi Ika o Te Arawa v Treaty of Waitangi Fisheries Commission (CP395/93 HC Wellington 31 March 1999 p14 Anderson J) is one such decision. But this is not an issue which requires extensive consideration, Mr Taylor at the hearing resiling somewhat from TEC’s original ground of opposition that what the universities sought contravened the New Zealand Bill of Rights Act 1990 s14 to a submission that suppression of the International
Comparison contravened the principles of freedom of expression enshrined in the section rather than breach of the section itself. In that, he relied principally on defamation cases though those principles, as Mr Miles submitted, are based on different precepts at considerable remove from the present issue. That notwithstanding, Mr Miles submitted that cases where that has been done, such as Newspapers Publishers Association of New Zealand (Inc.) v Family Court [1999] 2 NZLR 344 were distinguishable as not dealing with public bodies and a case such as R v Advertising Standards Authority Ltd ex p Vernons Organisation Ltd [1992] 1 WLR 1289 where interim orders to halt publication were refused on grounds of freedom of expression were anomalous by comparison with other English authority such as R v Advertising Standards Authority Ltd ex p Direct Line Financial Services Ltd [1997] EWHC Admin.770 paras 15-16 p4, a successful application for an injunction to restrain publication of an adjudication where libel analogies were discounted
[59] Mr Taylor submitted TEC’s decision to publish the International Comparison was not affected by considerations of natural justice, relying on Southern Ocean Trawlers Ltd v Director-General of Agriculture and Fisheries [1993] 2 NZLR 53 where a fishing company’s application for judicial review of the Director-General’s allocation of previously forfeited quota to its previous holder was struck out partly on the basis of absence of any statutory obligation to consult the fishing industry or other quota-holders before making the decision. Mr Taylor submitted the argument for consultation in Southern Ocean was greater than could possibly lie on TEC since the Director-General’s decision denied other quota-holders their chance of increased allocation. He submitted publication of the International Comparison could have no effect on TEC’s allocation of research funding under PBRF.
[60] Alternatively, he submitted that TEC had fully complied with any minimal obligations of natural justice which might be found to affect its proceedings relying on Peters v Collinge [1993] 2 NZLR 554, 556-557 (following Russell v Duke of Norfolk [1949] 1 All ER 109, 118) that the requirements of natural justice vary according to the circumstances of the case. Mr Taylor made the point the International Comparison makes no findings of unlawful conduct or wrongdoing, affects no rights for funding under the PBRF and is no more than a piece of research
making broad comparisons between TEOs and the British institutions which could not possibly lead to litigation. That being the case, he submitted the most that could be required of TEC was to make the universities aware of the possibility of the International Comparison, show it to them before public release and possibly give them an opportunity to comment. TEC had, he submitted with reference to the chronology and evidence, fully complied.
[61] Mr Taylor also argued the universities could have no legitimate expectation that the data supplied by them would be used exclusively for the PBRF, drawing a distinction between the information supplied for PBRF purposes and public results of the assessment. He submitted legitimate expectation could apply only to the Evidence Portfolios but TEC did not use them to construct the International Comparison. Even were it thought TEC was using the PBRF data for the International Comparison, that, too, would give rise to no legitimate expectation that the data would not be used for any other purpose. He relied on The New Zealand Association for Migration and Investment Inc v Attorney-General (M1700/02 HC Auckland 16 May 2003 Randerson J) a challenge to alterations to immigration policy making it harder for migrants, especially those for whom English is not their first language, to obtain residency in New Zealand. In a, with respect, helpful review of the current status of legitimate expectation in New Zealand law, Randerson J observed (para [142] p45) that “expectation may be engendered by promises that a public authority will act in a certain way or by the adoption of a settled practice or policy which the claimant can reasonably expect to continue” and on which it is reasonable for the affected person to rely. The learned Judge followed R (Bibi) v Newham London Borough Council [2002] 1 WLR 237, 244 para 20 that the prime question is “To what has the Authority committed itself?” a question of fact determined in the circumstances of the case including the context and form of the promise.
[62] Here, Mr Taylor submitted it was always in contemplation that the International Comparison might be made as indicated in the earlier publications and there was no representation by the Commission that data supplied would not be used for that purpose. He also made the point the universities were not seeking a
permanent order against publication of the International Comparison but merely a reconsideration.
[63] On irrationality, Mr Taylor attacked the lack of any pleading alleging bad faith and submitted any failure to act with reasonable care, diligence and skill was an attack on the merits not a ground for review, relying on Sloan (at 479) and Whalewatch (at 61). Here, too, he suggested the universities could not by way of review attack the International Comparison since it was no more than an expression of opinion, nor could they challenge the merits of TEC’s decision to publish.
[64] Turning to mistake of fact, Mr Taylor said the universities’ claim amounted to an assertion that it was impossible to conduct a valid comparison. That was not borne out by the evidence. Sir Graeme Davies highlighted the difficulties in making such a comparison for it to be valid but did not assert the impossibility of the exercise. In any event, he submitted error of fact is unreviewable unless it is condition precedent to the exercise of power or gives rise to an unreasonable decision (Lewis v Wilson & Horton Ltd [2000] 3 NZLR 546).
[65] Mr Taylor submitted the balance of convenience in this case lay with TEC. The universities acknowledged the inevitability of comparisons being made by others and their fears of damage were, he suggested, overstated and should be balanced against freedom of expression. Damage to the universities’ reputation and consequential difficulties in obtaining high-quality staff, students and other research funding was analogous to defamation, he submitted, and accordingly injunctions in the defamation area were relevant.
(b) Discussion:
[66] Despite the importance and number of the issues raised, at bottom this is a reasonably straightforward case.
[67] Judicial review concerns pursuit of appropriate process. In this case the process between April 2000-February 2004 involved the painstakingly detailed creation by TEC and its predecessors of a PBRF model unique to and appropriate for
New Zealand’s TEOs, that process being characterised by correspondingly detailed consultation and discussion between TEAC, TEC and the Working Group and the TEOs on the one hand and between TEC, the Minister and Cabinet on the other as the model evolved. The universities were initially sceptical that a PBRF model could realistically be achieved for New Zealand but gradually accepted PBRF as the way to the future for research funding and participated fully in the process.
[68] While, as is readily understandable, the discussions and consultations leading to the preparation of TEC’s various reports struggled to create an assessment system fair to all TEO staff across all the widely disparate disciplines, it could not be said that the few oblique references earlier cited from the various reports about the expert panel’s ability to make research assessments against international standards or their being parallel with overseas research assessments or possible facilitation of international comparability of research excellence were in any way adequate to forewarn the TEOs not only that the International Comparison might be undertaken on the first, so far untried, PBRF round but that it would in fact be undertaken.
[69] Between March and September 2003 the TEOs were deeply involved in preparation and submission of their Evidence Portfolios against the background of the TEC guidance papers issued in the middle of 2003 as to the format and assessment of that work. Again, the evidence shows no change in relation to international comparability of the research assessments which might have alerted the TEOs to changes in the Commission’s thinking in that regard.
[70] While Dr Hood and Prof McCutcheon say their respective universities may not have participated in the PBRF had they known TEC was contemplating the International Comparison, that seems an unlikely stance for their universities given the resultant impact on research funding. That said, it is not difficult to accept the data contained in the Evidence Portfolios might have been in different form with more or less data being included or the data being forwarded subject to additional caveats had the universities known the International Comparison was in contemplation.
[71] From September 2003-4 March 2004, from the universities’ point of view the PBRF exercise went silent. That was the period when the assessments were confidentially undertaken, moderated and evaluated by the peer review panels. The universities expected this. Indeed, they saw confidentiality during this stage as an essential component of the integrity of the PBRF process.
[72] It was only on 4 March 2004 that the universities and NZVCC were told of the International Comparison’s existence. Quite when it was prepared is, even now, unclear but in a media interview on 18 March 2004 Dr West, TEC chairman, stated that :
One individual, one Commissioner off his own bat, over Christmas, over a weekend, using a single computer, did this analysis.
[73] Whatever be the correct position in that regard, two things are noteworthy. The first is that the university representatives immediately protested to TEC about the lack of forewarning they had received that the International Comparison would be attempted and what they saw as its inappropriateness, a stance they reiterated at the meeting on 8 March 2004. The second matter of note is that those discussions took place without the TEOs having seen the whole or any part of the International Comparison: that was not provided by TEC until 15 March 2004. All this occurred against the background of TEC’s publicised release date for the PBRF Report of 23 March 20-04.
[74] Certain other aspects need to be borne in mind in assessing whether TEC’s process in preparing the International Comparison complies with legal requirements.
[75] The first and most important of those is that it can scarcely be doubted – and, to be fair, TEC did not strongly challenge the view – that although undoubtedly attempts will be made by students, academic staff, research funders and institutions around the world to compare the research excellence of New Zealand TEOs with other universities and research institutions, any International Comparison published by TEC - possessor of all the research data on New Zealand TEOs and skilled and experienced interpreters of RAE - will have very much more clout than any other. When a State agency such as TEC, experienced and skilled in comparing, assessing
and evaluating research, publishes a report comparing the research achievements of New Zealand TEOs against their British counterparts, the result will be accorded an official status, an imprimatur no other comparison could hope to approach.
[76] The next point is that, however carefully TEC has caveated and conditioned the International Comparison and explained the assumptions and compilation difficulties which underpin the graphical and the numerical comparison tables, the fact is that few are likely to read the 15 pages of closely-argued discussion, and fewer still before reading the tables. TEC may regard it as an elegant piece of academic research but only aficionados of such matters are likely to immerse themselves in the fine detail.
[77] The next point is the damage which may be suffered by New Zealand TEOs if the International Comparison is published in its present form. The evidence is clear that competition is intense for the best students, the best academic staff, the best researchers, and for access to scarce research funding both Government and private, both in New Zealand and overseas. Clearly, to publish an International Comparison which rates New Zealand TEOs poorly by contrast with their British counterparts, will likely result in New Zealand TEOs doing worse in that competitive atmosphere than might otherwise have been the case.
[78] Dr Hood and Prof McCutcheon make clear their institutions are unconcerned about competition with the other New Zealand TEOs. What concerns them is the effect on New Zealand TEOs arising from the International Comparison which generally rates the New Zealand institutions more poorly than those participating in the RAE and in a way which, the Vice-Chancellors say, is so dependent on assumptions and comparative difficulties likely to be largely unread that publication of the International Comparison breaches TEC’s obligations. The effect of such comparisons and the difficulties of making them validly, is the reason, university witnesses say, few other nations attempt the exercise or publish the results. And when they do, they confine themselves to internal comparisons, not international ones. That means, in Dr Hood’s view, that not only is TEC’s attempt to publish the International Comparison contrary to law, it is also unwise in the first round of the PBRF which, however diligently it has been modelled and executed, is unlikely to
command universal acclaim. He makes the point the RAE has been repeated on a number of occasions over the past 18 years so the reliability of its results is more widely accepted.
[79] Returning to the claims advanced on behalf of the universities and applying the modern approach to judicial review in the authorities, in the circumstances of this case TEOs were entitled, prior to publication of TEC’s first PBRF Report, to know :
·All the purposes to which the data they supplied was being put. The comments in the reports up to and including the PBRF guide gave insufficient notice the data would be used for an International Comparison.
·Given the widespread consultation between TEC and its predecessors and the TEOs and the latter’s opportunity for input at each significant stage of the development of the PBRF model, TEOs were entitled to expect similar consultation and the opportunity for input concerning any change of direction by the Commission or the adoption of a new and unexpected aspect of the PBRF involving the use to which the TEOs’ data was being put.
·If TEC intended to use that data for a purpose other than that for which it was supplied, it was incumbent on it to give the TEOs sufficient forewarning of its intention and full opportunity either to limit the use to which the TEO’s data could be put or to dissuade TEC from using the data in that way and, in particular, from publishing the International Comparison.
[80]None of that occurred.
[81] Those findings mean the universities succeed on their first two grounds of review, breach of natural justice and breach of legitimate expectation.
[82] The contrary result must, however, be the appropriate conclusion in relation to the third and fourth grounds of review, namely irrationality and mistake of fact. Though Commission members have a statutory obligation to act with reasonable care, diligence and skill, there is no sufficient evidence they failed in that respect, much less they acted in bad faith. And while the International Comparison may be difficult, even flawed, it could not be said such a comparison was impossible.
[83] The universities have now had appropriate notice that TEC proposes to use their data for purposes of the International Comparison. It must follow that their sole, live, legitimate complaint is lack of consultation about all aspects of the International Comparison and lack of appropriate opportunity for input into those aspects, including whether it should be published at all at this stage. The universities need to recognise that one possible consequence is that, after the TEOs have had the opportunity of appropriate consultation and input, TEC may still decide to publish the International Comparison either in its present or a modified form and, the universities accepting that such publication would be within TEC’s powers, they will continue to run the risk of the damage which presently concerns them However, that is for the future.
Result
[84]In light of all of the above, the result is that
a)The applicants are entitled to continuation of the interim orders, namely that until further order of the Court TEC is prohibited from releasing Appendix D and paragraphs 24-26 and 159 of the PBRF Report to any person or enabling access to such report coupled with the other interim orders made on 17 March 2004 on the basis that the decision to include the International Comparison in the first PBRF Report and make it publicly available was unlawful in the sense those decisions were an exercise of statutory power and arrived at in breach of natural justice and the applicants’ legitimate expectations.
b)The applicants’ application for judicial review on grounds of irrationality and mistake of fact are dismissed.
c)Counsel are to confer and try to agree on the terms of the formal orders. If agreement proves impossible, leave is reserved to apply in that respect.
d)Those orders may bring the substantive proceeding to an end. If, however, such proves not to be the case, there is to be a telephone conference with counsel and Williams J on Tuesday, 11 May 2003 at 9:00a.m. with leave to bring the matter on earlier. If no such conference is required, counsel are to advise the Registrar.
e)The applicants are entitled to costs. Counsel are to endeavour to agree but, again, if agreement proves impossible, leave is reserved to apply or to file memoranda. If memoranda are to be filed, the applicants and respondent are to file the same within 35 and 42 days respectively of the date of delivery of this judgment.
………………………………..
WILLIAMS J
Signed at ……5:57pm…….. on the…5th…day of …April ... 2004
0
0
0