A Study on Tools to Measure Various Aspects of Quality Assurance In Education

Stakeholders are individuals and groups who have a major interest in higher education institutions and systems, and their work and achievements. Quality assurance serves a number of purposes. Apart from protecting student and employer interests and facilitating international recognition of the standards of awards, it is an important element for public accountability purposes, particularly to satisfy taxpayers about value for money and that government subsidies are supporting education activities of an appropriate standard. Quality assurance helps inform student choice, especially in the light of a growing diversity of course offerings. It also can contribute to improved teaching and administrative processes and help disseminate good practice, leading to overall system improvement. Quality assurance is particularly important in an age of globalisation and growth of knowledge based economies. Globalisation has brought numerous benefits especially for countries that have been able to take advantage of liberalisation of trade and technological breakthroughs. At the same time, globalisation demands greater mobility of professional and skilled labour as well as increased efforts to achieve mutual recognition of university and college awards. In the modern world, more effective cooperation between higher education institutions and nations is essential Quality assurance today is driven by a number of pressures, many of which produce tensions and conflicts. These include the tensions and conflicts between the explosion and fragmentation of demand for student places on the one hand, and unemployment which affects an ever growing number of graduates in a number of countries of the region on the other; between the provision of equal access and opportunity, and the financial constraints that follow the mass extension of higher education; between the pressures for increased institutional autonomy versus those for growing public accountability; and between ethical and moral obligations and the various pressures for the generation and communication of new knowledge and scientific discoveries. Faced with these pressures, higher education must develop new visions and new forms of cooperation across both institutions and nations. As the conference brochure emphasized, the issue of quality cannot be disassociated from the quest for excellence, while the setting of academic standards and evaluation criteria should take into account the diversity of situations across the region. Each nation and higher

Available online at www.ignited.in Page 2

education system should aim to put in place quality assurance systems and procedures that meet the needs and the culture of the local society. Importation of systems from one country to another needs to handle with care so that any new set of arrangements and anchored firmly in cultural and social traditions.

(i) Feedback on Education : Processes Inasmuch as students are the primary reason for the existence of institutions of higher education, a system to generate feedback and assess the processes of design, development and delivery of curriculum is central to an institution's quality assurance system. Such feedback is critical to the validation and refinement of the core teaching and learning processes. Some instruments of feedback and improvement at NUS are as follows: • Student feedback • Peer Review • Video Recording of Lectures • Employers and Alumni Feedback • Department Consultative Committees Student feedback on the quality of teaching has been a core practice for more than a decade now. Both undergraduate and postgraduate students participate in end-of-course surveys and provide critiques on a course and the faculty member responsible. The information obtained is used to drive teaching effectiveness and subject matter relevancy. Peer reviews provide staff with the benefit of constructive feedback on teaching styles and techniques offered by experienced faculty members observing lectures and tutorials personally. The teaching faculty is encouraged to have their lectures video-taped for follow-up assessments of lecturing style. Such tapes can also be given voluntarily to Department Heads to evaluate the quality of teaching of faculty members. Employers and alumni provide another source input for curriculum design and professional training. Their views and expectations are regularly solicited through a myriad of surveys. Department Consultative Committees comprise the department management team and representatives from relevant government agencies and individuals. These committees meet regularly to assure that the curriculum is kept abreast of developments outside academia, as well as to inform academics the expectations of employers and society of our graduates. (ii) Boards of Examiners : A comprehensive university such NUS must assure that there is some balance and compatibility in the award of various classes of qualifications among the various programmes of study. Boards of Examiners meet to moderate the examination scores and determine the different classes of Honours to be assigned to graduates. Recommendations for prizes, medals and other awards are also deliberated on at such meetings. (iii) Assessment by Professional Bodies : As a supplier of skilled manpower to the country, the involvement of and accreditation by industry, as represented by the appropriate professional institutions, provide an element of endorsement of our teaching programmes. (iv) External Examiners : A first-rate institution has to demonstrate the international recognition of its degrees. An External Examiners system is used at NUS in which senior academics from reputable overseas universities are appointed to scrutinize examination papers; they also visit the respective departments at least once in a two-year term, sit in at project presentations and examine marked scripts and theses. Apart from ensuring that our academic programmes are maintained at a high international standard, the external examiners also advise the faculty on matters relating to teaching and curriculum design to ensure that the desired level of quality is defined, monitored and maintained. As for graduate degrees, a thesis must be examined by external examiners before the oral defense is called. Such elaborate arrangement ensures that international standards are kept and there is no danger of intellectual inbreeding - given the very small number of institutions of higher learning in Singapore, especially for the professional disciplines. Quality Assurance in Research Apart from teaching excellence, the other pillar of a world-class institution must be its research. I shall outline some current mechanisms by which NUS maintains the quality of its research activities. (i) International Academic Advisory Panels : To ensure that research activities keep abreast of the latest developments and are aligned with internationally accepted norms, a panel of international advisors is established in each Faculty. Members of the panel would comprise

Available online at www.ignited.in Page 3

renowned academics in specialized fields. The panel is charged with the responsibility to review research output and advise on directions for research. The panel visits NUS on an average of once in two years. Apart from subjecting the faculty’s research areas to careful scrutiny, the close interaction that follows between panel members and faculty staff often creates a much-valued synergy and impetus of ideas for future research. (ii) Research Project Evaluation: Being funded and supported primarily by the government, NUS practises a high level of financial prudence, especially in research. A system is in place to evaluate project proposals submitted for research funding. Depending on the decision arrived at, the source of funding can be either from the university research budget or externally from various government agencies as well as industry. A project proposal requesting a grant higher than some threshold amount would be subjected to an external and independent review as well. All these levels of scrutiny are co-ordinated by the university's Office of Research and Graduate Studies. For continuing funding, researchers are required to submit regular progress reports on their projects. Management committees critique these reports and a decision is reached to determine if a given project continues to be viable. Key indicators of good research output include the number of patents filed, research papers published in refereed journals of good standing, and the impact of research findings. (iii) Assessment of Research Publications: Standards for quality assurance of research publications are most appropriately determined and set at the faculty and even department level. At NUS, the current system involves the compilation of a list of high quality and well-recognized international and regional refereed journals in a given field of research. This list is constantly reviewed and updated, with some faculties even taking the extra effort to seek views and comments from overseas experts on the accuracy of their lists. The list is then distributed to faculty members who in turn are encouraged to submit their research findings for publication in top-tier journals. In a field such as Computer Science where constant generation of conference papers arising from rapid advances in the field is the norm, the department may also set an internal standard by ranking conferences based on quality and reputation. This, however, is undertaken in conjunction with inputs from leading researchers in various fields.

We have, then, come a long way in the last ten years: institutional and national agendas for quality assurance have become ever more closely entwined; and the exposure to different forms of external review and the increased emphasis on demonstrating accountability for quality and standards has undoubtedly had a significant impact on what we in institutions do now, and the way we do it. For many institutions, the experience of being asked the deceptively simple academic audit questions of what are you doing, why are you doing it, how do you know it is the best way, and how do you know it works, has occasionally been an uncomfortable one. But it has also opened up areas for debate and enquiry from which much has already been learned and where we all recognise that more can be done. One of the most challenging things to get right is the information produced from quality assurance and the ends to which it is put. Information generated from accountability and for enhancement can also - as many of us have found - be used for other purposes. In the UK, the development of national approaches to quality assurance and of ‘quality league tables’ have gone hand in hand. Information about the effectiveness of an institution’s quality assurance arrangements, or indeed its academic standards, is not easy to convey to a wider ‘lay’ public in straightforward terms. Simple descriptive indicators (‘excellent’, ‘approved’) or numerical scores are valued by the press and might seem, at face value, to be helpful to the public. However, they may obscure as much as they reveal. Published reports on quality management and subject assessments need to carry a ‘health warning’ reminding the reader that they are only snapshots in time. What is surely of greater importance to the student/parent/employer is to see evidence of an institution’s continuing awareness of its responsibilities for quality and standards, of its willingness to compare itself with others and of openness to critical scrutiny. In a system like the UK where there is no formal hierarchy of universities, but a persistent and still persuasive informal ‘ranking’ of institutions, independent and reliable information about quality and standards could put a powerful tool in the hands of the press and public. The environment for institutional quality management and for teaching and learning has changed very rapidly in the last ten years. Many argue that the electronic revolution in

Available online at www.ignited.in Page 4

communication and information technology will transform our understanding of the university or college as a geographically-grounded teaching and learning institution, and enormously expand the horizons of the learners. We are just beginning to ask ourselves whether our investment so far in quality assurance has prepared us sufficiently for the challenges of the ‘global’ learning environment.

The first point is to reiterate that the individual university is responsible for the quality of the degrees and diplomas it awards to students: any external system of quality evaluation must not subvert that responsibility. Nevertheless students and employer do need to be assured that the university is fulfilling that responsibility. External quality assurance systems (which are usually national) can however be concerned with two distinct phenomena. On the one hand they can start with acceptance of the universities right to set the standard of its degrees and evaluate the effectiveness with which the teaching and learning achieves these standards that is the evaluation assesses the delivery of education. This is a test of fitness for purpose. Or, alternatively, the exercise can try to judge the standards themselves. This is to test fitness of purpose. The latter is much more difficult, but it is ultimately of greater significance to both students and employers and any scheme which provides a really useful comparison across universities must assess standards and create benchmarks. Any attempt to create benchmarks which facilitate comparability raises considerable problems, even if limited to a national scale. A fundamental issue is to decide whether the comparison is to be restricted to a minimum level of performance – that is the lowest level necessary for the academic award – or should also attempt to identify a range of levels of achievement or grades. This might seem an easy problem to resolve, at any rate in the first instance, by agreeing initially to benchmark the minimum level only. But this is not as easy as it might seem for current practice often recognises a range of levels, while appearing to be diverse in not always equating 'minimum' to 'satisfactory' performance. This view may seem perverse since it is difficult to understand how performance which was regarded as good enough to justify an award (or at least the completion of an essential step towards the award) could be held to be unsatisfactory. But the underlying perception that there can be such a distinction has come to light in an analysis of some pilot studies in benchmarking in the U.K. This view seems to be related to subject-specific cultures, for in some disciplines (for example chemistry) the assessment of student performance begins with a perception of an 'ideal' performance and marks down to a minimum, with an assumption that anything less than the ideal is in a sense unsatisfactory. In Law on the other hand, the reverse process operates, that is the starting point is to define the minimum. The significance of this finding is that it exposes some deep - and usually unarticulated - assumptions about the purposes of evaluation of student achievement. Such differences in approach between subjects might be manageable if all student programmes were restricted to watertight disciplinary compartments, but increasingly interdisciplinary courses are becoming the norm. Furthermore the current concern that students should leave university not only with academic knowledge and skills but also with a high level of employment skills, has led to a considerable diversity into both the incorporation of education for the latter but also, and more crucially for benchmarking, into their assessment. If it is difficult to agree on a common benchmarking at the national level, the problems are multiplied on an international scale, because of differences between countries in the basic structures of educational systems, in particular in the length of initial courses. Partly as a consequence of globalisation, matters that are now envisaged when the word ‘quality’ is used are very wide-ranging, from academic excellence through national development to international recognition. It is time to take a more flexible approach to our concepts of quality, qualifications, quality assurance, and mutual recognition.

• Ball, Christopher (1985) Fitness of Purpose, NFER and NFER-Nelson, Guilford. Brennan, John, de Vries, Peter, and Williams, Ruth (Eds) (1997) Standards and Quality in Higher Education, Jessica Kingsley, London. • Department of Education, Training and Youth Affairs (1998) The Characteristics and Performance of Higher Education Institutions, Higher Education Division, Canberra. • Harman, Grant and Meek, V Lynn (2000) Repositioning Quality Assurance and Accreditation in Australian Higher Education, Evaluations and Investigations Programme, Higher Education.

Available online at www.ignited.in Page 5

• Anderson, D, Johnson, R and Milligan, B (2000), Quality Assurance and Accreditation in Australian Higher Education, Evaluations and Investigations Programme, Higher Education Division, Department of Education, Training and Youth Affairs, Canberra. • Harman, Grant (1995) Quality Assurance for Higher Education: Developing and Managing Quality Assurance for Higher Education Systems and Institutions in the Asia and the Pacific, Asia-Pacific Centre of Educational Innovation for Development, Unesco Principal Regional Office for Asia and the Pacific, Bangkok, 1996.