Transparent, traceable and verifiable: guidelines for assessing the quality of social science research

19 March 2013

The Dutch State Secretary for Education, Sander Dekker, will take receipt today of an advisory report concerning quality assessment in the social sciences. The report provides guidelines for assessing the quality of research in a manner that allows for the enormous diversity of quality cultures in the social sciences.

The report challenges stakeholders to work with the method so that their impressions can be taken into account in the new Standard Evaluation Protocol (SEP), which will become effective in 2015.

In Towards a framework for the quality assessment of social science research, the Royal Netherlands Academy of Arts and Sciences gives researchers, deans, research directors and external review committees  a framework for assessing the quality of social science research. The report was drawn up by the 'Quality Indicators for the Social Sciences' Committee, chaired by Jozien Bensing, Professor of Clinical Psychology and Health Psychology at Utrecht University. The report is one part of a triptych. The other parts consist of two earlier advisory reports, one concerning Quality indicators in the humanities (May 2011) and the other concerning Quality assessment in the design and engineering disciplines (November 2010). You can download the report from www.knaw.nl/publications.

Background

The Academy had three reasons for producing this report. The first was the heterogeneous nature of the social sciences. So far, research quality assessments have not made enough allowance for the enormous differences between the various fields, hampering an honest appraisal. This is a problem that both funding bodies and university administrators face. Second, the growing interest in knowledge valorisation has sparked off a discussion of the relevance of research to society, and it has also increased the demand for more visible quality and relevance. Finally, researchers have grown increasingly dissatisfied with the tendency to express the quality of research in one-dimensional and largely quantitative terms, and to adhere to the principle of 'more is better'.

Advisory report

The advisory report does not give administrators and researchers a ready-made system, but rather a method for shaping the form and content of quality assessments. The method allows for the huge differences within the social sciences and for important trends in society, such as the growing interest in knowledge valorisation. It also helps researchers in the various fields take responsibility for making quality in their field transparent, traceable and verifiable. The report offers a set of generally applicable assessment criteria. It also explains the principles that define the framework of the assessment system. How these principles will be put into practice is left to the researchers and administrators, who can use guidelines for different types of assessments, such as annual appraisal interviews with staff, appointments of professors, faculty investment policy, national research reviews, and so on. The Academy asks researchers and administrators in the social sciences to work with the proposed methodology so that it can draw on actual experience when implementing the new 'Standard Evaluation Protocol' (SEP), which enters into effect in 2015.

Study by the NWO Social Sciences Division

In line with the Academy’s report, the NWO Social Sciences Division (MaGW) commissioned a study on how the publication cultures in the various social science disciplines and subdisciplines differ from one another. The outcome of the study is a transparent categorisation and description of the publication cultures within the primary social science disciplines and subdisciplines. A pilot project is now being carried out to test whether these descriptions can be applied in an assessment procedure (in connection with the NWO’s VENI grant). The aim is for the descriptions to serve as a guideline for committee members who are asked to assess a list of publications in a discipline whose publication culture is unfamiliar to them. A brochure concerning the publication culture study can be downloaded from www.nwo.nl.