Fine-tuning quality assessment in the design and engineering disciplines

22 November 2010

At the request of the three Dutch universities of technology, the Royal Netherlands Academy of Arts and Sciences has advised on the way in which quality can be adequately assessed in the design and engineering disciplines.

The advisory report published today, Kwaliteitsbeoordeling in de ontwerpende en construerende disciplines [Quality assessment in the design and engineering disciplines], recommends differentiating between disciplines when it comes to assessing output.

The advisory report sketches a system consisting of two assessment criteria: research quality and societal relevance. Disciplinary differentiation is achieved by using a system of indicators to determine how closely a research proposal, individual researcher or research group satisfies these two criteria.

The advisory report contains a table listing the indicators regarded as relevant for the design and engineering disciplines. The table is reproduced here. The next stage is for the scientists themselves to select the relevant indicators and to compare their choices with their peers for each of the various assessment situations, in close consultation with university administrators. The Academy hopes that the universities of technology and STW Technology Foundation will waste little time adopting this flexible assessment method, which offers scope for diversity in the sciences.

The advisory report was prompted by the problems scientists in design and engineering encounter in the assessment of the quality of their research output, whether that assessment takes place within the context of an external evaluation, an academic appointment or promotion, or an application for funding. The quality indicators used in such situations are borrowed from the more basic sciences (publications in ISI journals with a high impact factor, citation scores, the Hirsch Index, and so on). The products of the design and engineering disciplines, however, consist not only of peer-reviewed journal articles, but also of conference proceedings, designs, software and structures.

Kwaliteitsbeoordeling in de ontwerpende en construerende disciplines. Een systematisch kader was written by an Academy committee chaired by Prof. Anthonie Meijers (Eindhoven University of Technology).

The report reflects the recommendations of the Academy advisory report Kwaliteitszorg in de wetenschap: van SEP naar KEP (2008), which recommends a more flexible quality assessment protocol.