UPDATES March 2017: SIPER
SIPER – Science and Innovation Policy Evaluations Repository – is a central source of knowledge on research and innovation policy evaluations, run by a team at the Manchester Institute of Innovation Research. Its aim is twofold:
- – to provide on-line access to a unique collection of policy evaluation reports, located at a single location;
- – to allow policy and evaluation learning by providing a detailed characterisation of evaluations in a way that is searchable for policy makers and other stakeholders and provides the basis for additional academic analysis.
The SIPER project is based on the consideration that a central source of knowledge on research and innovation policies lies in the evaluations that are periodically made by most countries in the world (and especially OECD and OECD-related countries). It builds on an experiment which collected and reviewed some 150 evaluation reports on European innovation policies (IPER) carried out under the EC project ‘INNO-Appraisal’. This demonstrated that such an exercise was feasible and that it could be opened as a ‘service’, provided that the repository has a relevant set of metadata enabling interested scholars or policymakers to select those evaluations relevant to the questions they ask.
SIPER consists of a web portal enabling distant access to the repository of evaluation reports and associated documents (in pdf format) and a database of metadata drawn from the individual evaluation reports. This metadata is based on an analysis carried out according to an extensive characterisation template which examines a range of aspects of the evaluation report, including: the purpose of the evaluation, the data collection and analysis methodologies used, the type of instrument evaluated, etc. The characterisation also reviews aspects of the quality of the evaluation (based on both internal and policy maker perceptions).
The SIPER team is also working in close cooperation with the OECD and World Bank with the aim of integrating SIPER into the OECD Innovation Policy Platform (IPP).
We aim to include evaluations of science and innovation policy programmes conducted after 2000. Starting with OECD countries, SIPER aims at a global geographical scope.
SIPER is constructed through a four-stage process:
- Step 1: Collecting science and innovation policy evaluations from a wide range of sources across the world;
- Step 2: Uploading evaluation documents into the SIPER Portal;
- Step 3: Characterizing the factual information (timing, topics, methods, designs, etc.) of each evaluation and its related policy measure (target group, modality, objectives, country, etc.) so that evaluations are searchable by these characteristics, by any user through a simple web interface;
- Step 4: Inviting policymakers to provide their judgements (on quality and usefulness, etc.) on a confidential basis for the purposes of academic research.
All characterisations are done by evaluation experts and researchers trained by experienced evaluators following strict explicit guidelines and quality controls.
SIPER was officially launched at the Open Evaluation Conference, Vienna, November 24-25, 2016. Prof Jakob Edler and Dr Paul Cunningham presented a practical demonstration of the way in which SIPER may be searched according to various evaluation characteristics.