Following the lead of the Regulatory Impact Unit (RIU) in Britain's Cabinet Office, impact assessments are in fashion in governments across Europe at the moment, with the European Commission replacing its earlier interest in ex ante evaluation with impact assessment. But does impact assessment actually deliver the goods or is it a paper-pushing game that bureaucrats play in order to get their regulations through the hoops of Parliament? This note argues that impact assessment is not as easy as it looks, if the early history in the Commission is observed.
The 2002 Communication of the European Commission on Impact Assessment committed the Commission to undertake impact assessment of all major policy proposals in order to “improve the quality and coherence of the policy development process.” It was also seen as contributing to regulatory reform.
By 2003, a unified system of integrated impact assessment had been introduced which replaced several separate, more specialised forms of assessment that were in use in different parts of the Commission. The new system consisted of ‘preliminary’ assessment of all proposals in the Commission’s work programme and ‘extended’ assessments of major proposals. In the first year of use, a total of 20 Extended Impact Assessments (Ex IA) were completed with a further 46 intended for 2004. However, during the course of 2004, the Commission decided to drop the ‘Extended’ label on the grounds that all policy proposals were deserving of integrated impact assessment.
What evaluators found useful in the Extended Impact Assessment system was the quality review package which showed how to assess an Ex IA. Like its predecessors, the IA Review Package contains two main components, a list of Review Topics and a Review Procedure. However, where the review package differs from previous iterations is that:
1) the integrated impact assessment covers economic, social and environmental impacts, rather than just one of them at a time, and 2)
the review package assesses the quality of the assessment process, not just the assessment report.
While the genre of impact assessment was being developed in the Commission, further work was done to extend the ideas into the IST domains. A study in 2004 by a consortium led by IZET into the impact of the IST programme and its predecessors identified 58 “high-impact” projects, though the project methodologies limited the study to the expected impact at the end of the study, rather than actual impacts.
Further, a November 2004 study aimed at developing a pragmatic methodology for integrated impact analysis of all IST R&D activities found that the IST priority in FP4 and FP5 led to the increase of the knowledge base, researcher skills, and the development of the research networks in many areas where Europe is currently the IST leader, such as Mobile Communications and Microsystems.
Historically, and none of these studies changed the facts on this score, it has been difficult to measure progress of FP-supported research activities against the EU’s higher level policy objectives, such as the Lisbon agenda or the creation of the European Research Area (ERA), primarily because of the perceived “attribution gap” between research outputs and higher level outcomes. The commonly held view that “one thing automatically leads to another” perhaps reinforced the use of quantitative targets and performance indicators as evidence of these higher level impacts, even if it were not necessarily the case.
In a future note, I will outline a case-based view of conducting impact assessments that seeks to address some of the limitations in the current methodologies.
Comments