6.1 RIS3 monitoring

6.1 RIS3 monitoring
6.1 RIS3 monitoring logo

This method defines the overall process / roadmap for RIS3 monitoring, which usually encompasses all sorts of activities that have to do with the collection and processing of information about the achievement of expected results and the degree of implementation of policy measures.

A recent definition of monitoring is presented by Gianelle and Kleibrink (2015), stating that monitoring usually encompasses all sorts of activities that have to do with the collection and processing of information regarding the achievement of expected results and the degree of implementation of policy measures. Given the above, the investigation and design of both monitoring and evaluation processes taking place throughout the implementation of RIS3 projects is a crucial step. M&E must be properly organised and designed in detail for the implemented activities to achieve the highest possible socio-economic impacts.

Monitoring is an integral part of RIS3, and to receive Structural Funds assistance, all regions are required to develop tailored monitoring. The mapping exercised showed that detailed RIS3 M&E frameworks are currently in development in many regions, and that there is a great interest in inter-regional experience exchanges on this aspect of smart specialisation design.

Description of the method

Three main purposes referring to the utility of RIS3 monitoring mechanism should be highlighted here (Kleibrink et al., 2016). First, learning-and-acting is one of the core purposes of monitoring process, indicating the need for policy makers to be well informed of the reality conditions and the risks of failing, as well as the development of a learning system for participants. Second, trust-building amongst stakeholders and citizens, is another important dimension of monitoring processes. This characteristic should be accompanied by motivation, commitment and aversion to report failure. Last but not least, accountability is considered to be an integral part of this mechanism, built on clear rules for counting/measuring and bottom layer auditing.

In general, monitoring can be effectively interpreted in combination with the general RIS3 context, due to its strong connection with the RIS3 strategy structure. This means that depending on the diversified characteristics of the various RIS3 strategies that are implemented in EU countries, monitoring can take different forms. Furthermore, given the fact that monitoring can be regarded as an information gathering mechanism, one of its main utilities is to constitute a base upon which a periodic refinement and refocusing could be made.

Under this perspective, in order to design the monitoring system for a RIS3 strategy, it is important to understand in detail the mechanism lying behind it. Following the work of Gianelle and Kleibrink (2015), we can acknowledge the existence of three main discrete stages that constitute the main pillar of a RIS3 intervention logic: (i) recognition of challenges and needs; (ii) formulation of vision statement and strategic objectives; and (iii) selection of envisaged solutions to achieve these strategic objectives. The interconnections between these stages are shown clearly in Figure 15.

The complexity arising from the implementation of RIS3 strategic interventions within a country becomes clear when looking at Figure 15 below. Focusing on the schematic representation of the RIS3 implementation process, it is important to highlight that moving from phase 2 to phase 3 constitutes a crucial step, as it denotes the transition from the general to the more specific part of the planning procedure. During phase 3, the selection of the “means” (levers – E.1) that will be used to achieve the desirable “ends” (goals) referring to specific socio-economic areas (E.2) will ultimately define the set of priorities that will be determined.

Having chosen a specific set of priorities it is then possible to define explicit variables and indicators, which could be used to measure the expected change made by the selected levers on the targeted socio-economic areas of intervention. More specifically, according to Gianelle and Kleibrink (2015) expected change is defined by: (i) a variable of socio-economic nature; (ii) a set of baseline and target values; and (iii) a timeframe for variable observation. The identification process of the expected change measures therefore sets the foundation of the M&E systems. As a result, it becomes evident that the design for a well-organised and effective M&E mechanism is considered to be an integral part of the whole strategic RIS3 planning process.

Figure 15 The RIS3 structure

Figure 15 The RIS3 structure

Source: Gianelle and Kleibrink (2015, p.6)

Usability and impact

The monitoring and evaluation (M&E) system is considered to be an important part of the overall governance system, alongside budget, human resources and auditing systems (Kusek & Rist, 2004). Its main contribution is to provide an analytical feedback system regarding the outcomes, impacts and effects of the implemented policies and programs.

According to the World Bank (2016) there are two key mechanisms through which monitoring and evaluation (M&E) can affect the effectiveness of the implemented projects. First, M&E provides crucial evidence with respect to the degree to whicih each project achieves its goals, and its results comparative to the target values. Secondly, there is evidence throughout literature (Legovini, Di Maro and Piza, 2015) that M&E can affect the overall quality of project supervision and implementation, resulting in higher quality outputs.

Monitoring and evaluation are considered to be intrinsically interdependent, as the results derived from monitoring exercises include all the information needed to perform an evaluation analysis (Kleibrink et al., 2016). Monitoring by itself constitutes an early warning system, in cases of ineffective or even bad implementation of policy measures or actions. Thus, it is crucial to develop a design process that could provide the policy-makers with the ability to effectively choose indicators for monitoring. The more proper indicators are chosen, the better the results of their transformation into interventions to the already implemented policies and projects will be.

EU regional cohesion policy shift towards the smart specialisation framework requires a similar change in the way the monitoring process is implemented. According to Farole et al. (2011), under this new policy context, in order to improve the delivery and effectiveness of the intervention, better and leaner monitoring procedures should be included.

Required data

Policy makers define the data needs,, as they decide which indicators and databases will be used for the monitoring of the implemented policies. However, potential data sources for this method that could be used include Eurostat database, as well as regional and national reports. The frequency of the monitoring process should not be ignored when choosing the appropriate databases, as data availability, especially at a regional level, is in many cases restricted.

Relevant data sources

The data sources for the implementation process are not yet available, as the process by itself requires data collection throughout the period when the action programmes are being implemented.

Implementation roadmap

Regarding the structure of the application that could support this methodology, there are some key features that should be applied throughout it. As it is shown in Figure 16, the collection of new or already existing data during the data selection process will be used as input for the development of intermediate reports throughout the implementation process. Therefore, the results arising from these intermediate reports will lead to corrective policy measures, increasing the effectiveness of the implemented actions.

Figure 16 Steps of implementation of the monitoring process

Figure 16 Steps of implementation of the monitoring process

  • Alexander Kleibrink, Carlo Gianelle & Mathieu Doussineau (2016) Monitoring innovation and territorial development in Europe: emergent strategic management, European Planning Studies, 24:8, 1438-1458, DOI: 10.1080/09654313.2016.1181717
  • Farole, T., Rodríguez-Pose, A., & Storper, M. (2011). Cohesion policy in the European Union: growth, geography, institutions. JCMS: Journal of Common Market Studies, 49(5), 1089-1111.
  • Gianelle, C., & Kleibrink, A. (2015). Monitoring Mechanisms for Smart Specialisation Strategies. S3 Policy Brief Series, (13).
  • Kusek, J. Z., & Rist, R. C. (2004). Ten steps to a results-based monitoring and evaluation system: a handbook for development practitioners. World Bank Publications.
  • Leeuw, F. L., & Furubo, J. E. (2008). Evaluation Systems What Are They and Why Study Them? Evaluation, 14(2), 157-169.
  • Legovini A., Di Maro, V. Piza C. (2015) Impact Evaluation helps Deliver Development Projects. World Bank Policy Research Working Paper 7157.
  • Raimondo, E. (2016) What Difference Does Good Monitoring & Evaluation Make to World Bank Project Performance? World Bank Policy Research Working Paper 7726.