2.4 Benchmarking

2.4 Benchmarking
2.4 Benchmarking logo

Benchmarking is the process of improving performance by continuously identifying, understanding and adapting outstanding practices and processes found inside and outside an organisation (company, public organisation, university, etc.). The increasing competition among countries and regions stemming from globalisation has led to the progressive transfer and application of benchmarking approaches to the territorial context, not only to national governments, but also to European Union policies and regions (Koellreuter, 2002).

Many regional strategy-building and development initiatives contain some form of benchmarking in order to establish or further regional economic and innovation strategies (Huggins, 2008). Regions use international benchmarking practices as a tool to found their priority setting process (OECD, 2013). As it has been shown in the mapping exercise of RIS3 strategies, benchmarking is one of eight most common methods used across RIS3 strategy design. More than 60% of regions have used benchmarking during the context analysis phase and 30% also during the phases of vision building and/or policy mix. Some regions conduct systematic comparisons at a national and international regional level in order to diagnose their current situation and improve their ranking, and others implement a benchmarking process in specific sectors or a combination of them.

Among the regions that have applied benchmarking in RIS3 development, are: Wien (AT), Wallonia (BE), Bayern (DE), Central Macedonia (EL), Eastern Macedonia (EL), Midi-Pyrenees (FR), Noord-Holland (NL), Zuid-Holland (NL), Warmińsko-Mazurskie (PL), Centro (PT), East Sweden (SE), Stockholm (Sweden) and Northern Ireland (UK).

Description of the method

Between 1980 and 1990, benchmarking was one of the most popular and widely adopted management methods, and helped many organizations to improve their competitive advantage (Adebanjo et al. 2009). There are several publications that provide a broad review of the literature on benchmarking (Yasin 2002; Kumar and Chandra 2001; Wainwright et al., 2005). The benchmarking process involves comparing one’s organization performance within a set of measurable parameters of strategic importance against that of another organization that is known to have achieved the best performance using the same indicators (Kelessidis, 2000).

However, regional benchmarking differs considerably from business benchmarking where the transfer of best performances or best practices can be applied more easily. Unlike companies, territories do not have the ultimate goal of seeking to maximize profit. On the contrary, they are characterised by frequent trade-offs among multiple goals that public policies try and/or are compelled to pursue simultaneously (Schuldi, 2003). Regional benchmarking can be a very helpful tool for making strategic decisions within the process of the design and implementation of regional Research and Innovation Strategies for Smart Specialisation (RIS3) (Navarro et al., 2014). Through comparative analysis, it can provide us with useful information about the position of a region with respect to other regions as well as facilitate policy learning based on the transfer of good practices across borders.

This report aims at providing a methodology for the development of a generic benchmarking tool that will support the design and implementation of regional Research and Innovation Strategies for Smart Specialisation. This method allows for the comparison of the performance of a regional or national entity with respect to other similar entities for a specific aspect. Benchmarking must involve the following elements:

Selection of regions with which the region under examination wants to be benchmarked

Based on the idea that a comparison is likely to be more valuable when it is carried out between fundamentally equivalent entities, it makes sense to select regions that have similar structural features, such as regions that specialise in the same industries or that have similar demographic characteristics. However, it can be also useful to compare neighbouring regions for issues of transregional complementarities, competition and cooperation. The basic options for using benchmarking could be to compare:

  • Neighbouring regions
  • Regions in the same country
  • Regions willing to cooperate and learn from each other
  • Regions with similar profile and/or facing similar problems or challenges
  • Regions with best performance or best practices

Thus, the selection of the benchmarked regions depends on the overall scope of the benchmarking exercise and its specific objectives. The JRC has developed an interactive tool for regional benchmarking[5] based on structural similarities between regions. More specifically, the comparison is realised according to a synthetic index of structural distance that takes into account various regional characteristics, such as geo-demography, educational level, technological specialisation, etc. However, in this case users do not choose the compared regions; they only select one region, and the structurally similar examples for comparison are automatically provided.

Comparison of the performance of the chosen regions in a specific aspect

Performance benchmarking could cover a wide range of different aspects: economic development, environmental issues, demographic characteristics, social dimensions etc. These aspects could be generic, such as the aforementioned examples, or specific, for example employment per industrial sector, CO2 emissions per capita or even ICT patent applications. In the event that a generic aspect is chosen, a series of indicators that represent this aspect must also be selected. It is important to choose indicators that, when combined, reflect adequately the complex nature of the benchmarked element. In the case of the benchmarking tool developed by JRC, all regional characteristics are combined into a single synthetic index and regions are compared as a whole and not in specific aspects.

Analysis and interpretation of the obtained results

Once the collection of all the necessary data for the benchmarking process is done, data analysis is conducted. Through the performance of calculations on selected indicators, basic statistical metrics are obtained (minimum, maximum, mean, mode, quartiles etc), in order to rank the regions according to the aspect that is examined. Comparable information and statistical measures are essential for the development and implementation of coherent and comprehensive policy strategies. Therefore, the benchmarking exercise in this context concludes with the critical review of the obtained results and the elaboration of a complete structured benchmarking report which highlights the observed performance and provides useful insight regarding the practices that are responsible for this performance.

Usability and impact

Benchmarking is considered a useful tool that can help to identify the strengths and weaknesses of territories (OECD, 2005). More specifically, it can be a valuable tool for the identification of regional specialisation patterns and the comparison of economic activities, including agriculture, as well as strengths with other regions in the EU. Together with other tools like cluster analysis, benchmarking can be used for starting the assessment of regional specialisation patterns and comparing statistical findings among regions (Foray et al. 2012).

Moreover, understanding factors underlying regional performance can provide useful knowledge that can be applied to strategic planning and policies. Benchmarking is an exercise generating applicable in-depth knowledge about the regional economy focusing on its comparative advantages and disadvantages (Iurcovich et al. 2006). The benchmarking process should be part of a holistic approach for strategic policy. This means that it should be conducted in coordination with other tools such as regional foresight and regional assets mapping.

Required data

The Benchmarking tool covers all key regional indicators that characterize a regional profile. These indicators can be considered to represent the key dimensions of a region as presented in the Regional assets mapping, which are:

  • Geography: basic regional characteristics and connectivity
  • Demography and Society: population, density, education etc.
  • Economy and Labour: employment, GDP, growth rate etc.
  • Sectoral structure: distribution of economy and industry including agriculture, business, manufacturing etc.
  • Business characteristics: active enterprises, company size etc.
  • Innovation System: knowledge institutions, R&D etc.

However, it should be remarked that some statistical data at the level of region are not always available across the EU and, therefore, additional efforts should be made by some regions to complement existing data sets by more detailed quantitative and qualitative information.

Relevant data sources

For the identification of the indicators for benchmarking (see Step 2 below), various possible sources of information can be found. The primary data sources include among others:

  • Eurostat (demography, geography, education, economy, industry)

http://ec.europa.eu/eurostat/web/regions/data/database)

  • OECD Regional Statistics and Indicators

http://www.oecd.org/gov/regional-policy/regionalstatisticsandindicators.htm

  • Regional Innovation Scoreboard

http://ec.europa.eu/growth/industry/innovation/facts-figures/regional_es

  • Regional Innovation Monitor

https://ec.europa.eu/growth/tools-databases/regional-innovation-monitor/

  • European Social Survey (Human values, politics, social aspects)

http://www.europeansocialsurvey.org/

  • National or regional statistical offices

It should be remarked that some statistical data at the regional level are not always available across the EU and, therefore, additional efforts should be made by some regions to complement existing data sets by more detailed quantitative and qualitative information.

Implementation roadmap

An analytical roadmap regarding the steps for the implementation of the benchmarking process is presented below (Figure 1). The blue colour represents steps that involve input processes and orange indicates output processes.

Step 1. Objective and scope of the benchmarking exercise. It is important to define the overall scope of benchmarking, in order to plan appropriately the process and obtain useful insights.

Step 2. Selection of regions to compare. The European regions of member states are already defined in the platform and the user has a drop-down list to choose which of them will be benchmarked.

The number of selected regions is open to the user.

Step 3. Definition of the aspect(s) to be benchmarked. The user can define the aspect to be benchmarked either selecting from an existing pool of aspects (drop-down list) or inserting the element manually.

Step 4. Identification of indicators. The indicators that are selected should reflect the multifaceted nature of the benchmarking element more appropriately. For example, GDP is not a sufficient variable for the economic development of a region. Similarly with the previous step, the user can define the indicators either from an existing pool of indicators and variables (drop-down list) or inserting additional ones manually.

Step 5. Creation of the benchmarking database. Having obtained the above data, the necessary information is gathered and stored in the platform.

Step 6. Production of the benchmarking results. Calculations on selected indicators from different regions, providing the main statistics and graphs for the statistically significant indicators (minimum, maximum, mean, mode, quartiles). Based on the results, the region in focus is positioned within the statistical range of these statistics and the user can choose between different types of visualizations. The obtained results can be either exported by the user in the form of tables and figures, or integrated in the final benchmarking report produced by this application.

Step 7. Interpretation of statistics. Fields for the interpretation of statistics, which show cause and effect in terms of the observed performance and the practices that are responsible for this performance; the fields will be part of a complete structured benchmarking report and will have to be filled by the regional authorities experts.


Roadmap for the implementation of the benchmarking process

Figure 5 Roadmap for the implementation of the benchmarking process

References
  • Adebanjo, D., Abbas A. & Mann, R. (2010) An investigation of the adoption and implementation of benchmarking, International Journal of Operations & Production Management, Vol. 30, No. 11, pp. 1140 -1169.
  • Foray, D., Goddard, J., Goenaga, B. X., Landabaso, M., McCann, P., Morgan, K., Nauwelaers, C. & Ortega-Argilés, R. (2012) Guide to Research and Innovation Strategies for Smart Specialisation. European Commission.
  • Huggins, R. (2010) Regional Competitive Intelligence: Benchmarking and Policy-making, Regional Studies, Vol. 44, No. 5, pp. 639-658.
  • Iurcovich, L., Komninos, N., Reid, A., Heydebreck, P. & Pierrakis, Y. (2006) Blueprint for Regional Innovation Benchmarking, Mutual Learning Platform, Regional Benchmarking Report.
  • Kelessidis, V. (2000) Benchmarking in InnoRegio Consortium, 21 Innovation Management Technologies, European Commission, Directorate General Regional Policy, Recite Programme.
  • Koellreuter, C. (2002) Regional Benchmarking as a tool to improve regional foresight, Paper for the STRATA-ETAN Expert Group Action on “Mobilising regional foresight potential for an enlarged EU”, European Commission Research DG.
  • Kumar, S. and Chandra, C. (2001) Enhancing the effectiveness of benchmarking in manufacturing organizations, Industrial Management & Data Systems, Vol. 101, No. 2, pp.80-89.
  • Navarro, M., Gibaja, J.J., Franco, S., Murciego, A., Gianelle, C., Hegyi, F. B. and Kleibrink, A. (2014) Regional benchmarking in the smart specialization process: Identification of reference regions based on structural similarity, JRC Technical Reports, S3 Working Paper Series No. 03/2014.
  • OECD (2013) Innovation-driven Growth in Regions: The Role of Smart Specialisation. Organisation for Economic Co-Operation and Development, Committee for Scientific and Technological Policy.
  • OECD (2005) Micro-policies for growth and productivity, Synthesis and benchmarking user guide, Paris: OECD.
  • Yasin, M. (2002) The theory and practice of benchmarking: then and now, Benchmarking: An International Journal, Vol. 9, No. 3, pp.217-243.
  • [5] Benchmarking Regional Structure, Smart Specialisation Platform, Source: http://s3platform.jrc.ec.europa.eu/regional-benchmarking, [Access 22 August 2016].