1.2 Stakeholder engagement

1.2 Stakeholder engagement
Stakeholder engagement logo

Several applications (e.g. ideascale.com, allourideas.org or www.mywejit.com) have been designed and used to facilitate knowledge transfer, crowdsourcing and collaboration to enable consultative and deliberative processes (e.g., Lathrop and Ruma, 2010; De Cindio and Stortone, 2013; Castells, 2015). The method for facilitating stakeholder engagement in RIS 3 needs to provide opportunities to invite a diverse set of RIS3 stakeholders to use online deliberation functionalities specifically tailored to promote the entrepreneurial discovery process. Thus, the following elements are key for the anticipated method and application (sorted by importance with respect to RIS): (1) facilitate discussions; (2) co-creation procedures including provision of feedback (e.g. voting of priorities according to impact; plausibility; rank other people ideas thereby allowing the emergence of the most popular ideas); and (3) reputation management system (delegated voting system / “liquid democracy”) allowing voting that is “weighted” based on expertise in the field. Based on these requested fundamental features the open-source software LiquidFeedback appears to offer a promising method and application. LiquidFeedback offers a deliberative process over which proposed suggestions can be debated, voted, supported, and written in a collaborative way, or questioned. The application is built on the Schultze method, an improved version of the Condorcet’s one (Schulze, 2011).

The method applied by LiquidFeedback was considered and developed to support policymaking processes of a German Party (Domanski, 2012) but it has been also implemented by non-profit associations such as the “Interaktive Demokratie” or other activities toward civic engagement (De Cindio and Schuler, 2012). Furthermore, civil society organizations such as “Slow Food Germany” and local communities such as the “County of Friesland” have been taken advantage of LiquidFeedback for facilitating a broad participation and public deliberation. LiquidFeedback represents a platform to reform democracy. In particular, the web-based application provides a mix of direct and representative democracy, in other words a liquid democracy to boost knowledge exchange in a large civic context (De Cindio and Stortone, 2013).

To conclude, the following functionalities provide important highlights for the RIS context

  • The method delivers robust outcomes based on the stakeholders provided proposals, information, suggestion, and needs. Via LiquidFeedback, decisions can be made within an interactive democracy or it can be used as an innovative communication channel between different stakeholders and engagement level.
  • In RIS3, procedures should be adaptable to any topic. Thus, the entrepreneurial discovery process within RIS3 can be facilitated by providing information linked to this adaptability. Via nonbinding suggestions to binding decisions within the method’s functions creative processes among stakeholders are supported.
  • The method represents a direct channel for a broad participation. It can be used to poll the opinion of the public with both yes/no–questions but also allow citizens to rephrase questions and provide answers. In other words, it empowers citizens to co-create.
  • LiquidFeedback is an open-source software established by the “Public Software Group e. V.” to empower democratic decision-making processes online. This allows every individual an equal opportunity to contribute in the democratic process via a web browser. Stakeholders are not required to install the software.
  • Although the mapping exercise found no region that has used similar methods to LiquidFeedback in RIS3 process, based on the literature review (De Cindio and Stortone, 2013) this method appears to be a promising application to enhance the entrepreneurial discovery process via stakeholder engagement in RIS3.

Description of the method
LiquidFeedback is a free open-source software for (political) opinion formation and decision making, merging the direct and representative democracy approach. Features of the platform for proposal development and decision making such as the delegated voting system (“liquid democracy“) enables an innovative procedure of political representation and stakeholder engagement taking the knowledge inequality of its participants into consideration. The proxy voting feature generates power structures comparable to representative democracy.
The methods applied by LiquidFeedback support decision making based on a comprehensive depiction of the opinions from stakeholders without hierarchies. Everyone is encouraged to contribute their own ideas and initiatives. The methods are designed to work with a large set of stakeholders and groups. Conflicts are mitigated by the application of strict rules in a predefined process without moderator interference. Different settings for timings, quorums and supermajority requirements are possible. So-called “policies” for different types of decisions can be chosen. The outcomes provide rich information, suggestions, directive, or binding decisions (Behrens et al., 2014).
The required method represents an implementation of Liquid Democracy but also offers features for a unique proposition development system, where a diverse set of stakeholders may discuss and decide in a self-organized way (Bertone et al., 2015). This provides an appropriate framework for processes for entrepreneurial discovery in line with the recommendation of The World Bank (2016). The method encourages a culture of a public–private dialogue, expands public administration’s online presence, including in social media, and facilitates collaboration among stakeholders.
The description of the method can be summarized as follows:

  • a stakeholder driven and self-organized process with collective moderation
  • every stakeholder is allowed to propose ideas, initiate proposals or add suggestions
  • feedback is quantified and constructive
  • every stakeholder is allowed to initiate alternative proposals or suggestions
  • alternative initiatives are voted upon together using >Cloneproof Schwartz Sequential Dropping (Schulze Method; Schulze, 2011)

The methods are characterised by the following elements (>www.liquidfeedback.org):


  • Liquid democracy: Liquid Democracy can be referred to as “delegated” or “proxy voting”, an idea of transitive, revocable delegations by topic. Individuals’ votes possess a “liquid” authority meaning that the voter can delegate his/her vote to other stakeholders (e.g. another expert, colleague etc.) (Edick, 2015). The fundamental idea is a democratic arrangement in which issues are (strongly) suggested or decided by direct ballot. Votes can be flexible, delegated by theme. Allocations are transitive and can be withdrawn.
  • Collective moderation: All individuals receive equal rights in a scalable structured discussion process in which minorities are considered adequately. Every individual can propose discussions, ideas, suggestions etc.
  • Transparent decision process: Predefined rules, recorded votes and timings on decision processes are transparent. All relevant data is made available to all stakeholders in adequate formats. Thus, participants can validate the applied procedures. This also provides a protection against non-transparent lobbying.
  • Preferential voting: The applied method inspires stakeholders to suggest alternatives. A voting system enables stakeholders to express their (dis)agreements. The underlining mathematical environment eliminates vote–splitting. Furthermore, similar proposals don’t harm each other. In this way equal treatment of competing alternatives is ensured.

The basics of the methods – the functionalities

The method applied by LiquidFeedback is a deliberative application, with a rich and articulated structure as well as text-based user interface (De Cindio and Stortone, 2013). As illustrated in Figure 1, the tool categorises stakeholders’ contribution, so called “issues” into “units” and “areas”. This is built by an administrator. The alphabetically listed units and areas, sorted by number of contributors (participants weight), can be added or deactivated (partly hidden). Unit visibility can be restricted to registered stakeholders or open to the wider public (Bertone et al., 2015). Labels should “be chosen wisely, keeping in mind it should be as clear as possible to determine which subject area a new topic should be assigned to.” (Behrens et al., 2014; p. 124).

Figure 1 LiquidFeedback structure. Source: Bertone et al., 2015

Main objects and actions

Stakeholders participate by clicking the button of an area. Here, they can open an “issue”, which gets a numerical identifier (e.g., #3333). This issue represents a (preliminary) proposal to solve a problem by proposing a so called “suggestion” to solve it. The name can be chosen freely. Any proposal can be disputed by another stakeholder who proposes an alternative solution to the same issue. The initiatives may gain support or receive further suggestions. This will encourage the authors to modify their initial proposals. Stakeholders rank both their own and other individual’s suggestions for implementation (Behrens et al., 2014), making them (potential) supporters of the initiatives and suggestions. Authors can invite other stakeholders to co-create an initiative. (Counter)Proposals, which can be also reformulated and improved by the authors, can be supported and can gather suggestions. Figure 2 depicts an example for the items issue, initiatives, and suggestions.

Figure 2 Issues, initiatives and suggestions. Source: Bertone et al., 2015

The issue with its “support” actions such as initiatives, suggestions, and endorsements establishes a deliberative environment within a given area. Stakeholders announce interest in the issue directly, by ticking the relevant button, or indirectly, by being active in the issue’s environment. The participating stakeholder community in the different areas creates the “reference population” (Behrens et al., 2014; p. 71-72) whose size influences the issue’s deliberation path (Bertone et al., 2015).
Deliberation path

As illustrated in Figure 3, the proposed issue is based on a deliberation path of sequential stages entitled admission, discussion, verification, and voting. This path represents the “issue lifetime.” Shifts from one phase to another will be subject to the set policies setting regarding timing parameters and/or quorums on behalf of the administrator. Two quorums need to be reached during the deliberation path (during the admission and during the verification phase), which are measured as number of supporters based on the reference population. Thus, the state transition happens based on set conditions such as time (e.g., after seven days of discussion) and/or a “quorum” (e.g., if at least there are 40% supporters of the reference population) (Bertone et al., 2015).

Figure 3 The deliberation path. Source: Bertone et al., 2015

Figure 3 The deliberation path. Source: Bertone et al., 2015

The administrator chooses the settings for the policies, which will depend on the type of decision. When opening a new issue, the author chooses a policy to his/her area. Proposals which receive the necessary support are primarily frozen to allow other stakeholders to read the last version. Then they can vote. All the proposals referring to the same issue are voted on in one pool: voters can choose positive, neutral or negative opinions. Furthermore, priorities can be chosen. The “winning” (or approved) proposals represents the results (Bertone et al., 2015), which are measured based on the Schultze algorithm (Schulze, 2011).
Figure 4 Voting interface, generic example. Source: Bertone et al., 2015

Figure 4 Voting interface, generic example. Source: Bertone et al., 2015

If an initiative on an issue passes the first quorum during the admission phase, the whole issue with all its initiatives progress to the next step, the discussion phase. If not, it ends. Individuals can still contribute to the issue during the discussion stage for a given period of time. For instance, stakeholders can give/revoke support, suggest/ co-create alternatives, edit/revoke extant initiatives). A notification system empowers stakeholders to shadow the activities. After the discussion, the issue enters the verification phase. This means proposals are frozen as a final version and can no longer be changed. Now, the stakeholders have time to read this final version (Bertone et al., 2015).
The verification phase, which also lasts for a period of time, is for initiatives that achieve a set second quorum progress to the voting phase. In LiquidFeedback votes are sorted on a given issue. As illustrated in Figure 4, stakeholders can choose in favor (green) or neutral (gray) or against (red) for voting. The Schulze (2011) method regulates outcomes, e.g., using the example from Figure 2, “if 35% of voters support the park, 20% the sports center and 45% the shopping center, the latter would not win as in usual single-choice voting, since a clear majority favours some form of public leisure.” (Bertone et al., 2015, p. 5). This voting scheme allows stakeholder to express their different levels of agreements to several alternatives.
Delegating to allow for getting advice from experts

Via delegating, LiquidFeedback follows liquid democracy principles, “i.e. transitive proxy voting, a distinctive, innovative feature. Participants can delegate (and then revoke) other members to act (not only vote) on their behalf. These proxies may, in turn, choose other participants as proxies” (Bertone et al., 2015, p. 5). Overall, the method applied by LiquidFeedback enables three levels of delegation (Behrens et al., 2014; p. 26):

(1) Delegation for all issues in all subject areas, within a single unit;

(2) Delegation for all issues in a given subject area;

(3) Delegation for a single issue.

“Any finer delegation (e. g., for a particular issue) overrides a more general delegation (e. g., for the corresponding area). Any form of direct participation will suspend existing delegations” (Bertone et al., 2015, p. 5).


Usability and impact
Usability aspects of the method of LiquidFeedback on the quality of RIS3 process

  • Through a participatory contract stakeholders are asked to suggest proposals. The method’s / LiquidFeedback’s authentication policy requires stakeholders to register, which in turn is dependent on an invitation from the platform’s administrator. This process appears to be sufficient for ONLINE S3; however, authentication policies should be examined and, if necessary, improved to ensure adequate settings for future RIS contexts.


  • The configuration setting of the method comprises several parameters for shaping the deliberative path. This forms the transition from one stage to the subsequent stage. These parameters are: (1) a quorum for the proposal to start the “discussion” phase; (2) a quorum for progress to the vote phase; (3) the longest duration of each stage and (4) the minimum number of positive votes to be defined as an approved proposal. These parameters effect the number of approved proposals. These flexible settings are essential to adopting the RIS3 development phase to national and regional contexts in Europe. In the framework of a rather fast and competitive electoral campaign, the aim is to fulfil stakeholders’ anticipation of seeing their ideas and suggestions considered, while also fostering collaboration, co-creation and aggregation among stakeholders (De Cindio and Stortone, 2013).


  • Users are not required to install LiquidFeedback. Stakeholders can access it via a web browser. LiquidFeedback is platform independent.


  • Poor usability and an outdated graphical user interface is often the primary reason for not using LiquidFeedback (Bertone et al., 2015). This provides room for improvement within ONLINE S3.


  • LiquidFeedback is an open-source software and project and thus provides a suitable format to be embedded in the RIS3 process within ONLINE S3.


  • The method should ( and LiquidFeedback can) be embedded into a richer platform. It has a very “Spartan” interface, and runs on the open-source software platform openDCN (openDCN.org). Thus, social media sharing facilities are possible (De Cindio and Stortone, 2013). Broad participation is key in RIS3. Thus, this compatible environment facilitates RIS3’s objectives.

Impact #1: A book entitled “The Principles of LiquidFeedback” provides a comprehensive insight and background information related to the philosophical, political and technological aspects of decision making via LiquidFeedback. Within RIS this software design could empower stakeholder groups to make democratic decisions, giving every individual an equal chance to participate in a democratic process. The principles and rules of procedures developed for LiquidFeedback are explained in detail in this book, discussing the essential features for democratic self-organisation. These principles can be used for the RIS community as a point of reference too. (http://principles.liquidfeedback.org)
Impact #2: The Liquid Democracy Journal is devoted toward the democratic principle of Liquid Democracy that follows transitive delegations to represent direct and representative democracy. Apart from liquid democracy this journal is dedicated toward further topics such as electronic participation, collective moderation, and voting systems. This dissemination channel could also provide an opportunity for ONLINE S3 to disseminate its contributions. (http://www.liquid-democracy-journal.org)


Required data
LiquidFeedback (www.liquidfeedback.org and www.liquidfeedback.com) is an open-source software and an independent open source project. The software is published under MIT license by the Public Software Group of Berlin, Germany (http://www.public-software-group.org/licenses). The LiquidFeedback software and source code may be obtained free of charge, and there are no royalties. The source code can be downloaded from the website of the Public Software Group e.V. (dev.liquidfeedback.org/trac/lf and www.public-software-group.org/mercurial). “The Public Software Group’s liberal licensing model avoids license incompatibilities when merging the software with other software components. LiquidFeedback and its dependencies are based on MIT– and BSD–licensed components” (liquidfeedback.org).
“The LiquidFeedback Maintainers are granting access to the ​source code repository to developers. Contributors are kindly requested to make their contributions by email together with a feature proposal. Such patches will be reviewed by the maintainers and – if suitable for LiquidFeedback – incorporated” (dev.liquidfeedback.org).
The core comprises a database scheme for the PostgreSQL database, including the algorithms for delegations, feedback and the voting procedure implemented as SQL views and database procedures written in PL/pgSQL. As it is licensed under the liberal MIT/X11-License it can be included it in any software project, as long as there is PostgreSQL support for the programming language available (www.public-software-group.org). “The user frontend is provided by the LiquidFeedback Frontend. Currently the frontend accesses the SQL database directly and is thus also responsible for access control and locking” (dev.liquidfeedback.org).
The work title „LiquidFeedback“ is protected by law. If the software „LiquidFeedback“ is changed and distributed, the name „LiquidFeedback“ must not be used. LiquidFeeback is a registered trademark in the European Union and the United States of America and must not be used for commercial purposes without prior permission. „LiquidFeedback“ is properly quoted in CamelCase (e. g. in press, media, blogs, scientific work).


Relevant data sources
  • An ​installation guideline is part of the LiquidFeedback Frontend.
  • The main information page: ​http://liquidfeedback.org/
  • The Principles of LiquidFeedback: ​http://principles.liquidfeedback.org/
  • Website of the Public Software Group, publishing LiquidFeedback: ​http://public-software-group.org
  • Website of the Interaktive Demokratie, promoting the use of electronic media for democracy: http://interaktive-demokratie.org/
  • The Liquid Democracy Journal on electronic participation, collective moderation and participation: ​http://www.liquid-democracy-journal.org/

Software dependencies:

Frequently Asked Questions

  • http://www.liquid-democracy-journal.org/issue/2/The_Liquid_Democracy_Journal-
  • Issue002-08-Readers_Asked_-_LiquidFeedback_Developers_Answer_001.html


Implementation roadmap
The web application Liquid Feedback is a free open source product with no charges or licence fees. This tool is entirely web-based and emphasizes formal and structured feedback and voting processes, while leaving the means of informal and unstructured discussion to a web forum.
The applied method by which LiquidFeedback provides a mechanism for identification based on stated principles “it is not intended for pseudonymous use where participants within the system are hidden behind nicknames and only a special group of administrators know (or can guess) who really signed up” (Behrens et al., 2014; p. 121). After the responsible certifier has an adequate meeting with a stakeholder, the LiquidFeedback administrator creates the account for the user with the basic identification details. Then, an invitation code is sent to the stakeholder’s email address so that the new participant is able to complete further relevant information in the account settings. While the change of the identification name is not possible, individuals can select login name, password, and screen name/nickname on their own. This authentication removes the need for moderators, and LiquidFeedback can build on a “collective moderation” (Behrens et al., 2014; p. 63). Consequently, administrators have no user-management functions. Theoretically, nothing in the software stops administrators from generating accounts with unfilled identification details for potential stakeholders. Thus, any promise of a strong relationship between the created account and a real person would be lost. This offers room for improvement.
“After login, registered users operate in thematic areas through first-level activities (participating, declaring interest, delegating, creating new issues). Second-level activities are enabled for a given issue, depending on its phase. Logged-in users can view the identity of proponents, subscribers, supporters, and voters, who remain anonymous to non-registered users” (Bertone et al., 2015; p. 5).
The method applied by Liquid Feedback provides quantitative feedback for the proposers and an overview of supporters and opposers. It also delivers information under which conditions individuals are willing to change their mind. The individual participants can click to indicate agreement or disagreement. Furthermore, the system tracks the different draft versions of a proposal and allows the user to highlight the differences between them with a simple click. This method of structured feedback arranges communication among stakeholder – the creators of a proposal and the voters. The proposers get an idea how successful a proposition will turn out to be, and what to change in order to gain more support. Likewise, voters can influence the propositions via feedback and co-creation, or instigate a new initiative with a counter proposition. Overall, the system makes use of the innovative and established Schulze Method (Schulze, 2011) voting system which allows for a winner to be selected based on preferences. This means the voter only needs to indicate what proposals and amendments he or she likes and what proposals he or she dislikes – he or she can also rank them and support more than one proposal.
Another innovative feature of the method applied by LiquidFeedback is the transitive proxy voting in which stakeholders are able to delegate (and revoke) other stakeholders to propose alternatives and vote on their behalf in specific thematic areas and/or issues in which they are particular experts (De Cindio and Stortone, 2013).
Due to the applied functions LiquidFeedback can be used as an idea-gathering tool. Furthermore, its deliberative and collaborative nature supports the fair selection of submitted proposals and enabled co-operation and co-creation (De Cindio and Stortone, 2013). Although there are other supportive tools for idea gathering (e.g. ideascale.com), several contain weaknesses in the deliberative mechanisms to boost collaboration and co-design among stakeholders (De Cindio and Stortone, 2013)


  • Behrens, J., Kistner, A., Nitsche, A., Swierczek, B., 2014. The Principles of Liquidfeedback. Interaktive Demokratie e. V., Berlin.
  • Bertone, G., De Cindio, F., Stortone, S., 2015. Liquid Feedback in Large-Scale Civic Contexts: Framing Multiple Styles of Online Participation. Journal of Social Media for Organizations 2, 1.
  • Castells, M., 2015. Networks of Outrage and Hope: Social Movements in the Internet Age. Berkeley: John Wiley and Sons.
  • De Cindio, F., Schuler, D., 2012. Beyond Community Networks: From Local to Global, from Participation to Deliberation. The Journal of Community Informatics 8, http://www.ci-journal.net/index.php/ciej/article/view/908/933.
  • De Cindio, F., Stortone, S., 2013. Experimenting Liquidfeedback for Online Deliberation in Civic Contexts, International Federation for Information Processing, in LECTURE NOTES IN COMPUTER SCIENCE. Electronic Participation by Springer, Heidelberg, 147-158.
  • Domanski, D., 2012. Democratization through Social Innovation-Pirate Party Germany and New Methods of Civic Participation, The Second ISA Forum of Sociology. Second ISA Forum, Buenos Aires, Argentina.
  • Edick, C., 2015. The Golden Age of Piracy: Can Open-Source Democracy Redefine Citizenship for the Internet Age? Harvard International Review 36, 7.
  • Lathrop, D., Ruma, L., 2010. Open Government: Collaboration, Transparency, and Participation in Practice. O’Reilly Media, Inc., Safari Books Online.
  • Schulze, M., 2011. A New Monotonic, Clone-Independent, Reversal Symmetric, and Condorcet-Consistent Single-Winner Election Method. Social Choice and Welfare 36, 267-303.
  • The World Bank (2016). Toward an innovative Poland: The entrepreneurial discovery process and business needs analysis. Poland: Ministry of Economic Development. Accessed on 08.11.2016 http://documents.worldbank.org/curated/en/801221468186841613/pdf/106148-REPLACEMENT-v2-English-REPORT-Web.pdf.