Skip to main content
SpringerLink
Log in

Closing the Evaluation Gap in e-Participation Research and Practice

  • Chapter
  • First Online:
Evaluating e-Participation

Part of the book series: Public Administration and Information Technology ((PAIT,volume 19))

Abstract

This chapter points out the upswing of citizen participation, the emergence of a broad range of participation forms, and the high expectations of the potentials of e-participation. Against this background, a twofold evaluation gap is identified: a lack of acknowledged success criteria and indicators and a lack of empirical studies analyzing, differentiating, and comparing ecologies of e-participation instead of undertaking isolated case studies. The second part reviews major types of evaluation criteria and different conceptual frameworks for evaluating e-participation processes. It concludes with a twofold “relativity theory” of evaluation and proposes an adapted Input–Activities–Output–Outcome–Impact model for the comparative evaluation of e-participation through a quasi-experimental field study design.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    In 2009, the Committee of Ministers of the Council of Europe adopted the recommendations of the Ad hoc Committee on e-democracy (CAHDE), including guidelines and principles as well as an accompanying document on practical tools, to which two of the editors of this book made their contributions. http://www.coe.int/t/dgap/democracy/activities/ggis/cahde/default_EN.asp. Accessed 27 July 2015

  2. 2.

    See http://ec.europa.eu/environment/aarhus/legislation.htm. Accessed 27 July 2015

  3. 3.

    See http://www.sustainablecities.eu/aalborg-process/. Accessed 27 July 2015

  4. 4.

    Proponents of direct democracy most often refer to the theory of deliberative discourse and the public sphere by Jürgen Habermas (1996; summary by Chambers 2003).

  5. 5.

    See http://www.demo-net.org/. Accessed 27 July 2015

  6. 6.

    See http://www.un.org/en/events/publicserviceday/award.shtml. Accessed 27 July 2015

  7. 7.

    See http://www.unpan.org/DPADM/UNPSDayAwards/UNPublicServiceAwards/tabid/1095/language/en-US/Default.aspx. Accessed 27 July 2015

  8. 8.

    See http://www.epractice.eu/files/download/Awards2007SubmissionGuidanceNotes_en.pdf. Accessed 29 May 2014

  9. 9.

    See http://www.epsa2011.eu/en/content/show/&tid=92. Accessed 27 July 2015

  10. 10.

    See http://www.e-democracy.org/uk/. Accessed 27 July 2015

  11. 11.

    A more elaborated version is contained in Aichholzer and Westholm (2009).

  12. 12.

    This model has been developed for assessing environmental projects within the UN Environmental Programme and offers the possibility of looking at the relation of inputs to different kinds of results (output, outcome, and impact). The earliest source is probably the Performance Monitoring Indicators Handbook by the World Bank (Mosse and Sontheimer 1996). For small differences in definitions by the European Commission see http://ec.europa.eu/europeaid/evaluation/methodology/glossary/glo_en.htm. Accessed July 27, 2015.

  13. 13.

    See http://www.footprintnetwork.org (accessed 27 July 2015) and Chap. 8 for details.

References

  • Abelson J, Gauvin F-P (2006) Assessing the impacts of public participation: concepts, evidence and policy implications. Canadian Policy Research Networks Inc (CPRN), Ottawa

    Google Scholar 

  • Aichholzer G, Westholm H (2009) Evaluating eParticipation projects. Practical examples and outline of an evaluation framework. Eur J ePractice 7:27–44. https://joinup.ec.europa.eu/sites/default/files/91/da/cc/ePractice%20Journal-Vol.7-March%202009.pdf. Accessed 27 July 2015

    Google Scholar 

  • Andersen K, Henriksen HZ, Secher C, Medaglia R (2007) Costs of eparticipation: the management challenges. Transform Govern: People Process Policy 1(1):29–43

    Google Scholar 

  • Arnstein SR (1971) A ladder of citizen participation. J Am Plan Assoc 35(4):216–224. Reprinted in: Journal of the Royal Town Planning Institute, April 1971

    Google Scholar 

  • Auškalnienė L (2012) Assessing participation online: youth and their involvement in social media. Informacijos Mokslai 59:105–116

    Google Scholar 

  • Bicking M, Triantafillou A, Henderson F, Koussouris S, Wimmer MA (2011) Lessons from monitoring and assessing EC-funded eParticipation projects: citizen engagement and participation impact. IST-Africa 2011 Conference Proceedings (pp 1–8)

    Google Scholar 

  • Brooks J, Waylen KA, Mulder MB (2013) Assessing community-based conservation projects: a systematic review and multilevel analysis of attitudinal, behavioral, ecological, and economic outcomes. Environ Evid 2(2):1–34

    Google Scholar 

  • Chambers S (2003) Deliberative democratic theory. Annu Revi Polit Sci 6:307–326

    Article  Google Scholar 

  • Coleman S, Goetze J (2001) Bowling together. Online public engagement in policy deliberation. Hansard Society, London, BT, UK

    Google Scholar 

  • Congress of Local and Regional Authorities (2000) Guidelines for a policy on citizens’ responsible participation in municipal and regional life. Resolution 91. Council of Europe, Strasbourg

    Google Scholar 

  • Congress of Local and Regional Authorities (2008a) Electronic democracy and deliberative consultation on urban projects. Resolution 267. Council of Europe, Strasbourg

    Google Scholar 

  • Congress of Local and Regional Authorities (2008b) Electronic democracy and deliberative consultation on urban projects. Recommendation 249. Council of Europe, Strasbourg

    Google Scholar 

  • Council of Europe (2001) Recommendation Rec(2001)19 of the Committee of Ministers to member states on the participation of citizens in local public life. Council of Europe, Strasbourg

    Google Scholar 

  • Council of Europe (2009) Recommendation CM/Rec(2009)1 of the Committee of Ministers to member states on electronic democracy (e-democracy). Council of Europe, Strasbourg

    Google Scholar 

  • Creasy S, Gavelin K, Fisher H, Holmes L, Desai M (2007) Engage for change: the role of public engagement in climate change policy. The results of research undertaken for the Sustainable Development Commission. Involve. http://www.sd-commission.org.uk/publications.php?id=618. Accessed 27 July 2015

  • Dahl R (1994) A democratic dilemma: system effectiveness versus citizen participation. Polit Sci Quart 109(1):23–34

    Article  Google Scholar 

  • Dahlgren P (2012) Social media and counter-democracy: the contingences of participation. In: Tambouris E, Macintosh A, Sæbø Ø (eds) Electronic participation. Springer Berlin Heidelberg, Berlin, pp 1–12. http://link.springer.com/chapter/10.1007/978-3-642-33250-01. Accessed 27 July 2014

    Chapter  Google Scholar 

  • Department for Communities and Local Government (2008) Communities in control: real people, real power. The Stationery Office, White Paper London

    Google Scholar 

  • European Commission (2010) Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions. A Digital Agenda for Europe (COM/2010/0245 final). http://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=COM:2010:0245:FIN:EN:HTML. Accessed 27 July 2015

  • Frewer LJ, Rowe G (2005) Evaluating public participation exercises: strategic and practical issues. In: OECD (ed) Evaluating public participation in policy making. OECD, Paris, pp 85–108

    Google Scholar 

  • Geissel B (2012) Impacts of democratic innovations in Europe: findings and desiderata. In: Geissel B, Newton K (eds) Evaluating democratic innovations. Curing the democratic malaise? Routledge, New York, pp 163–183

    Google Scholar 

  • Gelders D, Brans M, Maesschalck J, Colsoul N (2010) Systematic evaluation of public participation projects: analytical framework and application based on two Belgian neighborhood watch projects. Govern Info Quart 27(2):134–140

    Article  Google Scholar 

  • Gibson R, Cantijoch M (2013) Conceptualizing and measuring participation in the Age of the Internet: is online political engagement really different to offline? J Polit 75(3):701–716

    Article  Google Scholar 

  • Green Mountain Institute for Environmental Democracy (2005) Environmental Democracy—What’s in it for me? Montpellier, France, VT. http://www.gmied.org/files/EDforme.pdf. Accessed 27 July 2015

  • Habermas J (1996) Between facts and norms. Contributions to a discourse theory of law and democracy. MIT Press, Cambridge

    Google Scholar 

  • IAP2 (2006)—International Association for Public Participation. Public Participation Toolbox. http://www.iap2.org/associations/4748/files/toolbox.pdf. Accessed 27 July 2015

  • IAP2 (2007)—International Association for Public Participation. Spectrum of Public Participation. http://www.iap2.org/displaycommon.cfm?an=5. Accessed 27 July 2015

  • Innes JE, Booher DE (2004) Reframing public participation: strategies for the twenty-first century. Plan Theor Pract 5(4):419–436

    Article  Google Scholar 

  • Irvin RA, Stansbury J (2004) Citizen participation in decision making: is it worth the effort? Public Admin Rev 64(1):55–65

    Article  Google Scholar 

  • Kubicek H (2007) Electronic Democracy: Achievements and Challenges. Final Report on the ESF Research Conference in Vadstena, Sweden, 21–25 November 2007.http://www.ifib.de/publikationsdateien/ESF_e-democracy_Report_2008.pdf. Accessed 25 July 2015

  • Kubicek H (2010) The potential of e-participation in urban planning: a European perspective. In: Silva CN (ed) Handbook of research on e-planning: ICTs for urban development and monitoring. IGI Global, Hershey, pp 168–194

    Chapter  Google Scholar 

  • Kubicek H, Westholm H (2010) Consensus building by blended participation in a local planning process. The case of the public swimming pool in Bremen. In: Insua DR, French S (eds) E-democracy: a group decision and negotiation perspective. Springer, Dordrecht, pp 323–341

    Chapter  Google Scholar 

  • Kubicek H, Lippa B, Westholm H (2009) Medienmix in der lokalen Demokratie. edition sigma, Berlin

    Google Scholar 

  • Kubicek H, Lippa B, Koop A (2011) Erfolgreich beteiligt? Nutzen und Erfolgsfaktoren internetgestützter Bürgerbeteiligung: Eine empirische Analyse von 12 Fallbeispielen. Verlag Bertelsmann Stiftung, Gütersloh

    Google Scholar 

  • Lippa B, Aichholzer G, Allhutter D, Freschi AC, Macintosh A, Moss G, Westholm H (2008) eParticipation Evaluation and Impact. DEMO-net. D13.3 booklet. http://www.ifib.de/publikationsdateien/DEMOnet_booklet_13.3_eParticipation_evaluation.pdf. Accessed 27 July 2015

  • London S (1995) Teledemocracy vs. deliberative democracy: a comparative look at two models of public talk. J Interpers Comput Technol 3(2):33–55. http://www.scottlondon.com/reports/tele.html. Accessed 27 July 2015

    Google Scholar 

  • Macintosh A (2003) Using information and communication technologies to enhance citizen engagement in the policy process. In: OECD (ed) Promise and problems of e-democracy: challenges of online citizen engagement. OECD, Paris, pp 19–142

    Google Scholar 

  • Macintosh A (2004) Characterizing e-participation in policy-making. Proceedings of the 37th Hawaii international conference on system sciences track 5—vol 5. IEEE Computer Society, Washington, DC. http://www.computer.org/csdl/proceedings/hicss/2004/2056/05/205650117a.pdf. Accessed 27 July 2015

    Google Scholar 

  • Macintosh A, Coleman S (2006) Multidisciplinary roadmap and report on eParticipation research. Demo-net D4.2. http://itc.napier.ac.uk/ITC/Documents/Demo-net%204_2_multidisciplinary_roadmap.pdf. Accessed 27 July 2015

  • Macintosh A, Whyte A (2006) Evaluating how e-participation changes local democracy. eGovernment Workshop ‘06 (eGOV06), 11 September 2006. Brunel University, London

    Google Scholar 

  • Macintosh A, Whyte A (2008) Towards an evaluation framework for eParticipation. Transform Govern: People Process Policy 2(1):16–30

    Article  Google Scholar 

  • Macintosh A, Coleman S, Lalljee M (2005) E-Methods for public engagement: helping local authorities to communicate with citizens. Bristol City Council, Bristol. http://itc.napier.ac.uk/ITC/Documents/eMethods_guide2005.pdf. Accessed 27 July 2015

    Google Scholar 

  • Märker O, Wehner J (2013) E-Partizipation—Politische Beteiligung als statistisches Ereignis. In: Passoth J-H, Wehner J (eds) Quoten, Kurven und Profile, Medien—Kultur—Kommunikation. Springer, Wiesbaden, pp 273–291 Fachmedien

    Chapter  Google Scholar 

  • Medaglia R (2011) eParticipation research: a longitudinal overview. In Tambouris E, Macintosh A, de Bruijn H (eds) Electronic participation. Springer Berlin Heidelberg, Berlin, pp 99–108. http://link.springer.com/chapter/10.1007/978-3-642-23333-39. Accessed 27 July 2015

    Chapter  Google Scholar 

  • Millard J, Meyerhoff Nielsen M, Warren R, Smith S, Macintosh A, Tarabanis K, Tambouris E, Panopoulou E, Efpraxia D, Parisopoulos K (2009) European eParticipation summary report. European Commission, Brussels. http://ec.europa.eu/information_society/newsroom/cf/dae/document.cfm?doc_id=1499. Accessed 27 July 2015

    Google Scholar 

  • Ministerial Declaration on eGovernment (2009) Brussels: European Commission. https://ec.europa.eu/digital-agenda/sites/digital-agenda/files/ministerial-declaration-on-egovernment-malmo.pdf. Accessed 27 July 2015

  • Morozov E (2011) The net delusion. How not to liberate the world. Penguin Books, London

    Google Scholar 

  • Mosse R, Sontheimer LE (1996) Performance monitoring indicators handbook. World Bank Technical Paper No. 334, Sept. The World Bank, Washington

    Google Scholar 

  • OECD (2001a) Citizens as partners: OECD handbook on information, consultation and public participation in policy-making. OECD, Paris

    Google Scholar 

  • OECD (2001b) Engaging citizens in policy-making: information, consultation and public participation. PUMA policy brief No. 10. OECD, Paris

    Google Scholar 

  • OECD (2002) Glossary of key terms in evaluation and results based management. OECD, Paris. http://www.oecd.org/development/peer-reviews/2754804.pdf. Accessed 27 July 2015

    Google Scholar 

  • OECD (2003) Promise and problems of E-democracy: challenges of online citizen engagement. OECD, Paris

    Google Scholar 

  • OECD (2005) Evaluating public participation in policy making. OECD, Paris

    Google Scholar 

  • Office of the President (2009) Transparency and open government. Memorandum of January 21. Memorandum for the heads of executive departments and agencies. Fed Regist 74(15):4658 Presidential Documents

    Google Scholar 

  • Oser J, Hooghe M, Marien S (2012) Is online participation distinct from offline participation? A latent class analysis of participation types and their stratification. Political Res Quart 66(1):91–101

    Article  Google Scholar 

  • Porwol L, Ojo A, Breslin J (2013) Structuring e-participation perspectives—mapping and aligning models to core facets. The proceedings of the 14th annual international conference on digital government research, 224–234 (doi:10.1145/2479724. 2479756)

    Google Scholar 

  • Pratchett L, Karakaya Polat R, Wingfield M (2005) Barriers to e-democracy: local government experiences and responses. Office of the Deputy Prime Minister (National Project on Local e-Democracy), London

    Google Scholar 

  • Pratchett L, Durose C, Lowndes V, Smith G, Stoker G, Wales C (2009) Empowering communities to influence local decision making. A systematic review of the evidence. Department for Communities and Local Government, London. http://www.ict-21.ch/com-ict/IMG/pdf/1241955Empowering-communities%20to-influence-local-decision-making.pdf. Accessed 27 July 2015

    Google Scholar 

  • Quittkat C (2011) The European commission’s online consultations: a success story? J Common Market Stud 49(3):653–674

    Article  Google Scholar 

  • Riehm U, Böhle K, Lindner R (2013) Elektronische Petitionssysteme. Analysen zur Modernisierung des parlamentarischen Petitionswesens in Deutschland und Europa. edition sigma, Berlin

    Google Scholar 

  • Rowe G, Frewer LG (2000) Public participation methods: a framework for evaluation. Sci Technol Human Values 25(1):3–29

    Article  Google Scholar 

  • Rowe G, Frewer LG (2004) Evaluating public participation exercises: a research agenda. Sci Technol Hum Val 29(4):512–556

    Article  Google Scholar 

  • Royal Town Planning Institute (RTPI) (2007) Guidelines on Effective Community Involvement and Consultation (Revised). RTPI Good Practice Note 1, London, UK. http://www.rtpi.org.uk/media/6313/Guidlelines-on-effective-community-involvement.pdf. Accessed 27 July 2015

  • Sæbø Ø, Rose J, Nyvang T (2009) The role of social networking services in eParticipation. In: Macintosh A, Tambouris E (eds) Electronic participation. Springer Berlin Heidelberg, Berlin, pp 46–55. http://link.springer.com/chapter/10.1007/978-3-642-03781-85. Accessed 27 July 2015

    Chapter  Google Scholar 

  • Schlozman KL, Verba S, Brady HE (2010) Weapon of the strong? Participatory inequality and the Internet. Perspect Politics 8(2):487–509

    Article  Google Scholar 

  • Smith G (2009) Democratic innovations. Designing institutions for citizen participation. Cambridge University Press, Cambridge

    Book  Google Scholar 

  • Smith S, Macintosh A, Millard J (2011) A three-layered framework for evaluating e-Participation. Int J Electron Gover 4(4):304–321

    Article  Google Scholar 

  • Tambouris E, Kalampokis E, Tarabanis K (2008) A survey of eParticipation research projects in the European Union. Int J Electron Bus 6(6):554–571

    Article  Google Scholar 

  • Tambouris E, Macintosh A, Smith S, Panopoulou E, Tarabanis K, Millard J (2012) Understanding eParticipation state of play in Europe. Inf Syst Manag 29(4):321–330

    Article  Google Scholar 

  • UN—United Nations (2012) E-Government Survey 2012: E-government for the people. http://unpan3.un.org/egovkb/en-us/Reports/UN-E-Government-Survey-2012. Accessed 27 July 2015

  • UNECE—United Nations Economic Commission for Europe (1998) Convention on Access to Information, Public Participation in Decision-making and Access to Justice in Environmental Matters, done at Aarhus, Denmark, on 25 June 1998. http://www.unece.org/fileadmin/DAM/env/pp/documents/cep43e.pdf. Accessed 27 July 2015

  • van Dijk J (2012) The network society, 3rd edn. Sage, London

    Google Scholar 

  • Wiedemann PM, Femers S (1993) Public participation in waste management decision-making. J Hazard Mat 33(3):355–368

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Information Management Bremen (ifib), University of Bremen, Bremen, Germany

    Herbert Kubicek

  2. Institute of Technology Assessment, Austrian Academy of Sciences, Strohgasse 45, 1030, Vienna, Austria

    Georg Aichholzer

Authors
  1. Herbert Kubicek
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Georg Aichholzer
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Herbert Kubicek .

Editor information

Editors and Affiliations

  1. Austrian Academy of Sciences, Institute of Technology Assessment Austrian Academy of Sciences, Vienna, Austria

    Georg Aichholzer

  2. Institute for Information Management, University of Bremen Institute for Information Management, Bremen, Germany

    Herbert Kubicek

  3. Dept. Accounting and Finance, University of Saragossa Dept. Accounting and Finance, Saragossa, Spain

    Lourdes Torres

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Kubicek, H., Aichholzer, G. (2016). Closing the Evaluation Gap in e-Participation Research and Practice. In: Aichholzer, G., Kubicek, H., Torres, L. (eds) Evaluating e-Participation. Public Administration and Information Technology, vol 19. Springer, Cham. https://doi.org/10.1007/978-3-319-25403-6_2

Download citation

Publish with us

Policies and ethics

Search

Navigation