Ramine Tinati
Max Van Kleek
Markus Luczak-Rösch
ABSTRACT
Designing an effective and sustainable citizen science (CS)project requires consideration of a great number of factors. This makes the overall process unpredictable, even when a sound, user-centred design approach is followed by an experienced team of UX designers. Moreover, when such systems are deployed, the complexity of the resulting interactions challenges any attempt to generalisation from retrospective analysis. In this paper, we present a case study of the largest single platform of citizen driven data analysis projects to date, the Zooniverse. By eliciting, through structured reflection, experiences of core members of its design team, our grounded analysis yielded four sets of themes, focusing on Task Specificity, Community Development, Task Design and Public Relations and Engagement, supported by two-to-four specific design claims each. For each, we propose a set of design claims (DCs), drawing comparisons to the literature on crowdsourcing and online communities to contextualise our findings.
- Arguello, J., Butler, B., and Joyce, E. Talk to me: foundations for successful individual-group interactions in online communities. Proceedings of the ... (2006), 1--10. Google Scholar
Digital Library
- Backstrom, L., Huttenlocher, D., Kleinberg, J., and Lan, X. Group formation in large social networks: Membership, growth, and evolution. In Proceedings of the 12th ACM SIGKDD, KDD '06, ACM (New York, NY, USA, 2006), 44--54. Google ScholarDigital Library
- Bruckman, A., and Resnick, M. The mediamoo project constructionism and professional community. Convergence: The International Journal of Research into New Media Technologies 1, 1 (1995), 94--109.Google Scholar
- C, B. Helping to get you started reflecting on your experiences, 3rd edition ed. Oxford Blackwell, 2004.Google Scholar
- Cardamone, C., Schawinski, K., Sarzi, M., Bamford, S. P., Bennert, N., Urry, C., Lintott, C., Keel, W. C., Parejko, J., Nichol, R. C., et al. Galaxy zoo green peas: discovery of a class of compact extremely star-forming galaxies. Monthly Notices of the Royal Astronomical Society 399, 3 (2009), 1191--1205.Google ScholarCross Ref
- Cardamone, C. N. The story of the peas: Writing a scientific paper. http://blog.galaxyzoo.org/2009/07/02/the-storyof-the-peas-writing-a-scientific-paper, 2009.Google Scholar
- Cooper, C. Zen in the art of citizen science: Apps for collective discovery and the 1 percent rule of the web. http://blogs.scientificamerican.com/guestblog/2013/09/11/zen-in-the-art-of-citizenscience-apps-for-collective-discovery-and-the1-rule-of-the-web/, 2013.Google Scholar
- Cordero, P., Lucks, J. B., and Das, R. An rna mapping database for curating rna structure mapping experiments. Bioinformatics 28, 22 (2012), 3006--3008. Google ScholarDigital Library
- Dawson, R., and Bynghall, S. Getting Results from Crowds. Advanced Human Technologies, 2012.Google Scholar
- Druschke, C. G., and Seltzer, C. E. Failures of engagement: Lessons learned from a citizen science pilot study. Applied Environmental Education & Communication 11, 3--4 (2012), 178--188.Google ScholarCross Ref
- Ebner, W., Leimeister, J. M., and Krcmar, H. Community engineering for innovations: the ideas competition as a method to nurture a virtual community for innovations. R&D Management 39, 4 (2009), 342--356.Google ScholarCross Ref
- Fortson, L., Masters, K., Nichol, R., Borne, K., Edmondson, E., Lintott, C., Raddick, J., Schawinski, K., and Wallin, J. Galaxy zoo: Morphological classification and citizen science. arXiv preprint arXiv:1104.5513 (2011).Google Scholar
- Harper, F. M., Moy, D., and Konstan, J. A. Facts or friends?: Distinguishing informational and conversational questions in social q&a sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '09, ACM (New York, NY, USA, 2009), 759--768. Google ScholarDigital Library
- Hassman, K. D., Mugar, G., Østerlund, C., and Jackson, C. Learning at the seafloor, looking at the sky: The relationship between individual tasks and collaborative engagement in two citizen science projects. In Computer Supported Collaborative Learning (2013).Google Scholar
- Heinzelman, J., and Waters, C. Crowdsourcing crisis information in disaster-affected haiti. http://www.humanitarianforum.org/data/files/ crowdsourcinghaiti.pdf, 2010.Google Scholar
- Hilliard, C. Using structured reflection on a critical incident to develop a professional portfolio. Nursing standard 21, 2 (2006), 35--40.Google Scholar
- http://OpenScientist.org. Finalizing a definition of "citizen science" and "citizen scientists". http://www.openscientist.org/2011/09/ finalizing-definition-of-citizen.html.Google Scholar
- Kairam, S. R., Wang, D. J., and Leskovec, J. The life and death of online groups: Predicting group growth and longevity. In Proceedings of the Fifth ACM International Conference on Web Search and Data Mining, WSDM '12, ACM (New York, NY, USA, 2012), 673--682. Google ScholarDigital Library
- Kaufmann, N., Schulze, T., and Veit, D. More than fun and money. worker motivation in crowdsourcing-a study on mechanical turk. In AMCIS (2011).Google Scholar
- Kawrykow, A., Roumanis, G., Kam, A., Kwak, D., Leung, C., Wu, C., Zarour, E., Sarmenta, L., Blanchette, M., Waldispühl, J., et al. Phylo: a citizen science approach for improving multiple sequence alignment. PloS one 7, 3 (2012), e31362.Google Scholar
- Khatib, F., Cooper, S., Tyka, M. D., Xu, K., Makedon, I., Popović, Z., Baker, D., and Players, F. Algorithm discovery by protein folding game players. Proceedings of the National Academy of Sciences 108, 47 (2011), 18949--18953.Google ScholarCross Ref
- Kittur, A., and Kraut, R. E. Harnessing the wisdom of crowds in wikipedia: Quality through coordination. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work, CSCW '08, ACM (New York, NY, USA, 2008), 37--46. Google ScholarDigital Library
- Kluger, A. N., and DeNisi, A. The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological bulletin 119, 2 (1996), 254.Google Scholar
- Kosmala, M. We need an 'i dont know' button! http://blog.snapshotserengeti.org/2012/12/14/ we-need-an-i-dont-know-button/, 2012.Google Scholar
- Kraut, R. E., Resnick, P., Kiesler, S., Burke, M., Chen, Y., Kittur, N., Konstan, J., Ren, Y., and Riedl, J. Building successful online communities: Evidence-based social design. Mit Press, 2012. Google ScholarCross Ref
- Krieger, M., and Stark, E. Coordinating tasks on the commons: designing for personal goals, expertise and serendipity. Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI) (2009), 1485--1494. Google ScholarDigital Library
- Kuittinen, J., Kultima, A., Niemelä, J., and Paavilainen, J. Casual games discussion. In Proceedings of the 2007 Conference on Future Play, Future Play '07, ACM (New York, NY, USA, 2007), 105--112. Google ScholarDigital Library
- Kumar, R., Novak, J., and Tomkins, A. Structure and evolution of online social networks. Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '06 (2006), 611. Google ScholarDigital Library
- Lampe, C., Wash, R., Velasquez, A., and Ozkaya, E. Motivations to participate in online communities. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2010), 1927--1936. Google ScholarDigital Library
- Law, E., and von Ahn, L. Input-agreement: A new mechanism for collecting data using human computation games. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '09, ACM (2009), 1197--1206. Google ScholarDigital Library
- Lintott, C. J., et al. Galaxy zoo: morphologies derived from visual inspection of galaxies from the sloan digital sky survey. Monthly Notices of the Royal Astronomical Society 389, 3 (2008), 1179--1189.Google ScholarCross Ref
- Lintott, C. J., et al. Galaxy zoo:hanny's voorwerp, a quasar light echo?. Monthly Notices of the Royal Astronomical Society 399, 1 (2009), 129--140.Google ScholarCross Ref
- Lukyanenko, R., Parsons, J., and Wiersma, Y. F. The iq of the crowd: Understanding and improving information quality in structured user-generated content. Information Systems Research 25, 4 (2014), 669--689.Google ScholarDigital Library
- Mazzolini, M., and Maddison, S. Sage, guide or ghost? the effect of instructor intervention on student participation in online discussion forums. Computers & Education 40, 3 (2003), 237--253. Google ScholarDigital Library
- Milligan, C., Littlejohn, A., and Margaryan, A. Patterns of engagement in connectivist moocs. MERLOT Journal of Online Learning and Teaching 9, 2 (2013).Google Scholar
- Mugar, G., Osterlund, C., Hassman, K. D., Crowston, K., and Jackson, C. B. Planet hunters and seafloor explorers: Legitimate peripheral participation through practice proxies in online citizen science. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work, CSCW '14, ACM (New York, NY, USA, 2014), 109--119. Google ScholarDigital Library
- Murphy, E., and Coleman, E. Graduate students' experiences of challenges in online asynchronous discussions. Canadian Journal of Learning and Technology 30, 2 (2004).Google ScholarCross Ref
- Norman, D. A. The design of everyday things. Basic books, 2002. Google ScholarDigital Library
- Ockerman, J., and Pritchett, A. A review and reappraisal of task guidance: Aiding workers in procedure following. International Journal of Cognitive Ergonomics 4, 3 (2000), 191--212.Google ScholarCross Ref
- Oleson, D., Sorokin, A., Laughlin, G. P., Hester, V., Le, J., and Biewald, L. Programmatic gold: Targeted and scalable quality assurance in crowdsourcing. In Human Computation, vol. WS-11--11 of AAAI Workshops, AAAI (2011).Google Scholar
- Raddick, J., Lintott, C., Bamford, S., Land, K., Locksmith, D., Murray, P., Nichol, B., Schawinski, K., Slosar, A., Szalay, A., et al. Galaxy zoo: Motivations of citizen scientists. In Bulletin of the American Astronomical Society, vol. 40 (2008), 240.Google Scholar
- Raddick, M. J., Bracey, G., Gay, P. L., Lintott, C. J., Murray, P., Schawinski, K., Szalay, A. S., and Vandenberg, J. Galaxy zoo: Exploring the motivations of citizen science volunteers. Astronomy Education Review 9 (2010), 010103.Google ScholarCross Ref
- Rotman, D., Preece, J., Hammock, J., Procita, K., Hansen, D., Parr, C., Lewis, D., and Jacobs, D. Dynamic changes in motivation in collaborative citizen-science projects. In Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, ACM (2012), 217--226. Google ScholarDigital Library
- Schroer, J., and Hertel, G. Voluntary engagement in an open web-based encyclopedia: Wikipedians and why they do it. Media Psychology 12, 1 (2009), 96--120.Google ScholarCross Ref
- Schwamb, M. E., Orosz, J. A., Carter, J. A., Welsh, W. F., Fischer, D. A., Torres, G., Howard, A. W., Crepp, J. R., Keel, W. C., Lintott, C. J., et al. Planet hunters: A transiting circumbinary planet in a quadruple star system. The Astrophysical Journal 768, 2 (2013), 127.Google ScholarCross Ref
- Silvertown, J. A new dawn for citizen science. Trends in ecology & evolution 24, 9 (2009), 467--471.Google Scholar
- Szell, M., Lambiotte, R., and Thurner, S. Multirelational organization of large-scale social networks in an online world. Proceedings of the National Academy of Sciences 107, 31 (2010), 13636--13641.Google ScholarCross Ref
- Thaler, S., Simperl, E., Siorpaes, K., and Wlger, S. SpotTheLink: A Game-based Approach to the Alignment of Ontologies. IGI Global, 2012.Google ScholarCross Ref
- Tinati, R., Luczak-Roesch, M., Simperl, E., and Shadbolt, N. Motivations of citizen scientists: A quantitative investigation of forum participation. In Proceedings of the 2014 ACM Conference on Web Science, WebSci '14, ACM (New York, NY, USA, 2014), 295--296. Google ScholarDigital Library
- Vasilescu, B., Filkov, V., and Serebrenik, A. Stackoverflow and github: associations between software development and crowdsourced knowledge. In Social Computing (SocialCom), 2013 International Conference on, IEEE (2013), 188--195. Google ScholarDigital Library
- von Ahn, L., and Dabbish, L. Designing games with a purpose. Communications of the ACM 51, 8 (2008), 58--67. Google ScholarDigital Library
- Westphal, A. J., Butterworth, A. L., Snead, C. J., Craig, N., Anderson, D., Jones, S. M., Brownlee, D. E., Farnsworth, R., and Zolensky, M. E. Stardust@ home: a massively distributed public search for interstellar dust in the stardust interstellar dust collector. In 36th Annual Lunar and Planetary Science Conference, vol. 36 (2005), 1908.Google Scholar
- Wiggins, A., and Crowston, K. From conservation to crowdsourcing: A typology of citizen science. In System Sciences (HICSS), 2011 44th Hawaii International Conference on, IEEE (2011), 1--10. Google ScholarDigital Library
- Zichermann, G., and Cunningham, C. Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps, 1st ed. O'Reilly Media, Inc., 2011. Google ScholarDigital Library
- Zook, M., Graham, M., Shelton, T., and Gorman, S. Volunteered geographic information and crowdsourcing disaster relief: a case study of the haitian earthquake. World Medical & Health Policy 2, 2 (2010), 7--33.Google ScholarCross Ref
Index Terms
- Designing for Citizen Data Analysis: A Cross-Sectional Case Study of a Multi-Domain Citizen Science Platform
Recommendations
Co-designing technologies for citizen scientist engagement & saving species
OzCHI '17: Proceedings of the 29th Australian Conference on Computer-Human InteractionThis full-day workshop will explore how human computer interaction (HCI) design approaches can expand, diversify, and improve ways that members of the public engage with nature and science to help save species as citizen scientists. Prospective ...
User-centered design, activity-centered design, and goal-directed design: a review of three methods for designing web applications
SIGDOC '09: Proceedings of the 27th ACM international conference on Design of communicationWhen conducting research with users in order to design web applications, the practitioner has a variety of methods from which to choose. This paper examines three such methods'User-Centered Design (UCD), Goal-Directed Design (GDD), and Activity-Centered ...
Analysis of Reviews on Greek Municipalities to Improve Public Service Delivery and Citizen Satisfaction: A Tool for Co-creation and Co-design
PCI '22: Proceedings of the 26th Pan-Hellenic Conference on InformaticsThe plethora of digital means of expression available to the general public has provided citizens with the opportunity to share their views and opinions on government services online. Citizen (or customer) satisfaction is determined and influenced by a ...
Comments