Abstract
We describe an expert evaluation for user requirement elicitation of an annotation system - The Digital Library Annotation Service, DiLAS, that facilitates collaborative information access and sharing. An analytical evaluation was conducted as a Participatory Group Evaluation, which involved presentation beyond the written papers of the objectives and rationale behind the development of the prototype. The empirical evaluation of DiLAS consisted of two experiments. The first evaluation experiment was a bottom up evaluation of the usability of the interface using a qualitative approach. The second stage of our evaluation moved towards a broader work context with a User and Work Centred Evaluation involving an entire, collaborative task situation, which required knowledge sharing on a common real life work task. This paper describes a first evaluation stage in an iterative evaluation process, and the preliminary result is a set of requirements that will inform the next stage of the DiLAS.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Pejtersen, A.M., Hansen, P., Albrechtsen, H.: A Work Centered Approach to Evaluation of a DL Annotation System for Knowledge Sharing in Collaborative Work. In: ICDL. Proceedings of the International Conference on Digital Libraries, December 5-8, 2006, vol. I, pp. 78–90. TERI, New Delhi (2006)
Agosti, M., Ferro, N.: Annotations as Context for Searching Documents. In: Crestani, F., Ruthven, I. (eds.) CoLIS 2005. LNCS, vol. 3507, pp. 155–170. Springer, Heidelberg (2005)
Marshall, C., Brush, A.J.: Exploring the relationship between personal and public annotations. In: Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries, Tucson, AZ, USA, pp. 349–357. ACM Press, New York (2004)
Klas, C.-P., Fuhr, N., Schaefer, A.: Evaluating Strategic Support for Information Access in the DAFFODIL System. In: Heery, R., Lyon, L. (eds.) ECDL 2004. LNCS, vol. 3232, pp. 476–487. Springer, Heidelberg (2004)
Shipman, F., Hsieh, H., Moore, M., Zacchi, A.: Supporting personal collections across digital libraries in spatial hypertext. In: Proceedings of the 4th ACM/IEEE-CS joint conference on Digital libraries, Tucson, AZ, USA, pp. 358–367. ACM Press, Tucson (2004)
Golovchinsky, G., Price, M.N., Schilit, B.: From reading to retrieval: freeform ink annotations as queries. In: Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, Berkeley, California, United States, pp. 19 – 25 (1999)
Cleal, B.R., Andersen, H.H.K., Albrechtsen, H.: Collaboration, communication and categorical complexity: A case study in collaboratory evaluation. J. Digital Inf. Manag. 2(1), 13–20 (2004)
Frommholz, I., Brocks, H., Thiel, U., Neuhold, E., Iannone, L., Semeraro, G., Berardi, M., Ceci, M.: Document-Centered Collaboration for Scholars in the Humanities - The COLLATE System. In: Koch, T., Sølvberg, I.T. (eds.) ECDL 2003. LNCS, vol. 2769, pp. 434–445. Springer, Heidelberg (2003)
Pejtersen, A.M., Albrechtsen, H., Cleal, B., Hansen, C.B., Hertzum, M.: A Web-based Multimedia Collaboratory. Empirical Work Studies in Film Archives. Roskilde: Risø National Laboratory (2001)
Hansen, P., Järvelin, K.: Collaborative Information Searching in an Information-Intensive Work Domain: Preliminary Results. Journal of Digital Information Management 2(1), 26–30 (2004)
Hansen, P., Järvelin, K.: Collaborative Information Retrieval in an information-intensive domain. Information Processing & Management 41(5), 1101–1119 (2005)
Marshall, C., Bly, S.: Sharing encountered information: digital libraries get a social life. In: Proceedings of the 4th ACM/IEEE- joint conference on Digital libraries, Tuscon, AZ, USA, pp. 218–227. ACM Press, New York (2004)
O’Day, V., Jeffries, R.: Information artisans: Patterns of Result Sharing by Information Searchers. In: COOCS 1993. Proceedings of the ACM Conference on Organizational Computing Systems, November 1-4, 1993, pp. 98–107. ACM Press, Milpitas, CA, New York (1993)
Hertzum, M.: People as Carriers of Experience and Sources of Commitment: Information Seeking in a Software Design Project. New Review of Information Behaviour Research 1, 135–149 (2000)
Nielsen, J., Mack, R.: Usability Inspection Methods. Wiley, New York (1994)
Lewis, Wharton: Cognitive walkthroughs: a method for theory-based evaluation of user interfaces. Journal of Man-Machine Studies 36(5), 741–773 (1997)
Carroll, J.M.: Scenario-based Design. In: Envisioning Work and Technology in System Development, Wiley, New York (1995)
Greebaum, J., Kyng, M.: Design at Work: Cooperative Design of Computer Systems. Lawrence Erlbaum Associates, Hillsdale, N.J (1991)
Spinuzzi, C.: The Methodology of Participatory Design. Technical Communication 52(2), 163–174 (2005)
Winograd, T.: Bringing Design to Software. Addison-Wesley, Reading (1996)
Rasmussen, J., Pejtersen, A.M., Goodstein, L.P.: Cognitive Systems Engineering. John Wiley & Sons, New York (1994)
Bottoni, P., Civica, R., Levialdi, S., Orso, L., Panizzi, E., Trinchese, R.: MADCOW: a Multimedia Digital Annotation System. In: AVI 2004. Proc. Working Conference on Advanced Visual Interfaces, pp. 55–62. ACM Press, New York (2004)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Hansen, P., Pejtersen, A.M., Albrechtsen, H. (2007). Evaluation and Requirements Elicitation of a DL Annotation System for Collaborative Information Sharing. In: Thanos, C., Borri, F., Candela, L. (eds) Digital Libraries: Research and Development. DELOS 2007. Lecture Notes in Computer Science, vol 4877. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77088-6_17
Download citation
DOI: https://doi.org/10.1007/978-3-540-77088-6_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77087-9
Online ISBN: 978-3-540-77088-6
eBook Packages: Computer ScienceComputer Science (R0)