End-User Development of Mashups with NaturalMash

https://doi.org/10.1016/j.jvlc.2013.12.004Get rights and content

Highlights

  • Enabling users to address part of their situational needs through ad-hoc mashups.

  • Finding the sweet-spot in the trade-off between expressive power and usability.

  • Introducing a novel hybrid end-user programming technique.

  • Proposing a comparative evaluation of the state-of-the-art mashup tools.

  • Proposing a classification of the state-of-the-art mashup tools.

Abstract

Context: The emergence of the long-tail in the market of software applications is shifting the role of end-users from mere consumers to becoming developers of applications addressing their unique, personal, and transient needs. On the Web, a popular form of such applications is called mashup, built out of the lightweight composition of Web APIs (reusable software components delivered as a service through the Web). To enable end-users to build mashups, there is a key problem that must be overcome: End-users lack programming knowledge as well as the interest to learn how to master the complex set of Web technologies required to develop mashups. End-User Development (EUD) is an emerging research field dealing with this type of problems. Its main goal is to design tools and techniques facilitating the development of software applications by non-programmers.

Objective: The paper describes the design and evaluation of NaturalMash, an innovative EUD tool for mashups (a mashup tool). NaturalMash aims at enabling non-professional users without any knowledge of programming languages and skills to create feature-rich, interactive, and useful mashups.

Methods: The design of NaturalMash adopts a formative evaluation approach, and has completed three design and evaluation iterations. The formative evaluations utilize usability testing, think aloud protocol, questionnaires, observation, and unstructured interviews. Additionally, we compare the expressive power of naturalmash with the state-of-the-art mashup tools.

Results: The results from the formative evaluations helped us identify important usability problems. From an assessment point of view, the results were promising and sggested that the proposed tool has a short and gentle learning curve in a way that even non-programmers are able to rapidly build useful mashups. Also, the comparative evaluation results showed that NaturalMash offers a competitive level of expressive power compared with existing mashup tools targeting non-programmers.

Conclusion: As the evaluation results indicate, NaturalMash provides a high level of expressive power while it is still highly usable by non-programmers. These suggest that we have successfully achieved the objective of the proposed tool, distinguishing it from existing mashup tools that are either too limited or highly specialized for non-professional users.

Introduction

With the proliferation of Web APIs (i.e., reusable software components published on the Web), the Web [1] has become a highly programmable platform. A lightweight form of Web applications that is widely developed and used on this platform is called mashup. Mashups are usually built by users themselves by composing different Web APIs in an ad hoc fashion [2]. As a result, they provide users with the opportunity of rapidly satisfying their situational needs in various domains of application [3], [4], ranging from daily utilities of Web users to specialized domains, such as e-learning [5], bioinformatics [6], health care [7], emergency management [8] and enterprise integration [3].

In spite of the growing demand for mashups, their development barriers (e.g., knowing how to code in Web scripting languages like PHP and JavaScript, understanding Web API protocols such as HTTP) can hinder their proliferation. This is due to the fact that the dominant type of mashup users in various application domains are those with little or no knowledge in programming and related technologies. In order to cope with this challenge, therefore, these non-professional users need to be empowered to create mashups. End-User Development (EUD) [9], [10] is a research area that is committed to address this type of problems. Research and development in EUD for mashups have resulted in the emergence of dedicated mashup tools [11] that provide end-users with an intuitive composition language and environment for on-the-fly and code-free development of mashups.

In this paper, we present in detail the design and the evaluation of an innovative mashup tool called NaturalMash. NaturalMash provides adequate expressive power to create non-trivial, feature-rich, and interactive mashups out of the composition of Web APIs provided through different technologies (ranging from REST and SOAP services to JavaScript and HTML5 widgets). NaturalMash is designed to be usable by non-professional users by ensuring that it is easy to understand and easy to learn with a gently sloped learning curve (thanks to a highly interactive, live programming environment, featuring immediate feedback and autocompletion). Many mashup tools with the same level of expressive power (e.g., IBM Mashup Center (http://www.ibm.com/software/info/mashup-center), and JackBe Presto (http://www.jackbe.com/) are, however, designed in a way that is too specialized for non-professional users. On the other hand, mashup tools explicitly targeting non-professional users, such as IFTTT (https://ifttt.com) and ServFace Builder [12], do not provide adequate expressive power to freely compose any type of Web APIs.

This paper also contributes a novel, hybrid end-user programming technique [13] based on natural language programming [14], live programming, WYSIWYG [15] (What You See Is What You Get), and Programming by Demonstration [16] (PbD). NaturalMash is one of the first live mashup tools [17] that combines natural language processing techniques [18] with model-driven Web engineering [19] in order to provide immediate feedback to the users and show them the resulting mashup as they are typing up its recipe. NaturalMash was first introduced in [20]. This paper includes additional material describing our user-centric design approach with the complete history of its formative evaluations, an extensive comparison with related approaches, as well as additional usage examples to demonstrate its expressive power and information on the internal architecture of NaturalMash.

A formative user-centered design approach enabled us to collect early feedback on the system by two groups of users differing in their computer science knowledge: programmers and non-programmers. This approach helped us better focus the design and avoid gaps between the user expectations and the delivered system. As of yet we have completed three iterations of design and evaluation. Initial findings from the evaluations indicate that users with little or no programming experience can become productive and successfully build useful mashups, confirming the validity of some of the design decisions behind NaturalMash.

The rest of the paper is organized as follows. Section 2 presents the goals, requirements, and rationale behind the design of NaturalMash. We explain our approach to use natural language programming for mashup development in Section 3. Sections 4 and 5 thoroughly describe, respectively, the graphical user interface environment and the architecture of NaturalMash. Section 6 reports on the formative evaluation (second iteration) of the system and discusses the impact of users′ feedback in terms of usability assessment and suggested areas to improve. In Section 7 we compare NaturalMash against the state-of-the-art mashup tools in terms of their expressive power and the chosen end-user programming techniques. We provide a comprehensive discussion – summarizing the lessons learned in form of design guidelines – of the evaluation and comparison results in Section 8. We draw the conclusions in Section 9.

Section snippets

Design goals, requirements, and decisions

One of the main challenges in designing mashup tools consists of balancing the trade-off between the tool expressive power and the assumed user skills [21]. In addressing this challenge, the design of NaturalMash seeks to empower non-professional users (i.e., those who do not know programming) to rapidly create useful and feature-rich mashups with minimal prior learning. To achieve this, we tailored our design to meet three requirements:

  • (i)

    a high degree of usability by non-professional users (R1),

  • (ii)

NaturalMash controlled natural language

The NaturalMash CNL is an abstract, executable language for modeling the presentation integration, process integration, and data integration layers of mashups. Before describing its syntax and semantics we introduce the language with a few examples.

Listing 1 is the recipe (executable text written in the CNL) of a mashup that searches Slideshare (it is a Website for sharing and finding presentations and documents, http://www.slideshare.net/developers) for a topic or event (in this example “

NaturalMash composition environment

The NaturalMash environment is designed to provide an innovative selection of features that are meant to enhance the user experience and the ability of users to build sophisticated mashups. The design of the environment has been evolved over two years, as a result of a formative user-centered process. In this section, we consider the current version of the environment as this paper is being written. We postpone the details of the evolution and evaluation of the environment to Section 6. In the

Architecture

NaturalMash is designed as a live mashup tool, which completely automates the repetitive task of compiling, deploying, and running mashup recipes. Considering that mashups are compositions of remote and distributed Web APIs, it is rather technically challenging to comply with the requirements of liveness, as the changes made by the user to the mashup design must be reflected in the result of the mashup execution with minimal delay.

We present the client/server architecture of NaturalMash (Fig. 5

Formative evaluation

As it was mentioned earlier in this paper, NaturalMash evolved over the past two years following a formative user-centered design approach [31], which proposes an iterative and incremental process for design and development of software systems. In the process, each iteration cycle consists of design, implementation, and formative evaluation. The evaluation is conducted at the end of each iteration to inform the next iteration and ensure that users were kept central in the design so as to avoid

Related work

In recent years a number of mashup tools have been designed in both academia and industry. In this section, we give an overview and comparison of the state-of-the-art mashup tools in terms of the level of expressive power they offer as well as the end-user programming techniques they utilize. We also review the related works in natural language programming to put our approach in context.

Discussion

From a technical perspective, one of the main tasks of mashup tools is to hide the heterogeneity and complexity of Web technologies behind an easy-to-understand abstraction. From a user modeling perspective, the challenge lies in the broad diversity of user skills that need to be targeted and in the large number of domains in which mashups can be applied to. The evaluations presented in this paper (iteration formative evaluations in Section 6 and comparative expressive power evaluation in

Conclusion

In this paper we presented NaturalMash, a “natural” tool for end-user mashup development. NaturalMash is based on a novel hybrid composition technique combining a controlled natural language tuned for mashup development with an interactive WYSIWYG and drag-and-drop interface allowing PbD and live execution preview and modification of the resulting mashup user interface. The design of NaturalMash has adopted an incremental, user-driven approach in which iterative formative evaluations inform the

Acknowledgements

We are grateful for the support and the expertise of Monica Landoni and Antonella De Angeli with the preparations and the analysis of the formative evaluation results. This work is partially supported by the Swiss National Science Foundation with the SOSOA project (SINERGIA grant nr. CRSI22 127386).

References (76)

  • C. Goble et al.

    State of the nation in data integration for bioinformatics

    J. Biomed. Inform.

    (2008)
  • S.L. Tanimoto

    VIVAa visual language for image processing

    J. Vis. Lang. Comput.

    (1990)
  • C. Pautasso et al.

    The JOpera visual composition language

    J. Vis. Lang. Comput.

    (2005)
  • B.A. Myers

    Taxonomies of visual programming and program visualization

    J. Vis. Lang. Comput.

    (1990)
  • T. O′Reilly

    What is Web 2.0design patterns and business models for the next generation of software

    Commun. Strateg.

    (2007)
  • D. Benslimane et al.

    Services mashups: the new generation of web applications

    IEEE Internet Comput.

    (2008)
  • A. Jhingran, Enterprise information mashups: integrating information, simply, in: Proceedings of the 32nd International...
  • C. Anderson, The Long tail: why the future of business is selling less of more, Hyperion,...
  • M. Eisenstadt, Does elearning have to be so awful? (time to mashup or shutup), in: Proceedings of the 7th IEEE...
  • M.N. Kamel Boulos et al.

    The emerging web 2.0 social software: an enabling suite of sociable technologies in health and health care education1

    Health Inf. Libr. J.

    (2007)
  • A. Bellucci, A. Malizia, P. Diaz, I. Aedo, Framing the design space for novel crisis-related mashups: the estorys...
  • H. Lieberman, F. Paternò, M. Klann, V. Wulf, End-user development: an emerging paradigm, in: End User Development,...
  • F. Casati, How end-user development will save composition technologies from their continuing failures, in: End-User...
  • S. Aghaee, M. Nowak, C. Pautasso, Reusable decision space for mashup tool design, in: Proceedings of the 4th ACM SIGCHI...
  • T. Nestler, M. Feldmann, G. Hübsch, A. Preußner, U. Jugel, The servface builder—a WYSIWYG approach for building...
  • B.A. Nardi

    A Small Matter of ProgrammingPerspectives on End User Computing

    (1993)
  • L.A. Miller

    Natural language programmingstyles, strategies, and contrasts

    IBM Syst. J.

    (1981)
  • J. Rode, M.B. Rosson, Programming at runtime: requirements and paradigms for nonprogrammer web application development,...
  • A. Cypher et al.

    Watch What I DoProgramming by Demonstration

    (1993)
  • S. Aghaee, C. Pautasso, Live mashup tools: challenges and opportunities, in: Proceedings of the 1st International...
  • R. Mihalcea, H. Liu, H. Lieberman, NLP (natural language processing) for NLP (natural language programming), in:...
  • S. Casteleyn et al.

    Engineering Web Applications

    (2009)
  • S. Aghaee, C. Pautasso, A. De Angeli, Natural end-user development of mashups, in: Proceedings of the IEEE Symposium on...
  • S. Aghaee, C. Pautasso, End-user programming for web mashups: open research challenges, in: Proceedings of the 11th...
  • A. Namoun, T. Nestler, A. De Angeli, Service composition for non-programmers: prospects, problems, and design...
  • N. Collins et al.

    Live coding in laptop performance

    Org. Sound

    (2003)
  • D.A. Norman, S.W. Draper, User Centered System Design: New Perspectives on Human–Computer Interaction, L. Erlbaum...
  • A. Repenning, A. Ioannidou, What makes end-user development tick? 13 design guidelines, in: End User Development,...
  • G. Bergmann et al.

    Change-driven model transformations

    Softw. Syst. Model.

    (2012)
  • A. Strunk, QoS-aware service composition: a survey, in: Proceedings of the 8th IEEE European Conference on Web Services...
  • P. Lubbers, B. Albers, Harnessing the power of HTML5 web sockets to create scalable real-time applications...
  • K. Vredenburg, J.-Y. Mao, P.W. Smith, T. Carey, A survey of user-centered design practice, in: Proceedings of the...
  • G. Wang, S. Yang, Y. Han, Mashroom: end-user mashup programming using nested tables, in: Proceedings of the 18th...
  • R. Tuchinda et al.

    Building mashups by demonstration

    ACM Trans. Web (TWEB)

    (2011)
  • R. Ennals et al.

    Intel mash makerjoin the web

    ACM SIGMOD Record

    (2007)
  • J. Lin, J. Wong, J. Nichols, A. Cypher, T.A. Lau, End-user programming of mashups with vegemite, in: Proceedings of the...
  • J. Wong, J.I. Hong, Making mashups with marmite: towards end-user programming for the web, in: Proceedings of the...
  • F. Daniel, F. Casati, B. Benatallah, M.-C. Shan, Hosted universal composition: models, languages and infrastructure in...
  • Cited by (42)

    • End-user development for personalizing applications, things, and robots

      2019, International Journal of Human Computer Studies
      Citation Excerpt :

      In this area one approach often considered is the mashup approach characterized by the possibility of creating new applications by interactively composing components from existing applications. NaturalMash (Aghaee and Pautasso, 2014) is a Web-based environment that allows non-programmers to exploit existing Web resources by combining their input/output. NaturalMash users start defining a mashup by picking ingredients from a toolbar that includes services/contents available through Web APIs.

    • How end-user programmers debug visual web-based programs: An information foraging theory perspective

      2019, Journal of Computer Languages
      Citation Excerpt :

      Gross et al. created a mashup model consisting of the integration layer, presentation layer, and UI components [26]. Aghaee et al. created a mashup a high level of expressive mashup environment, NaturalMash, that allows end-user programmers to create mashups [4]. Dinmore and Boylls [18] empirically studied end-user programming behaviors in the Yahoo! Pipes environment.

    • Exploring spatially-aware cross-device interaction techniques for mobile collaborative sensemaking

      2019, International Journal of Human Computer Studies
      Citation Excerpt :

      A completely different approach has been implemented in DashMash, a Web mashup platform that, by means of an event-driven paradigm and without distinction between editing and execution time, allows to create and synchronize Web services (Cappiello et al., 2011). Another composition paradigm has been introduced in NaturalMash, a tool that allows users to indicate, using a natural language, services they want into their application and how to orchestrate them (Aghaee and Pautasso, 2014). In order to ensure the accuracy of queries, NaturalMash constraints the user to using a limited vocabulary and grammar.

    • Usability driven DSL development with USE-ME

      2018, Computer Languages, Systems and Structures
      Citation Excerpt :

      This study contributes further by putting into evidence the usefulness of Natural Programming design process, which applies principles of UCD [122]. Aghee and Pautasso [123] iteratively evaluated a DSL for developing of Mashups using formative evaluations. Angelini et al. [124] presents an innovative visualisation environment, which eases and makes more effective the experimental assessment process.

    View all citing articles on Scopus

    This paper has been recommended for acceptance by Shi Kho Chang.

    View full text